Compare commits
18 Commits
autoparser
...
master
Author | SHA1 | Date |
---|---|---|
Greg Shuflin | 57a0d0a603 | |
Greg Shuflin | b44fda3283 | |
Greg Shuflin | 1be26eb453 | |
Greg Shuflin | 7067edc86f | |
Greg Shuflin | dc771fc7ad | |
Greg Shuflin | 45c4d08fb9 | |
Greg Shuflin | 77257d0eb7 | |
Greg Shuflin | f33195ab28 | |
Greg Shuflin | 8cde20641b | |
Greg Shuflin | ba4ccfe6bf | |
Greg Shuflin | 7bc92aef97 | |
Greg Shuflin | 95e22567e7 | |
Greg Shuflin | dc09d804ef | |
Greg Shuflin | 49e6e3a71d | |
Greg Shuflin | cf7a2ff9ba | |
Greg Shuflin | aff809e4ce | |
Greg Shuflin | 18b4ac0d4b | |
Greg Shuflin | ab53cfdb7d |
|
@ -1,4 +1,3 @@
|
|||
Cargo.lock
|
||||
target
|
||||
.schala_repl
|
||||
.schala_history
|
||||
node_modules/
|
||||
experiments/tree-sitter-test/src
|
||||
|
|
|
@ -0,0 +1,15 @@
|
|||
# This file is automatically @generated by Cargo.
|
||||
# It is not intended for manual editing.
|
||||
version = 3
|
||||
|
||||
[[package]]
|
||||
name = "experiments"
|
||||
version = "0.1.0"
|
||||
|
||||
[[package]]
|
||||
name = "schala-main"
|
||||
version = "0.1.0"
|
||||
|
||||
[[package]]
|
||||
name = "schala-parser"
|
||||
version = "0.1.0"
|
25
Cargo.toml
25
Cargo.toml
|
@ -1,20 +1,7 @@
|
|||
[package]
|
||||
name = "schala"
|
||||
version = "0.1.0"
|
||||
authors = ["greg <greg.shuflin@protonmail.com>"]
|
||||
|
||||
[dependencies]
|
||||
|
||||
llvm-sys = "*"
|
||||
take_mut = "0.1.3"
|
||||
itertools = "0.5.8"
|
||||
lazy_static = "0.2.8"
|
||||
maplit = "*"
|
||||
colored = "1.5"
|
||||
|
||||
schala-lib = { path = "schala-lib" }
|
||||
|
||||
[build-dependencies]
|
||||
includedir_codegen = "0.2.0"
|
||||
|
||||
[workspace]
|
||||
members = [
|
||||
"schala-main",
|
||||
"schala-parser",
|
||||
"experiments",
|
||||
]
|
||||
resolver = "2"
|
||||
|
|
31
Grammar
31
Grammar
|
@ -1,31 +0,0 @@
|
|||
|
||||
|
||||
<program> := <statements> EOF
|
||||
|
||||
<statements> := <statement>
|
||||
| <statement> SEP <statements>
|
||||
|
||||
<statement> := let <id> = <expr>
|
||||
| <expr>
|
||||
| <fn_block>
|
||||
|
||||
<fn_block> := fn <id> ( <arg_list> ) <statements> end
|
||||
|
||||
<arg_list> := e
|
||||
| <id>
|
||||
| <id> , <arg_list>
|
||||
|
||||
<expr> := if <expr> then <statements> end
|
||||
| if <expr> then <statements> else <statements> end
|
||||
| while <expr> SEP <statements> end
|
||||
| ( <expr> )
|
||||
| <binop>
|
||||
|
||||
<binop> := <simple_expr>
|
||||
| <simple_expr> <id> <binop>
|
||||
|
||||
<simple_expr> := <id>
|
||||
| <number>
|
||||
| <string>
|
||||
|
||||
|
79
README.md
79
README.md
|
@ -1,77 +1,4 @@
|
|||
# Schala - A Programming Language Implementation
|
||||
|
||||
# Schala - a programming language meta-interpreter
|
||||
|
||||
Schala is a Rust framework written to make it easy to
|
||||
create and experiment with toy programming languages. It provides
|
||||
a common REPL, and a trait `ProgrammingLanguage` with provisions
|
||||
for tokenizing text, parsing tokens, evaluating an abstract syntax tree,
|
||||
and other tasks that are common to all programming languages.
|
||||
|
||||
Schala is implemented as a Rust library `schala_lib`, which provides a
|
||||
`schala_main` function. This function serves as the main loop of the REPL, if run
|
||||
interactively, or otherwise reads and interprets programming language source
|
||||
files. It expects as input a vector of `PLIGenerator`, which is a type representing
|
||||
a closure that returns a boxed trait object that implements the `ProgrammingLanguage` trait,
|
||||
and stores any persistent state relevant to that programming language. The ability
|
||||
to share state between different programming languages is in the works.
|
||||
|
||||
## About
|
||||
|
||||
Schala started out life as an experiment in writing a Javascript-like
|
||||
programming language that would never encounter any kind of runtime value
|
||||
error, but rather always return `null` under any kind of error condition. I had
|
||||
seen one too many Javascript `Uncaught TypeError: Cannot read property ___ of
|
||||
undefined` messages, and I was a bit frustrated. Plus I had always wanted to
|
||||
write a programming langauge from scratch, and Rust is a fun language to
|
||||
program in. Over time I became interested in playing around with other sorts
|
||||
of programming languages as well, and wanted to make the process as general as
|
||||
possible.
|
||||
|
||||
The name of the project comes from Schala the Princess of Zeal from the 1995
|
||||
SNES RPG *Chrono Trigger*. I like classic JRPGs and enjoyed the thought of
|
||||
creating a language name confusingly close to Scala. The naming scheme for
|
||||
languages implemented with the Schala meta-interpreter is Chrono Trigger
|
||||
characters.
|
||||
|
||||
Schala is incomplete alpha software and is not ready for public release.
|
||||
|
||||
## Languages implemented using the meta-interpreter
|
||||
|
||||
* The eponymous *Schala* language is an interpreted/compiled scripting langauge,
|
||||
designed to be relatively simple, but with a reasonably sophisticated type
|
||||
system.
|
||||
|
||||
* *Maaru* was the original Schala (since renamed to free up the name *Schala*
|
||||
for the above language), a very simple dynamically-typed scripting language
|
||||
such that all possible runtime errors result in null rather than program
|
||||
failure.
|
||||
|
||||
* *Robo* is an experiment in creating a lazy, functional, strongly-typed language
|
||||
much like Haskell
|
||||
|
||||
* *Rukka* is a straightforward LISP implementation
|
||||
|
||||
## Reference works
|
||||
|
||||
Here's a partial list of resources I've made use of in the process
|
||||
of learning how to write a programming language.
|
||||
|
||||
### Type-checking
|
||||
https://skillsmatter.com/skillscasts/10868-inside-the-rust-compiler
|
||||
|
||||
### Evaluation
|
||||
*Understanding Computation*, Tom Stuart, O'Reilly 2013
|
||||
|
||||
*Basics of Compiler Design*, Torben Mogensen
|
||||
|
||||
### Parsing
|
||||
http://journal.stuffwithstuff.com/2011/03/19/pratt-parsers-expression-parsing-made-easy/
|
||||
|
||||
[Crafting Interpreters](http://www.craftinginterpreters.com/)
|
||||
|
||||
### LLVM
|
||||
http://blog.ulysse.io/2016/07/03/llvm-getting-started.html
|
||||
|
||||
###Rust resources
|
||||
https://thefullsnack.com/en/rust-for-the-web.html
|
||||
https://rocket.rs/guide/getting-started/
|
||||
`schala` is an implementation of a yet-unnamed quasi-functional programming
|
||||
language.
|
||||
|
|
46
TODO.md
46
TODO.md
|
@ -1,46 +0,0 @@
|
|||
|
||||
# TODO Items
|
||||
|
||||
* Share state between programming languages
|
||||
|
||||
* idea for Schala - scoped types - be able to define a quick enum type scoped to a function ro something, that only is meant to be used as a quick bespoke interface between two other things
|
||||
|
||||
* another idea, allow:
|
||||
type enum {
|
||||
type enum MySubVariant {
|
||||
SubVariant1, SubVariant2, etc.
|
||||
}
|
||||
Variant1(MySubVariant),
|
||||
Variant2(...),
|
||||
}
|
||||
|
||||
|
||||
|
||||
* idea for Schala: both currying *and* default arguments!
|
||||
ex. fn a(b: Int, c:Int, d:Int = 1) -> Int
|
||||
a(1,2) : Int
|
||||
a(1,2,d=2): Int
|
||||
a(_,1,3) : Int -> Int
|
||||
a(1,2, c=_): Int -> Int
|
||||
a(_,_,_) : Int -> Int -> Int -> Int
|
||||
|
||||
|
||||
|
||||
- AST : maybe replace the Expression type with "Ascription(TypeName, Box<Expression>) nodes??
|
||||
- parser: add a "debug" field to the Parser struct for all debug-related things
|
||||
|
||||
-scala-style html"dfasfsadf${}" string interpolations!
|
||||
|
||||
*Compiler passes architecture
|
||||
|
||||
-ProgrammingLanguageInterface defines a evaluate_in_repl() and evaluate_no_repl() functions
|
||||
-these take in a vec of CompilerPasses
|
||||
|
||||
struct CompilerPass {
|
||||
name: String,
|
||||
run: fn(PrevPass) -> NextPass
|
||||
}
|
||||
|
||||
-change "Type...." names in parser.rs to "Anno..." for non-collision with names in typechecking.rs
|
||||
|
||||
-get rid of code pertaining to compilation specifically, have a more generation notion of "execution type"
|
|
@ -0,0 +1,8 @@
|
|||
[package]
|
||||
name = "experiments"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
|
||||
|
||||
[dependencies]
|
|
@ -0,0 +1,3 @@
|
|||
fn main() {
|
||||
println!("Hello, world!");
|
||||
}
|
|
@ -0,0 +1,48 @@
|
|||
module.exports = grammar({
|
||||
name: "TestLang",
|
||||
rules: {
|
||||
source_file: $ => repeat($._definition),
|
||||
|
||||
_definition: $ => choice(
|
||||
$.function_definition
|
||||
//TODO others
|
||||
),
|
||||
|
||||
function_definition: $ => seq(
|
||||
'fn',
|
||||
$.identifier,
|
||||
$.parameter_list,
|
||||
field("return_type", optional($._type)),
|
||||
$.block,
|
||||
),
|
||||
parameter_list: $ => seq("(", /* TODO */ ")"),
|
||||
|
||||
block: $ => seq(
|
||||
"{",
|
||||
choice(
|
||||
repeat($._statement),
|
||||
"",
|
||||
),
|
||||
"}"
|
||||
),
|
||||
|
||||
_statement: $ => choice(
|
||||
$._return_statement
|
||||
),
|
||||
|
||||
_return_statement: $ => seq("return", $._expression, ";"),
|
||||
|
||||
_expression: $ => choice($.identifier, $.unary, $.binary),
|
||||
|
||||
unary: $ => prec(4, choice(seq("-", $._expression), seq("!", $._expression))),
|
||||
binary: $ => choice(prec.left(2, seq($._expression, "*", $._expression)), prec.left(1, seq($._expression, "+", $._expression))),
|
||||
|
||||
_type: $ => "bool",
|
||||
_type: $ => choice(
|
||||
$.primitive_type,
|
||||
),
|
||||
primitive_type: $ => choice("bool", "int"),
|
||||
|
||||
identifier: $ => /[a-z]+/,
|
||||
}
|
||||
});
|
|
@ -0,0 +1,8 @@
|
|||
_default:
|
||||
just --list
|
||||
|
||||
# Test out the grammar
|
||||
test-grammar:
|
||||
#!/usr/bin/env bash
|
||||
tree-sitter generate
|
||||
tree-sitter test
|
|
@ -0,0 +1,380 @@
|
|||
{
|
||||
"name": "tree-sitter-test",
|
||||
"version": "1.0.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "tree-sitter-test",
|
||||
"version": "1.0.0",
|
||||
"hasInstallScript": true,
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"node-addon-api": "^7.1.0",
|
||||
"node-gyp-build": "^4.8.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"prebuildify": "^6.0.0",
|
||||
"tree-sitter-cli": "^0.22.5"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"tree-sitter": "^0.21.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"tree_sitter": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/base64-js": {
|
||||
"version": "1.5.1",
|
||||
"resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
|
||||
"integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",
|
||||
"dev": true,
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
]
|
||||
},
|
||||
"node_modules/bl": {
|
||||
"version": "4.1.0",
|
||||
"resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",
|
||||
"integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"buffer": "^5.5.0",
|
||||
"inherits": "^2.0.4",
|
||||
"readable-stream": "^3.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/buffer": {
|
||||
"version": "5.7.1",
|
||||
"resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
|
||||
"integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
|
||||
"dev": true,
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
],
|
||||
"dependencies": {
|
||||
"base64-js": "^1.3.1",
|
||||
"ieee754": "^1.1.13"
|
||||
}
|
||||
},
|
||||
"node_modules/chownr": {
|
||||
"version": "1.1.4",
|
||||
"resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",
|
||||
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/end-of-stream": {
|
||||
"version": "1.4.4",
|
||||
"resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.4.tgz",
|
||||
"integrity": "sha512-+uw1inIHVPQoaVuHzRyXd21icM+cnt4CzD5rW+NC1wjOUSTOs+Te7FOv7AhN7vS9x/oIyhLP5PR1H+phQAHu5Q==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"once": "^1.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/fs-constants": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",
|
||||
"integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/ieee754": {
|
||||
"version": "1.2.1",
|
||||
"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
|
||||
"integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
|
||||
"dev": true,
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
]
|
||||
},
|
||||
"node_modules/inherits": {
|
||||
"version": "2.0.4",
|
||||
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
|
||||
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/lru-cache": {
|
||||
"version": "6.0.0",
|
||||
"resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-6.0.0.tgz",
|
||||
"integrity": "sha512-Jo6dJ04CmSjuznwJSS3pUeWmd/H0ffTlkXXgwZi+eq1UCmqQwCh+eLsYOYCwY991i2Fah4h1BEMCx4qThGbsiA==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"yallist": "^4.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/minimist": {
|
||||
"version": "1.2.8",
|
||||
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
|
||||
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
|
||||
"dev": true,
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/mkdirp-classic": {
|
||||
"version": "0.5.3",
|
||||
"resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
|
||||
"integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/node-abi": {
|
||||
"version": "3.60.0",
|
||||
"resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.60.0.tgz",
|
||||
"integrity": "sha512-zcGgwoXbzw9NczqbGzAWL/ToDYAxv1V8gL1D67ClbdkIfeeDBbY0GelZtC25ayLvVjr2q2cloHeQV1R0QAWqRQ==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"semver": "^7.3.5"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/node-addon-api": {
|
||||
"version": "7.1.0",
|
||||
"resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-7.1.0.tgz",
|
||||
"integrity": "sha512-mNcltoe1R8o7STTegSOHdnJNN7s5EUvhoS7ShnTHDyOSd+8H+UdWODq6qSv67PjC8Zc5JRT8+oLAMCr0SIXw7g==",
|
||||
"engines": {
|
||||
"node": "^16 || ^18 || >= 20"
|
||||
}
|
||||
},
|
||||
"node_modules/node-gyp-build": {
|
||||
"version": "4.8.0",
|
||||
"resolved": "https://registry.npmjs.org/node-gyp-build/-/node-gyp-build-4.8.0.tgz",
|
||||
"integrity": "sha512-u6fs2AEUljNho3EYTJNBfImO5QTo/J/1Etd+NVdCj7qWKUSN/bSLkZwhDv7I+w/MSC6qJ4cknepkAYykDdK8og==",
|
||||
"bin": {
|
||||
"node-gyp-build": "bin.js",
|
||||
"node-gyp-build-optional": "optional.js",
|
||||
"node-gyp-build-test": "build-test.js"
|
||||
}
|
||||
},
|
||||
"node_modules/npm-run-path": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/npm-run-path/-/npm-run-path-3.1.0.tgz",
|
||||
"integrity": "sha512-Dbl4A/VfiVGLgQv29URL9xshU8XDY1GeLy+fsaZ1AA8JDSfjvr5P5+pzRbWqRSBxk6/DW7MIh8lTM/PaGnP2kg==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"path-key": "^3.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/once": {
|
||||
"version": "1.4.0",
|
||||
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
|
||||
"integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"wrappy": "1"
|
||||
}
|
||||
},
|
||||
"node_modules/path-key": {
|
||||
"version": "3.1.1",
|
||||
"resolved": "https://registry.npmjs.org/path-key/-/path-key-3.1.1.tgz",
|
||||
"integrity": "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==",
|
||||
"dev": true,
|
||||
"engines": {
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/prebuildify": {
|
||||
"version": "6.0.1",
|
||||
"resolved": "https://registry.npmjs.org/prebuildify/-/prebuildify-6.0.1.tgz",
|
||||
"integrity": "sha512-8Y2oOOateom/s8dNBsGIcnm6AxPmLH4/nanQzL5lQMU+sC0CMhzARZHizwr36pUPLdvBnOkCNQzxg4djuFSgIw==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"minimist": "^1.2.5",
|
||||
"mkdirp-classic": "^0.5.3",
|
||||
"node-abi": "^3.3.0",
|
||||
"npm-run-path": "^3.1.0",
|
||||
"pump": "^3.0.0",
|
||||
"tar-fs": "^2.1.0"
|
||||
},
|
||||
"bin": {
|
||||
"prebuildify": "bin.js"
|
||||
}
|
||||
},
|
||||
"node_modules/pump": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/pump/-/pump-3.0.0.tgz",
|
||||
"integrity": "sha512-LwZy+p3SFs1Pytd/jYct4wpv49HiYCqd9Rlc5ZVdk0V+8Yzv6jR5Blk3TRmPL1ft69TxP0IMZGJ+WPFU2BFhww==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"end-of-stream": "^1.1.0",
|
||||
"once": "^1.3.1"
|
||||
}
|
||||
},
|
||||
"node_modules/readable-stream": {
|
||||
"version": "3.6.2",
|
||||
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
|
||||
"integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"inherits": "^2.0.3",
|
||||
"string_decoder": "^1.1.1",
|
||||
"util-deprecate": "^1.0.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 6"
|
||||
}
|
||||
},
|
||||
"node_modules/safe-buffer": {
|
||||
"version": "5.2.1",
|
||||
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
|
||||
"integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
|
||||
"dev": true,
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
]
|
||||
},
|
||||
"node_modules/semver": {
|
||||
"version": "7.6.0",
|
||||
"resolved": "https://registry.npmjs.org/semver/-/semver-7.6.0.tgz",
|
||||
"integrity": "sha512-EnwXhrlwXMk9gKu5/flx5sv/an57AkRplG3hTK68W7FRDN+k+OWBj65M7719OkA82XLBxrcX0KSHj+X5COhOVg==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"lru-cache": "^6.0.0"
|
||||
},
|
||||
"bin": {
|
||||
"semver": "bin/semver.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/string_decoder": {
|
||||
"version": "1.3.0",
|
||||
"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
|
||||
"integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"safe-buffer": "~5.2.0"
|
||||
}
|
||||
},
|
||||
"node_modules/tar-fs": {
|
||||
"version": "2.1.1",
|
||||
"resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.1.tgz",
|
||||
"integrity": "sha512-V0r2Y9scmbDRLCNex/+hYzvp/zyYjvFbHPNgVTKfQvVrb6guiE/fxP+XblDNR011utopbkex2nM4dHNV6GDsng==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"chownr": "^1.1.1",
|
||||
"mkdirp-classic": "^0.5.2",
|
||||
"pump": "^3.0.0",
|
||||
"tar-stream": "^2.1.4"
|
||||
}
|
||||
},
|
||||
"node_modules/tar-stream": {
|
||||
"version": "2.2.0",
|
||||
"resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",
|
||||
"integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"bl": "^4.0.3",
|
||||
"end-of-stream": "^1.4.1",
|
||||
"fs-constants": "^1.0.0",
|
||||
"inherits": "^2.0.3",
|
||||
"readable-stream": "^3.1.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/tree-sitter": {
|
||||
"version": "0.21.1",
|
||||
"resolved": "https://registry.npmjs.org/tree-sitter/-/tree-sitter-0.21.1.tgz",
|
||||
"integrity": "sha512-7dxoA6kYvtgWw80265MyqJlkRl4yawIjO7S5MigytjELkX43fV2WsAXzsNfO7sBpPPCF5Gp0+XzHk0DwLCq3xQ==",
|
||||
"hasInstallScript": true,
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"node-addon-api": "^8.0.0",
|
||||
"node-gyp-build": "^4.8.0"
|
||||
}
|
||||
},
|
||||
"node_modules/tree-sitter-cli": {
|
||||
"version": "0.22.5",
|
||||
"resolved": "https://registry.npmjs.org/tree-sitter-cli/-/tree-sitter-cli-0.22.5.tgz",
|
||||
"integrity": "sha512-c3VT46Bc3a6pEd0JAwufbqEw9Q2FRLDp5E230hGvnr+Hivw+Y6jyeP+3T89KDptvn48MOPVmbgaLm69xYgLVTw==",
|
||||
"dev": true,
|
||||
"hasInstallScript": true,
|
||||
"bin": {
|
||||
"tree-sitter": "cli.js"
|
||||
}
|
||||
},
|
||||
"node_modules/tree-sitter/node_modules/node-addon-api": {
|
||||
"version": "8.0.0",
|
||||
"resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-8.0.0.tgz",
|
||||
"integrity": "sha512-ipO7rsHEBqa9STO5C5T10fj732ml+5kLN1cAG8/jdHd56ldQeGj3Q7+scUS+VHK/qy1zLEwC4wMK5+yM0btPvw==",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": "^18 || ^20 || >= 21"
|
||||
}
|
||||
},
|
||||
"node_modules/util-deprecate": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
|
||||
"integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/wrappy": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
|
||||
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/yallist": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/yallist/-/yallist-4.0.0.tgz",
|
||||
"integrity": "sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==",
|
||||
"dev": true
|
||||
}
|
||||
}
|
||||
}
|
|
@ -0,0 +1,38 @@
|
|||
{
|
||||
"name": "tree-sitter-test",
|
||||
"version": "1.0.0",
|
||||
"main": "index.js",
|
||||
"types": "bindings/node",
|
||||
"scripts": {
|
||||
"test": "echo \"Error: no test specified\" && exit 1",
|
||||
"install": "node-gyp-build",
|
||||
"prebuildify": "prebuildify --napi --strip"
|
||||
},
|
||||
"author": "",
|
||||
"license": "ISC",
|
||||
"description": "",
|
||||
"dependencies": {
|
||||
"node-addon-api": "^7.1.0",
|
||||
"node-gyp-build": "^4.8.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"tree-sitter": "^0.21.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"tree_sitter": {
|
||||
"optional": true
|
||||
}
|
||||
},
|
||||
"devDependencies": {
|
||||
"prebuildify": "^6.0.0",
|
||||
"tree-sitter-cli": "^0.22.5"
|
||||
},
|
||||
"files": [
|
||||
"grammar.js",
|
||||
"binding.gyp",
|
||||
"prebuilds/**",
|
||||
"bindings/node/*",
|
||||
"queries/*",
|
||||
"src/**"
|
||||
]
|
||||
}
|
|
@ -0,0 +1,26 @@
|
|||
=============
|
||||
Initial test
|
||||
=============
|
||||
fn main() {
|
||||
|
||||
}
|
||||
----
|
||||
(source_file
|
||||
(function_definition
|
||||
(identifier)
|
||||
(parameter_list)
|
||||
(block)
|
||||
)
|
||||
)
|
||||
|
||||
====
|
||||
Another test
|
||||
====
|
||||
|
||||
fn yolo() bool { }
|
||||
|
||||
----
|
||||
|
||||
(source_file
|
||||
(function_definition
|
||||
(identifier) (parameter_list) (primitive_type) (block)))
|
|
@ -1,25 +0,0 @@
|
|||
[package]
|
||||
name = "schala-lib"
|
||||
version = "0.1.0"
|
||||
authors = ["greg <greg.shuflin@protonmail.com>"]
|
||||
|
||||
[dependencies]
|
||||
llvm-sys = "*"
|
||||
take_mut = "0.1.3"
|
||||
itertools = "0.5.8"
|
||||
getopts = "*"
|
||||
lazy_static = "0.2.8"
|
||||
maplit = "*"
|
||||
colored = "1.5"
|
||||
serde = "1.0.15"
|
||||
serde_derive = "1.0.15"
|
||||
serde_json = "1.0.3"
|
||||
rocket = "0.3.5"
|
||||
rocket_codegen = "0.3.5"
|
||||
rocket_contrib = "0.3.5"
|
||||
phf = "0.7.12"
|
||||
includedir = "0.2.0"
|
||||
rustyline = "1.0.0"
|
||||
|
||||
[build-dependencies]
|
||||
includedir_codegen = "0.2.0"
|
|
@ -1,10 +0,0 @@
|
|||
extern crate includedir_codegen;
|
||||
|
||||
use includedir_codegen::Compression;
|
||||
|
||||
fn main() {
|
||||
includedir_codegen::start("WEBFILES")
|
||||
.dir("../static", Compression::Gzip)
|
||||
.build("static.rs")
|
||||
.unwrap();
|
||||
}
|
|
@ -1,106 +0,0 @@
|
|||
extern crate colored;
|
||||
|
||||
use self::colored::*;
|
||||
|
||||
pub struct LLVMCodeString(pub String);
|
||||
|
||||
#[derive(Debug, Default, Serialize, Deserialize)]
|
||||
pub struct EvalOptions {
|
||||
pub debug_tokens: bool,
|
||||
pub debug_parse: bool,
|
||||
pub debug_type: bool,
|
||||
pub debug_symbol_table: bool,
|
||||
pub show_llvm_ir: bool,
|
||||
pub trace_evaluation: bool,
|
||||
pub compile: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
pub struct LanguageOutput {
|
||||
output: String,
|
||||
artifacts: Vec<TraceArtifact>,
|
||||
failed: bool,
|
||||
}
|
||||
|
||||
impl LanguageOutput {
|
||||
pub fn add_artifact(&mut self, artifact: TraceArtifact) {
|
||||
self.artifacts.push(artifact);
|
||||
}
|
||||
pub fn add_output(&mut self, output: String) {
|
||||
self.output = output;
|
||||
}
|
||||
|
||||
pub fn to_string(&self) -> String {
|
||||
let mut acc = String::new();
|
||||
for line in self.artifacts.iter() {
|
||||
acc.push_str(&line.debug_output.color(line.text_color).to_string());
|
||||
acc.push_str(&"\n");
|
||||
}
|
||||
acc.push_str(&self.output);
|
||||
acc
|
||||
}
|
||||
|
||||
pub fn print_to_screen(&self) {
|
||||
for line in self.artifacts.iter() {
|
||||
let color = line.text_color;
|
||||
let stage = line.stage_name.color(color).to_string();
|
||||
let output = line.debug_output.color(color).to_string();
|
||||
println!("{}: {}", stage, output);
|
||||
}
|
||||
println!("{}", self.output);
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
//TODO I'll probably wanna implement this later
|
||||
#[derive(Debug)]
|
||||
pub struct CompilationOutput {
|
||||
output: LLVMCodeString,
|
||||
artifacts: Vec<TraceArtifact>,
|
||||
}
|
||||
*/
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct TraceArtifact {
|
||||
stage_name: String,
|
||||
debug_output: String,
|
||||
text_color: &'static str,
|
||||
}
|
||||
|
||||
impl TraceArtifact {
|
||||
pub fn new(stage: &str, debug: String) -> TraceArtifact {
|
||||
let color = match stage {
|
||||
"parse_trace" | "ast" => "red",
|
||||
"tokens" => "green",
|
||||
"type_check" => "magenta",
|
||||
_ => "blue",
|
||||
};
|
||||
TraceArtifact { stage_name: stage.to_string(), debug_output: debug, text_color: color}
|
||||
}
|
||||
|
||||
pub fn new_parse_trace(trace: Vec<String>) -> TraceArtifact {
|
||||
let mut output = String::new();
|
||||
|
||||
for t in trace {
|
||||
output.push_str(&t);
|
||||
output.push_str("\n");
|
||||
}
|
||||
|
||||
TraceArtifact { stage_name: "parse_trace".to_string(), debug_output: output, text_color: "red"}
|
||||
}
|
||||
}
|
||||
|
||||
pub trait ProgrammingLanguageInterface {
|
||||
fn evaluate_in_repl(&mut self, input: &str, eval_options: &EvalOptions) -> LanguageOutput;
|
||||
fn evaluate_noninteractive(&mut self, input: &str, eval_options: &EvalOptions) -> LanguageOutput {
|
||||
self.evaluate_in_repl(input, eval_options)
|
||||
}
|
||||
fn get_language_name(&self) -> String;
|
||||
fn get_source_file_suffix(&self) -> String;
|
||||
fn compile(&mut self, _input: &str) -> LLVMCodeString {
|
||||
LLVMCodeString("".to_string())
|
||||
}
|
||||
fn can_compile(&self) -> bool {
|
||||
false
|
||||
}
|
||||
}
|
|
@ -1,378 +0,0 @@
|
|||
#![feature(link_args)]
|
||||
#![feature(advanced_slice_patterns, slice_patterns, box_patterns, box_syntax)]
|
||||
#![feature(plugin)]
|
||||
#![plugin(rocket_codegen)]
|
||||
extern crate getopts;
|
||||
extern crate rustyline;
|
||||
extern crate itertools;
|
||||
#[macro_use]
|
||||
extern crate lazy_static;
|
||||
#[macro_use]
|
||||
extern crate maplit;
|
||||
#[macro_use]
|
||||
extern crate serde_derive;
|
||||
extern crate serde_json;
|
||||
extern crate rocket;
|
||||
extern crate rocket_contrib;
|
||||
extern crate includedir;
|
||||
extern crate phf;
|
||||
|
||||
use std::path::Path;
|
||||
use std::fs::File;
|
||||
use std::io::{Read, Write};
|
||||
use std::process::exit;
|
||||
use std::default::Default;
|
||||
|
||||
use rustyline::error::ReadlineError;
|
||||
use rustyline::Editor;
|
||||
|
||||
mod language;
|
||||
mod webapp;
|
||||
pub mod llvm_wrap;
|
||||
|
||||
include!(concat!(env!("OUT_DIR"), "/static.rs"));
|
||||
|
||||
pub use language::{ProgrammingLanguageInterface, EvalOptions, TraceArtifact, LanguageOutput, LLVMCodeString};
|
||||
pub type PLIGenerator = Box<Fn() -> Box<ProgrammingLanguageInterface> + Send + Sync>;
|
||||
|
||||
pub fn schala_main(generators: Vec<PLIGenerator>) {
|
||||
let languages: Vec<Box<ProgrammingLanguageInterface>> = generators.iter().map(|x| x()).collect();
|
||||
|
||||
let option_matches = program_options().parse(std::env::args()).unwrap_or_else(|e| {
|
||||
println!("{:?}", e);
|
||||
exit(1);
|
||||
});
|
||||
|
||||
if option_matches.opt_present("list-languages") {
|
||||
for lang in languages {
|
||||
println!("{}", lang.get_language_name());
|
||||
}
|
||||
exit(1);
|
||||
}
|
||||
|
||||
if option_matches.opt_present("help") {
|
||||
println!("{}", program_options().usage("Schala metainterpreter"));
|
||||
exit(0);
|
||||
}
|
||||
|
||||
if option_matches.opt_present("webapp") {
|
||||
webapp::web_main(generators);
|
||||
exit(0);
|
||||
}
|
||||
|
||||
let language_names: Vec<String> = languages.iter().map(|lang| {lang.get_language_name()}).collect();
|
||||
let initial_index: usize =
|
||||
option_matches.opt_str("lang")
|
||||
.and_then(|lang| { language_names.iter().position(|x| { x.to_lowercase() == lang.to_lowercase() }) })
|
||||
.unwrap_or(0);
|
||||
|
||||
let mut options = EvalOptions::default();
|
||||
options.compile = match option_matches.opt_str("eval-style") {
|
||||
Some(ref s) if s == "compile" => true,
|
||||
_ => false
|
||||
};
|
||||
|
||||
match option_matches.free[..] {
|
||||
[] | [_] => {
|
||||
let mut repl = Repl::new(languages, initial_index);
|
||||
repl.options.show_llvm_ir = true; //TODO make this be configurable
|
||||
repl.run();
|
||||
}
|
||||
[_, ref filename, _..] => {
|
||||
|
||||
run_noninteractive(filename, languages, options);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
fn run_noninteractive(filename: &str, languages: Vec<Box<ProgrammingLanguageInterface>>, options: EvalOptions) {
|
||||
let path = Path::new(filename);
|
||||
let ext = path.extension().and_then(|e| e.to_str()).unwrap_or_else(|| {
|
||||
println!("Source file lacks extension");
|
||||
exit(1);
|
||||
});
|
||||
let mut language = Box::new(languages.into_iter().find(|lang| lang.get_source_file_suffix() == ext)
|
||||
.unwrap_or_else(|| {
|
||||
println!("Extension .{} not recognized", ext);
|
||||
exit(1);
|
||||
}));
|
||||
|
||||
let mut source_file = File::open(path).unwrap();
|
||||
let mut buffer = String::new();
|
||||
|
||||
source_file.read_to_string(&mut buffer).unwrap();
|
||||
|
||||
if options.compile {
|
||||
if !language.can_compile() {
|
||||
panic!("Trying to compile a non-compileable language");
|
||||
} else {
|
||||
let llvm_bytecode = language.compile(&buffer);
|
||||
compilation_sequence(llvm_bytecode, filename);
|
||||
}
|
||||
} else {
|
||||
let output = language.evaluate_in_repl(&buffer, &options);
|
||||
// if output.has_error....
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
struct Repl {
|
||||
options: EvalOptions,
|
||||
languages: Vec<Box<ProgrammingLanguageInterface>>,
|
||||
current_language_index: usize,
|
||||
interpreter_directive_sigil: char,
|
||||
console: rustyline::Editor<()>,
|
||||
}
|
||||
|
||||
impl Repl {
|
||||
fn new(languages: Vec<Box<ProgrammingLanguageInterface>>, initial_index: usize) -> Repl {
|
||||
let i = if initial_index < languages.len() { initial_index } else { 0 };
|
||||
|
||||
let console = Editor::<()>::new();
|
||||
|
||||
Repl {
|
||||
options: Repl::get_options(),
|
||||
languages: languages,
|
||||
current_language_index: i,
|
||||
interpreter_directive_sigil: '.',
|
||||
console
|
||||
}
|
||||
}
|
||||
|
||||
fn get_options() -> EvalOptions {
|
||||
File::open(".schala_repl")
|
||||
.and_then(|mut file| {
|
||||
let mut contents = String::new();
|
||||
file.read_to_string(&mut contents)?;
|
||||
Ok(contents)
|
||||
})
|
||||
.and_then(|contents| {
|
||||
let options: EvalOptions = serde_json::from_str(&contents)?;
|
||||
Ok(options)
|
||||
}).unwrap_or(EvalOptions::default())
|
||||
}
|
||||
|
||||
fn save_options(&self) {
|
||||
let ref options = self.options;
|
||||
let read = File::create(".schala_repl")
|
||||
.and_then(|mut file| {
|
||||
let buf = serde_json::to_string(options).unwrap();
|
||||
file.write_all(buf.as_bytes())
|
||||
});
|
||||
|
||||
if let Err(err) = read {
|
||||
println!("Error saving .schala_repl file {}", err);
|
||||
}
|
||||
}
|
||||
|
||||
fn run(&mut self) {
|
||||
println!("MetaInterpreter v 0.05");
|
||||
|
||||
self.console.get_history().load(".schala_history").unwrap_or(());
|
||||
|
||||
loop {
|
||||
let language_name = self.languages[self.current_language_index].get_language_name();
|
||||
let prompt_str = format!("{} >> ", language_name);
|
||||
|
||||
match self.console.readline(&prompt_str) {
|
||||
Err(ReadlineError::Eof) | Err(ReadlineError::Interrupted) => break,
|
||||
Err(e) => {
|
||||
println!("Terminal read error: {}", e);
|
||||
},
|
||||
Ok(ref input) => {
|
||||
self.console.add_history_entry(input);
|
||||
if self.handle_interpreter_directive(input) {
|
||||
continue;
|
||||
}
|
||||
let output = self.input_handler(input);
|
||||
println!("=> {}", output);
|
||||
}
|
||||
_ => (),
|
||||
}
|
||||
}
|
||||
self.console.get_history().save(".schala_history").unwrap_or(());
|
||||
self.save_options();
|
||||
println!("Exiting...");
|
||||
}
|
||||
|
||||
fn input_handler(&mut self, input: &str) -> String {
|
||||
let ref mut language = self.languages[self.current_language_index];
|
||||
let interpreter_output = language.evaluate_in_repl(input, &self.options);
|
||||
interpreter_output.to_string()
|
||||
}
|
||||
|
||||
fn handle_interpreter_directive(&mut self, input: &str) -> bool {
|
||||
match input.chars().nth(0) {
|
||||
Some(ch) if ch == self.interpreter_directive_sigil => (),
|
||||
_ => return false
|
||||
}
|
||||
|
||||
let mut iter = input.chars();
|
||||
iter.next();
|
||||
let trimmed_sigil: &str = iter.as_str();
|
||||
|
||||
let commands: Vec<&str> = trimmed_sigil
|
||||
.split_whitespace()
|
||||
.collect();
|
||||
|
||||
let cmd: &str = match commands.get(0).clone() {
|
||||
None => return true,
|
||||
Some(s) => s
|
||||
};
|
||||
|
||||
match cmd {
|
||||
"exit" | "quit" => {
|
||||
self.save_options();
|
||||
exit(0)
|
||||
},
|
||||
"help" => {
|
||||
println!("Commands:");
|
||||
println!("exit | quit");
|
||||
println!("lang(uage) [go|show|next|previous]");
|
||||
println!("set [show|hide] [tokens|parse|symbols|eval|llvm]");
|
||||
}
|
||||
"lang" | "language" => {
|
||||
match commands.get(1) {
|
||||
Some(&"show") => {
|
||||
for (i, lang) in self.languages.iter().enumerate() {
|
||||
if i == self.current_language_index {
|
||||
println!("* {}", lang.get_language_name());
|
||||
} else {
|
||||
println!("{}", lang.get_language_name());
|
||||
}
|
||||
}
|
||||
},
|
||||
Some(&"go") => {
|
||||
match commands.get(2) {
|
||||
None => println!("Must specify a language name"),
|
||||
Some(&desired_name) => {
|
||||
for (i, _) in self.languages.iter().enumerate() {
|
||||
let lang_name = self.languages[i].get_language_name();
|
||||
if lang_name.to_lowercase() == desired_name.to_lowercase() {
|
||||
self.current_language_index = i;
|
||||
println!("Switching to {}", self.languages[self.current_language_index].get_language_name());
|
||||
return true;
|
||||
}
|
||||
}
|
||||
println!("Language {} not found", desired_name);
|
||||
}
|
||||
}
|
||||
},
|
||||
Some(&"next") => {
|
||||
self.current_language_index = (self.current_language_index + 1) % self.languages.len();
|
||||
println!("Switching to {}", self.languages[self.current_language_index].get_language_name());
|
||||
}
|
||||
Some(&"prev") | Some(&"previous") => {
|
||||
self.current_language_index = if self.current_language_index == 0 { self.languages.len() - 1 } else { self.current_language_index - 1 };
|
||||
println!("Switching to {}", self.languages[self.current_language_index].get_language_name());
|
||||
},
|
||||
Some(e) => println!("Bad `lang` argument: {}", e),
|
||||
None => println!("`lang` - valid arguments `show`, `next`, `prev`|`previous`"),
|
||||
}
|
||||
},
|
||||
"set" => {
|
||||
let show = match commands.get(1) {
|
||||
Some(&"show") => true,
|
||||
Some(&"hide") => false,
|
||||
Some(e) => {
|
||||
println!("Bad `set` argument: {}", e);
|
||||
return true;
|
||||
}
|
||||
None => {
|
||||
println!("`set` - valid arguments `show {{option}}`, `hide {{option}}`");
|
||||
return true;
|
||||
}
|
||||
};
|
||||
match commands.get(2) {
|
||||
Some(&"tokens") => self.options.debug_tokens = show,
|
||||
Some(&"parse") => self.options.debug_parse = show,
|
||||
Some(&"symbols") => self.options.debug_symbol_table = show,
|
||||
Some(&"eval") => {
|
||||
//let ref mut language = self.languages[self.current_language_index];
|
||||
//language.set_option("trace_evaluation", show);
|
||||
},
|
||||
Some(&"llvm") => self.options.show_llvm_ir = show,
|
||||
Some(e) => {
|
||||
println!("Bad `show`/`hide` argument: {}", e);
|
||||
return true;
|
||||
}
|
||||
None => {
|
||||
println!("`show`/`hide` requires an argument");
|
||||
return true;
|
||||
}
|
||||
}
|
||||
},
|
||||
e => println!("Unknown command: {}", e)
|
||||
}
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
pub fn compilation_sequence(llvm_code: LLVMCodeString, sourcefile: &str) {
|
||||
use std::process::Command;
|
||||
|
||||
let ll_filename = "out.ll";
|
||||
let obj_filename = "out.o";
|
||||
let q: Vec<&str> = sourcefile.split('.').collect();
|
||||
let bin_filename = match &q[..] {
|
||||
&[name, "maaru"] => name,
|
||||
_ => panic!("Bad filename {}", sourcefile),
|
||||
};
|
||||
|
||||
let LLVMCodeString(llvm_str) = llvm_code;
|
||||
|
||||
println!("Compilation process finished for {}", ll_filename);
|
||||
File::create(ll_filename)
|
||||
.and_then(|mut f| f.write_all(llvm_str.as_bytes()))
|
||||
.expect("Error writing file");
|
||||
|
||||
let llc_output = Command::new("llc")
|
||||
.args(&["-filetype=obj", ll_filename, "-o", obj_filename])
|
||||
.output()
|
||||
.expect("Failed to run llc");
|
||||
|
||||
|
||||
if !llc_output.status.success() {
|
||||
println!("{}", String::from_utf8_lossy(&llc_output.stderr));
|
||||
}
|
||||
|
||||
let gcc_output = Command::new("gcc")
|
||||
.args(&["-o", bin_filename, &obj_filename])
|
||||
.output()
|
||||
.expect("failed to run gcc");
|
||||
|
||||
if !gcc_output.status.success() {
|
||||
println!("{}", String::from_utf8_lossy(&gcc_output.stdout));
|
||||
println!("{}", String::from_utf8_lossy(&gcc_output.stderr));
|
||||
}
|
||||
|
||||
for filename in [obj_filename].iter() {
|
||||
Command::new("rm")
|
||||
.arg(filename)
|
||||
.output()
|
||||
.expect(&format!("failed to run rm {}", filename));
|
||||
}
|
||||
}
|
||||
|
||||
fn program_options() -> getopts::Options {
|
||||
let mut options = getopts::Options::new();
|
||||
options.optopt("s",
|
||||
"eval-style",
|
||||
"Specify whether to compile (if supported) or interpret the language. If not specified, the default is language-specific",
|
||||
"[compile|interpret]"
|
||||
);
|
||||
options.optflag("",
|
||||
"list-languages",
|
||||
"Show a list of all supported languages");
|
||||
options.optopt("l",
|
||||
"lang",
|
||||
"Start up REPL in a language",
|
||||
"LANGUAGE");
|
||||
options.optflag("h",
|
||||
"help",
|
||||
"Show help text");
|
||||
options.optflag("w",
|
||||
"webapp",
|
||||
"Start up web interpreter");
|
||||
options
|
||||
}
|
|
@ -1,279 +0,0 @@
|
|||
#![allow(non_snake_case)]
|
||||
#![allow(dead_code)]
|
||||
extern crate llvm_sys;
|
||||
|
||||
use self::llvm_sys::{LLVMIntPredicate, LLVMRealPredicate};
|
||||
use self::llvm_sys::prelude::*;
|
||||
use self::llvm_sys::core;
|
||||
use std::ptr;
|
||||
use std::ffi::{CString, CStr};
|
||||
use std::os::raw::c_char;
|
||||
|
||||
pub fn create_context() -> LLVMContextRef {
|
||||
unsafe { core::LLVMContextCreate() }
|
||||
}
|
||||
pub fn module_create_with_name(name: &str) -> LLVMModuleRef {
|
||||
unsafe {
|
||||
let n = name.as_ptr() as *const _;
|
||||
core::LLVMModuleCreateWithName(n)
|
||||
}
|
||||
}
|
||||
pub fn CreateBuilderInContext(context: LLVMContextRef) -> LLVMBuilderRef {
|
||||
unsafe { core::LLVMCreateBuilderInContext(context) }
|
||||
}
|
||||
|
||||
pub fn AppendBasicBlockInContext(context: LLVMContextRef,
|
||||
function: LLVMValueRef,
|
||||
name: &str)
|
||||
-> LLVMBasicBlockRef {
|
||||
let c_name = CString::new(name).unwrap();
|
||||
unsafe { core::LLVMAppendBasicBlockInContext(context, function, c_name.as_ptr()) }
|
||||
}
|
||||
|
||||
pub fn AddFunction(module: LLVMModuleRef, name: &str, function_type: LLVMTypeRef) -> LLVMValueRef {
|
||||
let c_name = CString::new(name).unwrap();
|
||||
unsafe { core::LLVMAddFunction(module, c_name.as_ptr(), function_type) }
|
||||
}
|
||||
|
||||
pub fn FunctionType(return_type: LLVMTypeRef,
|
||||
mut param_types: Vec<LLVMTypeRef>,
|
||||
is_var_rag: bool)
|
||||
-> LLVMTypeRef {
|
||||
let len = param_types.len();
|
||||
unsafe {
|
||||
let pointer = param_types.as_mut_ptr();
|
||||
core::LLVMFunctionType(return_type,
|
||||
pointer,
|
||||
len as u32,
|
||||
if is_var_rag { 1 } else { 0 })
|
||||
}
|
||||
}
|
||||
|
||||
pub fn GetNamedFunction(module: LLVMModuleRef,
|
||||
name: &str) -> Option<LLVMValueRef> {
|
||||
|
||||
let c_name = CString::new(name).unwrap();
|
||||
let ret = unsafe { core::LLVMGetNamedFunction(module, c_name.as_ptr()) };
|
||||
|
||||
if ret.is_null() {
|
||||
None
|
||||
} else {
|
||||
Some(ret)
|
||||
}
|
||||
}
|
||||
|
||||
pub fn VoidTypeInContext(context: LLVMContextRef) -> LLVMTypeRef {
|
||||
unsafe { core::LLVMVoidTypeInContext(context) }
|
||||
}
|
||||
|
||||
pub fn DisposeBuilder(builder: LLVMBuilderRef) {
|
||||
unsafe { core::LLVMDisposeBuilder(builder) }
|
||||
}
|
||||
|
||||
pub fn DisposeModule(module: LLVMModuleRef) {
|
||||
unsafe { core::LLVMDisposeModule(module) }
|
||||
}
|
||||
|
||||
pub fn ContextDispose(context: LLVMContextRef) {
|
||||
unsafe { core::LLVMContextDispose(context) }
|
||||
}
|
||||
|
||||
pub fn PositionBuilderAtEnd(builder: LLVMBuilderRef, basic_block: LLVMBasicBlockRef) {
|
||||
unsafe { core::LLVMPositionBuilderAtEnd(builder, basic_block) }
|
||||
}
|
||||
|
||||
pub fn BuildRet(builder: LLVMBuilderRef, val: LLVMValueRef) -> LLVMValueRef {
|
||||
unsafe { core::LLVMBuildRet(builder, val) }
|
||||
}
|
||||
|
||||
pub fn BuildRetVoid(builder: LLVMBuilderRef) -> LLVMValueRef {
|
||||
unsafe { core::LLVMBuildRetVoid(builder) }
|
||||
}
|
||||
|
||||
pub fn DumpModule(module: LLVMModuleRef) {
|
||||
unsafe { core::LLVMDumpModule(module) }
|
||||
}
|
||||
|
||||
pub fn Int64TypeInContext(context: LLVMContextRef) -> LLVMTypeRef {
|
||||
unsafe { core::LLVMInt64TypeInContext(context) }
|
||||
}
|
||||
|
||||
pub fn ConstInt(int_type: LLVMTypeRef, n: u64, sign_extend: bool) -> LLVMValueRef {
|
||||
unsafe { core::LLVMConstInt(int_type, n, if sign_extend { 1 } else { 0 }) }
|
||||
}
|
||||
|
||||
pub fn BuildAdd(builder: LLVMBuilderRef,
|
||||
lhs: LLVMValueRef,
|
||||
rhs: LLVMValueRef,
|
||||
reg_name: &str)
|
||||
-> LLVMValueRef {
|
||||
let name = CString::new(reg_name).unwrap();
|
||||
unsafe { core::LLVMBuildAdd(builder, lhs, rhs, name.as_ptr()) }
|
||||
}
|
||||
|
||||
pub fn BuildSub(builder: LLVMBuilderRef,
|
||||
lhs: LLVMValueRef,
|
||||
rhs: LLVMValueRef,
|
||||
reg_name: &str)
|
||||
-> LLVMValueRef {
|
||||
let name = CString::new(reg_name).unwrap();
|
||||
unsafe { core::LLVMBuildSub(builder, lhs, rhs, name.as_ptr()) }
|
||||
}
|
||||
|
||||
pub fn BuildMul(builder: LLVMBuilderRef,
|
||||
lhs: LLVMValueRef,
|
||||
rhs: LLVMValueRef,
|
||||
reg_name: &str)
|
||||
-> LLVMValueRef {
|
||||
let name = CString::new(reg_name).unwrap();
|
||||
unsafe { core::LLVMBuildMul(builder, lhs, rhs, name.as_ptr()) }
|
||||
}
|
||||
|
||||
pub fn BuildUDiv(builder: LLVMBuilderRef,
|
||||
lhs: LLVMValueRef,
|
||||
rhs: LLVMValueRef,
|
||||
reg_name: &str)
|
||||
-> LLVMValueRef {
|
||||
let name = CString::new(reg_name).unwrap();
|
||||
unsafe { core::LLVMBuildUDiv(builder, lhs, rhs, name.as_ptr()) }
|
||||
}
|
||||
|
||||
pub fn BuildSRem(builder: LLVMBuilderRef,
|
||||
lhs: LLVMValueRef,
|
||||
rhs: LLVMValueRef,
|
||||
reg_name: &str)
|
||||
-> LLVMValueRef {
|
||||
let name = CString::new(reg_name).unwrap();
|
||||
unsafe { core::LLVMBuildSRem(builder, lhs, rhs, name.as_ptr()) }
|
||||
}
|
||||
|
||||
pub fn BuildCondBr(builder: LLVMBuilderRef,
|
||||
if_expr: LLVMValueRef,
|
||||
then_expr: LLVMBasicBlockRef,
|
||||
else_expr: LLVMBasicBlockRef) -> LLVMValueRef {
|
||||
|
||||
|
||||
unsafe { core::LLVMBuildCondBr(builder, if_expr, then_expr, else_expr) }
|
||||
}
|
||||
|
||||
pub fn BuildBr(builder: LLVMBuilderRef,
|
||||
dest: LLVMBasicBlockRef) -> LLVMValueRef {
|
||||
unsafe { core::LLVMBuildBr(builder, dest) }
|
||||
}
|
||||
|
||||
pub fn GetInsertBlock(builder: LLVMBuilderRef) -> LLVMBasicBlockRef {
|
||||
unsafe { core::LLVMGetInsertBlock(builder) }
|
||||
}
|
||||
|
||||
pub fn BuildPhi(builder: LLVMBuilderRef, ty: LLVMTypeRef, name: &str) -> LLVMValueRef {
|
||||
let name = CString::new(name).unwrap();
|
||||
unsafe { core::LLVMBuildPhi(builder, ty, name.as_ptr()) }
|
||||
}
|
||||
|
||||
pub fn SetValueName(value: LLVMValueRef, name: &str) {
|
||||
let name = CString::new(name).unwrap();
|
||||
unsafe {
|
||||
core::LLVMSetValueName(value, name.as_ptr())
|
||||
}
|
||||
}
|
||||
|
||||
pub fn GetValueName(value: LLVMValueRef) -> String {
|
||||
unsafe {
|
||||
let name_ptr: *const c_char = core::LLVMGetValueName(value);
|
||||
CStr::from_ptr(name_ptr).to_string_lossy().into_owned()
|
||||
}
|
||||
}
|
||||
|
||||
pub fn GetParams(function: LLVMValueRef) -> Vec<LLVMValueRef> {
|
||||
let size = CountParams(function);
|
||||
unsafe {
|
||||
let mut container = Vec::with_capacity(size);
|
||||
container.set_len(size);
|
||||
core::LLVMGetParams(function, container.as_mut_ptr());
|
||||
container
|
||||
}
|
||||
}
|
||||
|
||||
pub fn CountParams(function: LLVMValueRef) -> usize {
|
||||
unsafe { core::LLVMCountParams(function) as usize }
|
||||
}
|
||||
|
||||
pub fn BuildFCmp(builder: LLVMBuilderRef,
|
||||
op: LLVMRealPredicate,
|
||||
lhs: LLVMValueRef,
|
||||
rhs: LLVMValueRef,
|
||||
name: &str) -> LLVMValueRef {
|
||||
let name = CString::new(name).unwrap();
|
||||
unsafe { core::LLVMBuildFCmp(builder, op, lhs, rhs, name.as_ptr()) }
|
||||
}
|
||||
|
||||
pub fn BuildZExt(builder: LLVMBuilderRef,
|
||||
val: LLVMValueRef,
|
||||
dest_type: LLVMTypeRef,
|
||||
name: &str) -> LLVMValueRef {
|
||||
let name = CString::new(name).unwrap();
|
||||
unsafe { core::LLVMBuildZExt(builder, val, dest_type, name.as_ptr()) }
|
||||
}
|
||||
|
||||
pub fn BuildUIToFP(builder: LLVMBuilderRef,
|
||||
val: LLVMValueRef,
|
||||
dest_type: LLVMTypeRef,
|
||||
name: &str) -> LLVMValueRef {
|
||||
|
||||
let name = CString::new(name).unwrap();
|
||||
unsafe { core::LLVMBuildUIToFP(builder, val, dest_type, name.as_ptr()) }
|
||||
}
|
||||
|
||||
pub fn BuildICmp(builder: LLVMBuilderRef,
|
||||
op: LLVMIntPredicate,
|
||||
lhs: LLVMValueRef,
|
||||
rhs: LLVMValueRef,
|
||||
name: &str) -> LLVMValueRef {
|
||||
let name = CString::new(name).unwrap();
|
||||
unsafe { core::LLVMBuildICmp(builder, op, lhs, rhs, name.as_ptr()) }
|
||||
}
|
||||
|
||||
pub fn GetBasicBlockParent(block: LLVMBasicBlockRef) -> LLVMValueRef {
|
||||
unsafe { core::LLVMGetBasicBlockParent(block) }
|
||||
}
|
||||
|
||||
pub fn GetBasicBlocks(function: LLVMValueRef) -> Vec<LLVMBasicBlockRef> {
|
||||
let size = CountBasicBlocks(function);
|
||||
unsafe {
|
||||
let mut container = Vec::with_capacity(size);
|
||||
container.set_len(size);
|
||||
core::LLVMGetBasicBlocks(function, container.as_mut_ptr());
|
||||
container
|
||||
}
|
||||
}
|
||||
|
||||
pub fn CountBasicBlocks(function: LLVMValueRef) -> usize {
|
||||
unsafe { core::LLVMCountBasicBlocks(function) as usize }
|
||||
}
|
||||
|
||||
pub fn PrintModuleToString(module: LLVMModuleRef) -> String {
|
||||
unsafe {
|
||||
let str_ptr: *const c_char = core::LLVMPrintModuleToString(module);
|
||||
CStr::from_ptr(str_ptr).to_string_lossy().into_owned()
|
||||
}
|
||||
}
|
||||
|
||||
pub fn AddIncoming(phi_node: LLVMValueRef, mut incoming_values: Vec<LLVMValueRef>,
|
||||
mut incoming_blocks: Vec<LLVMBasicBlockRef>) {
|
||||
|
||||
let count = incoming_blocks.len() as u32;
|
||||
if incoming_values.len() as u32 != count {
|
||||
panic!("Bad invocation of AddIncoming");
|
||||
}
|
||||
|
||||
unsafe {
|
||||
let vals = incoming_values.as_mut_ptr();
|
||||
let blocks = incoming_blocks.as_mut_ptr();
|
||||
core::LLVMAddIncoming(phi_node, vals, blocks, count)
|
||||
}
|
||||
}
|
||||
|
||||
pub fn PrintModuleToFile(module: LLVMModuleRef, filename: &str) -> LLVMBool {
|
||||
let out_file = CString::new(filename).unwrap();
|
||||
unsafe { core::LLVMPrintModuleToFile(module, out_file.as_ptr(), ptr::null_mut()) }
|
||||
}
|
|
@ -1,45 +0,0 @@
|
|||
use rocket;
|
||||
use rocket::State;
|
||||
use rocket::response::Content;
|
||||
use rocket::response::NamedFile;
|
||||
use rocket::http::ContentType;
|
||||
use rocket_contrib::Json;
|
||||
use language::{ProgrammingLanguageInterface, EvalOptions};
|
||||
use WEBFILES;
|
||||
use ::PLIGenerator;
|
||||
|
||||
#[get("/")]
|
||||
fn index() -> Content<String> {
|
||||
let path = "static/index.html";
|
||||
let html_contents = String::from_utf8(WEBFILES.get(path).unwrap().into_owned()).unwrap();
|
||||
Content(ContentType::HTML, html_contents)
|
||||
}
|
||||
|
||||
#[get("/bundle.js")]
|
||||
fn js_bundle() -> Content<String> {
|
||||
let path = "static/bundle.js";
|
||||
let js_contents = String::from_utf8(WEBFILES.get(path).unwrap().into_owned()).unwrap();
|
||||
Content(ContentType::JavaScript, js_contents)
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
struct Input {
|
||||
source: String,
|
||||
}
|
||||
|
||||
#[derive(Serialize, Deserialize)]
|
||||
struct Output {
|
||||
text: String,
|
||||
}
|
||||
|
||||
#[post("/input", format = "application/json", data = "<input>")]
|
||||
fn interpreter_input(input: Json<Input>, generators: State<Vec<PLIGenerator>>) -> Json<Output> {
|
||||
let schala_gen = generators.get(0).unwrap();
|
||||
let mut schala: Box<ProgrammingLanguageInterface> = schala_gen();
|
||||
let code_output = schala.evaluate_in_repl(&input.source, &EvalOptions::default());
|
||||
Json(Output { text: code_output.to_string() })
|
||||
}
|
||||
|
||||
pub fn web_main(language_generators: Vec<PLIGenerator>) {
|
||||
rocket::ignite().manage(language_generators).mount("/", routes![index, js_bundle, interpreter_input]).launch();
|
||||
}
|
Binary file not shown.
After Width: | Height: | Size: 5.8 KiB |
|
@ -0,0 +1,8 @@
|
|||
[package]
|
||||
name = "schala-main"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
|
||||
|
||||
[dependencies]
|
|
@ -0,0 +1,3 @@
|
|||
fn main() {
|
||||
println!("Schala");
|
||||
}
|
|
@ -0,0 +1,8 @@
|
|||
[package]
|
||||
name = "schala-parser"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
|
||||
|
||||
[dependencies]
|
|
@ -0,0 +1,3 @@
|
|||
fn main() {
|
||||
println!("Hello, world!");
|
||||
}
|
|
@ -1,11 +0,0 @@
|
|||
|
||||
|
||||
fn outer() {
|
||||
fn inner(a) {
|
||||
a + 10
|
||||
}
|
||||
|
||||
inner(20) + 8.3
|
||||
}
|
||||
|
||||
outer()
|
|
@ -1,21 +0,0 @@
|
|||
|
||||
fn hella(a, b) {
|
||||
a + b
|
||||
}
|
||||
|
||||
fn paha(x, y, z) {
|
||||
x * y * z
|
||||
}
|
||||
|
||||
a = 1
|
||||
|
||||
c = if a {
|
||||
10
|
||||
} else {
|
||||
20
|
||||
}
|
||||
|
||||
q = 4
|
||||
q = q + 2
|
||||
q + 1 + c
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
if 20 {
|
||||
a = 20
|
||||
b = 30
|
||||
c = 40
|
||||
a + b + c
|
||||
} else {
|
||||
Null
|
||||
}
|
|
@ -1,5 +0,0 @@
|
|||
|
||||
(fn(q) { q * 2 }(25))
|
||||
|
||||
a = fn(x) { x + 5 }
|
||||
a(2)
|
|
@ -1,17 +0,0 @@
|
|||
|
||||
fn add(a, b) {
|
||||
a + b
|
||||
}
|
||||
|
||||
fn subtract(a, b) {
|
||||
a - b
|
||||
}
|
||||
|
||||
fn main() {
|
||||
first_value = add(20, 20)
|
||||
second_value = subtract(700, 650)
|
||||
first_value + second_value
|
||||
}
|
||||
|
||||
main()
|
||||
|
|
@ -1,24 +0,0 @@
|
|||
|
||||
|
||||
fn hella(x) {
|
||||
print("hey")
|
||||
if x == 3 {
|
||||
Null
|
||||
} else {
|
||||
hella(x + 1)
|
||||
}
|
||||
}
|
||||
|
||||
hella(0)
|
||||
|
||||
|
||||
|
||||
fn fib(x) {
|
||||
if x < 3 {
|
||||
1
|
||||
} else {
|
||||
fib(x - 1) + fib(x - 2)
|
||||
}
|
||||
}
|
||||
|
||||
fib(10)
|
|
@ -1,10 +0,0 @@
|
|||
fn main() {
|
||||
const a = 10
|
||||
const b = 20
|
||||
a + b
|
||||
}
|
||||
|
||||
print(main())
|
||||
|
||||
const xxx
|
||||
|
|
@ -1,105 +0,0 @@
|
|||
|
||||
fn main() {
|
||||
|
||||
# comments are scripting-style
|
||||
#{ but can also be
|
||||
|
||||
}# blocks
|
||||
|
||||
@annotations are with @-
|
||||
|
||||
# variable expressions
|
||||
var a: I32 = 20
|
||||
const b: String = 20
|
||||
|
||||
there(); can(); be(); multiple(); statements(); per_line();
|
||||
|
||||
#string interpolation
|
||||
const yolo = "I have ${a + b} people in my house"
|
||||
|
||||
# let expressions ??? not sure if I want this
|
||||
let a = 10, b = 20, c = 30 in a + b + c
|
||||
|
||||
#list literal
|
||||
const q = [1,2,3,4]
|
||||
|
||||
#lambda literal ?? maybe? not sure how this should work
|
||||
q.map(|item| { item * 100 })
|
||||
|
||||
fn yolo(a: MyType, b: YourType): ReturnType<Param1, Param2> {
|
||||
if a == 20 {
|
||||
return "early"
|
||||
}
|
||||
var sex = 20
|
||||
sex
|
||||
}
|
||||
|
||||
|
||||
for {
|
||||
# infinite loop
|
||||
}
|
||||
|
||||
#iteration over a variable
|
||||
for i <- [1..1000] {
|
||||
|
||||
} #return type is return type of block
|
||||
|
||||
#while loop
|
||||
for a != 3 || fuckTard() {
|
||||
break
|
||||
} #return type is return type of block
|
||||
|
||||
#monadic decomposition
|
||||
for {
|
||||
a <- maybeInt();
|
||||
s <- foo()
|
||||
} return {
|
||||
a + s
|
||||
} #return type is Monad<return type of block>
|
||||
|
||||
# let statements too!!
|
||||
for (a = 20
|
||||
b = fuck) {
|
||||
a + b
|
||||
}
|
||||
|
||||
|
||||
# pattern-matching
|
||||
match <expr> {
|
||||
Some(a) => {
|
||||
|
||||
},
|
||||
None => {
|
||||
|
||||
},
|
||||
}
|
||||
|
||||
#syntax is, I guess, for <expr> <brace-block>, where <expr> is a bool, or a <arrow-expr>
|
||||
|
||||
# type level alises
|
||||
typealias <name> = <other type> #maybe thsi should be 'alias'?
|
||||
|
||||
#what if type A = B meant that you could had to create A's with A(B), but when you used A's the interface was exactly like B's?
|
||||
# maybe introduce a 'newtype' keyword for this
|
||||
|
||||
#declaring types of all stripes
|
||||
type MyData = { a: i32, b: String }
|
||||
type MyType = MyType
|
||||
type Option<a> = None | Some(a)
|
||||
type Signal = Absence | SimplePresence(i32) | ComplexPresence {a: i32, b: MyCustomData}
|
||||
|
||||
#traits
|
||||
|
||||
trait Bashable { }
|
||||
trait Luggable {
|
||||
fn lug(self, a: Option<Self>)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
# lambdas
|
||||
#
|
||||
|x,y| { }() #is probably fine
|
||||
const a = |x: Type, y|: RetType { <statementblock> }
|
||||
const a: X -> Y -> Z = |x,y| { }
|
|
@ -1,12 +0,0 @@
|
|||
|
||||
fn a(x) {
|
||||
x + 20
|
||||
}
|
||||
|
||||
fn x(x) {
|
||||
x + a(9384)
|
||||
}
|
||||
|
||||
a(0)
|
||||
x(1)
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
|
||||
(display (+ 1 2))
|
||||
(display "Hello")
|
|
@ -1,8 +0,0 @@
|
|||
|
||||
fn めんどくさい(a) {
|
||||
a + 20
|
||||
}
|
||||
|
||||
print(めんどくさい(394))
|
||||
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
|
||||
|
||||
a = 0
|
||||
while a < 100000
|
||||
print("hello", a)
|
||||
a = a + 1
|
||||
end
|
|
@ -1,279 +0,0 @@
|
|||
extern crate llvm_sys;
|
||||
|
||||
use std::collections::HashMap;
|
||||
|
||||
use self::llvm_sys::prelude::*;
|
||||
use self::llvm_sys::{LLVMIntPredicate};
|
||||
|
||||
use maaru_lang::parser::{AST, Statement, Function, Prototype, Expression, BinOp};
|
||||
use schala_lib::LLVMCodeString;
|
||||
|
||||
use schala_lib::llvm_wrap as LLVMWrap;
|
||||
|
||||
type VariableMap = HashMap<String, LLVMValueRef>;
|
||||
|
||||
struct CompilationData {
|
||||
context: LLVMContextRef,
|
||||
module: LLVMModuleRef,
|
||||
builder: LLVMBuilderRef,
|
||||
variables: VariableMap,
|
||||
main_function: LLVMValueRef,
|
||||
current_function: Option<LLVMValueRef>,
|
||||
}
|
||||
|
||||
pub fn compile_ast(ast: AST) -> LLVMCodeString {
|
||||
println!("Compiling!");
|
||||
let names: VariableMap = HashMap::new();
|
||||
|
||||
let context = LLVMWrap::create_context();
|
||||
let module = LLVMWrap::module_create_with_name("example module");
|
||||
let builder = LLVMWrap::CreateBuilderInContext(context);
|
||||
|
||||
let program_return_type = LLVMWrap::Int64TypeInContext(context);
|
||||
let main_function_type = LLVMWrap::FunctionType(program_return_type, Vec::new(), false);
|
||||
let main_function: LLVMValueRef = LLVMWrap::AddFunction(module, "main", main_function_type);
|
||||
|
||||
let mut data = CompilationData {
|
||||
context: context,
|
||||
builder: builder,
|
||||
module: module,
|
||||
variables: names,
|
||||
main_function: main_function,
|
||||
current_function: None,
|
||||
};
|
||||
|
||||
let bb = LLVMWrap::AppendBasicBlockInContext(data.context, data.main_function, "entry");
|
||||
LLVMWrap::PositionBuilderAtEnd(builder, bb);
|
||||
|
||||
let value = ast.codegen(&mut data);
|
||||
|
||||
LLVMWrap::BuildRet(builder, value);
|
||||
|
||||
let ret = LLVMWrap::PrintModuleToString(module);
|
||||
|
||||
// Clean up. Values created in the context mostly get cleaned up there.
|
||||
LLVMWrap::DisposeBuilder(builder);
|
||||
LLVMWrap::DisposeModule(module);
|
||||
LLVMWrap::ContextDispose(context);
|
||||
LLVMCodeString(ret)
|
||||
}
|
||||
|
||||
trait CodeGen {
|
||||
fn codegen(&self, &mut CompilationData) -> LLVMValueRef;
|
||||
}
|
||||
|
||||
impl CodeGen for AST {
|
||||
fn codegen(&self, data: &mut CompilationData) -> LLVMValueRef {
|
||||
|
||||
let int_type = LLVMWrap::Int64TypeInContext(data.context);
|
||||
let mut ret = LLVMWrap::ConstInt(int_type, 0, false);
|
||||
|
||||
for statement in self {
|
||||
ret = statement.codegen(data);
|
||||
}
|
||||
ret
|
||||
}
|
||||
}
|
||||
|
||||
impl CodeGen for Statement {
|
||||
fn codegen(&self, data: &mut CompilationData) -> LLVMValueRef {
|
||||
use self::Statement::*;
|
||||
match self {
|
||||
&ExprNode(ref expr) => expr.codegen(data),
|
||||
&FuncDefNode(ref func) => func.codegen(data),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl CodeGen for Function {
|
||||
fn codegen(&self, data: &mut CompilationData) -> LLVMValueRef {
|
||||
|
||||
/* should have a check here for function already being defined */
|
||||
let function = self.prototype.codegen(data);
|
||||
let ref body = self.body;
|
||||
|
||||
data.current_function = Some(function);
|
||||
|
||||
let return_type = LLVMWrap::Int64TypeInContext(data.context);
|
||||
let mut ret = LLVMWrap::ConstInt(return_type, 0, false);
|
||||
|
||||
let block = LLVMWrap::AppendBasicBlockInContext(data.context, function, "entry");
|
||||
LLVMWrap::PositionBuilderAtEnd(data.builder, block);
|
||||
|
||||
//insert function params into variables
|
||||
for value in LLVMWrap::GetParams(function) {
|
||||
let name = LLVMWrap::GetValueName(value);
|
||||
data.variables.insert(name, value);
|
||||
}
|
||||
|
||||
for expr in body {
|
||||
ret = expr.codegen(data);
|
||||
}
|
||||
|
||||
LLVMWrap::BuildRet(data.builder, ret);
|
||||
|
||||
// get basic block of main
|
||||
let main_bb = LLVMWrap::GetBasicBlocks(data.main_function).get(0).expect("Couldn't get first block of main").clone();
|
||||
LLVMWrap::PositionBuilderAtEnd(data.builder, main_bb);
|
||||
|
||||
data.current_function = None;
|
||||
|
||||
ret
|
||||
}
|
||||
}
|
||||
|
||||
impl CodeGen for Prototype {
|
||||
fn codegen(&self, data: &mut CompilationData) -> LLVMValueRef {
|
||||
let num_args = self.parameters.len();
|
||||
let return_type = LLVMWrap::Int64TypeInContext(data.context);
|
||||
let mut arguments: Vec<LLVMTypeRef> = vec![];
|
||||
|
||||
for _ in 0..num_args {
|
||||
arguments.push(LLVMWrap::Int64TypeInContext(data.context));
|
||||
}
|
||||
|
||||
let function_type =
|
||||
LLVMWrap::FunctionType(return_type,
|
||||
arguments,
|
||||
false);
|
||||
|
||||
let function = LLVMWrap::AddFunction(data.module,
|
||||
&*self.name,
|
||||
function_type);
|
||||
|
||||
let function_params = LLVMWrap::GetParams(function);
|
||||
for (index, param) in function_params.iter().enumerate() {
|
||||
let name = self.parameters.get(index).expect(&format!("Failed this check at index {}", index));
|
||||
let new = *param;
|
||||
|
||||
LLVMWrap::SetValueName(new, name);
|
||||
}
|
||||
|
||||
function
|
||||
}
|
||||
}
|
||||
|
||||
impl CodeGen for Expression {
|
||||
fn codegen(&self, data: &mut CompilationData) -> LLVMValueRef {
|
||||
use self::BinOp::*;
|
||||
use self::Expression::*;
|
||||
|
||||
let int_type = LLVMWrap::Int64TypeInContext(data.context);
|
||||
let zero = LLVMWrap::ConstInt(int_type, 0, false);
|
||||
|
||||
match *self {
|
||||
Variable(ref name) => *data.variables.get(&**name).expect(&format!("Can't find variable {}", name)),
|
||||
BinExp(Assign, ref left, ref right) => {
|
||||
if let Variable(ref name) = **left {
|
||||
let new_value = right.codegen(data);
|
||||
data.variables.insert((**name).clone(), new_value);
|
||||
new_value
|
||||
} else {
|
||||
panic!("Bad variable assignment")
|
||||
}
|
||||
}
|
||||
BinExp(ref op, ref left, ref right) => {
|
||||
let lhs = left.codegen(data);
|
||||
let rhs = right.codegen(data);
|
||||
op.codegen_with_ops(data, lhs, rhs)
|
||||
}
|
||||
Number(ref n) => {
|
||||
let native_val = *n as u64;
|
||||
let int_value: LLVMValueRef = LLVMWrap::ConstInt(int_type, native_val, false);
|
||||
int_value
|
||||
}
|
||||
Conditional(ref test, ref then_expr, ref else_expr) => {
|
||||
let condition_value = test.codegen(data);
|
||||
let is_nonzero =
|
||||
LLVMWrap::BuildICmp(data.builder,
|
||||
LLVMIntPredicate::LLVMIntNE,
|
||||
condition_value,
|
||||
zero,
|
||||
"ifcond");
|
||||
|
||||
let func = LLVMWrap::GetBasicBlockParent(LLVMWrap::GetInsertBlock(data.builder));
|
||||
|
||||
let mut then_block =
|
||||
LLVMWrap::AppendBasicBlockInContext(data.context, func, "then_block");
|
||||
let mut else_block =
|
||||
LLVMWrap::AppendBasicBlockInContext(data.context, func, "else_block");
|
||||
let merge_block =
|
||||
LLVMWrap::AppendBasicBlockInContext(data.context, func, "ifcont");
|
||||
|
||||
// add conditional branch to ifcond block
|
||||
LLVMWrap::BuildCondBr(data.builder, is_nonzero, then_block, else_block);
|
||||
|
||||
// start inserting into then block
|
||||
LLVMWrap::PositionBuilderAtEnd(data.builder, then_block);
|
||||
|
||||
// then-block codegen
|
||||
let then_return = then_expr.codegen(data);
|
||||
LLVMWrap::BuildBr(data.builder, merge_block);
|
||||
|
||||
// update then block b/c recursive codegen() call may have changed the notion of
|
||||
// the current block
|
||||
then_block = LLVMWrap::GetInsertBlock(data.builder);
|
||||
|
||||
// then do the same stuff again for the else branch
|
||||
//
|
||||
LLVMWrap::PositionBuilderAtEnd(data.builder, else_block);
|
||||
let else_return = match *else_expr {
|
||||
Some(ref e) => e.codegen(data),
|
||||
None => zero,
|
||||
};
|
||||
LLVMWrap::BuildBr(data.builder, merge_block);
|
||||
else_block = LLVMWrap::GetInsertBlock(data.builder);
|
||||
|
||||
LLVMWrap::PositionBuilderAtEnd(data.builder, merge_block);
|
||||
|
||||
let phi = LLVMWrap::BuildPhi(data.builder, int_type, "phinode");
|
||||
let values = vec![then_return, else_return];
|
||||
let blocks = vec![then_block, else_block];
|
||||
LLVMWrap::AddIncoming(phi, values, blocks);
|
||||
phi
|
||||
}
|
||||
Block(ref exprs) => {
|
||||
let mut ret = zero;
|
||||
for e in exprs.iter() {
|
||||
ret = e.codegen(data);
|
||||
}
|
||||
ret
|
||||
}
|
||||
ref e => {
|
||||
println!("Unimplemented {:?}", e);
|
||||
unimplemented!()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl BinOp {
|
||||
fn codegen_with_ops(&self, data: &CompilationData, lhs: LLVMValueRef, rhs: LLVMValueRef) -> LLVMValueRef {
|
||||
use self::BinOp::*;
|
||||
macro_rules! simple_binop {
|
||||
($fnname: expr, $name: expr) => {
|
||||
$fnname(data.builder, lhs, rhs, $name)
|
||||
}
|
||||
}
|
||||
let int_type = LLVMWrap::Int64TypeInContext(data.context);
|
||||
match *self {
|
||||
Add => simple_binop!(LLVMWrap::BuildAdd, "addtemp"),
|
||||
Sub => simple_binop!(LLVMWrap::BuildSub, "subtemp"),
|
||||
Mul => simple_binop!(LLVMWrap::BuildMul, "multemp"),
|
||||
Div => simple_binop!(LLVMWrap::BuildUDiv, "divtemp"),
|
||||
Mod => simple_binop!(LLVMWrap::BuildSRem, "remtemp"),
|
||||
Less => {
|
||||
let pred: LLVMValueRef =
|
||||
LLVMWrap::BuildICmp(data.builder, LLVMIntPredicate::LLVMIntULT, lhs, rhs, "tmp");
|
||||
LLVMWrap::BuildZExt(data.builder, pred, int_type, "temp")
|
||||
}
|
||||
Greater => {
|
||||
let pred: LLVMValueRef =
|
||||
LLVMWrap::BuildICmp(data.builder, LLVMIntPredicate::LLVMIntUGT, lhs, rhs, "tmp");
|
||||
LLVMWrap::BuildZExt(data.builder, pred, int_type, "temp")
|
||||
}
|
||||
ref unknown => panic!("Bad operator {:?}", unknown),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1,481 +0,0 @@
|
|||
extern crate take_mut;
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::collections::VecDeque;
|
||||
use maaru_lang::parser::{AST, Statement, Expression, Function, Callable, BinOp};
|
||||
use std::rc::Rc;
|
||||
use std::io::{Write, Stdout, BufWriter};
|
||||
use std::convert::From;
|
||||
|
||||
use maaru_lang::parser::Expression::*;
|
||||
use maaru_lang::parser::Statement::*;
|
||||
|
||||
type Reduction<T> = (T, Option<SideEffect>);
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
enum ReducedValue {
|
||||
StringLiteral(Rc<String>),
|
||||
ListLiteral(VecDeque<Expression>),
|
||||
StructLiteral(VecDeque<(Rc<String>, Expression)>),
|
||||
Number(f64),
|
||||
Lambda(Function),
|
||||
}
|
||||
|
||||
impl From<ReducedValue> for Expression {
|
||||
fn from(rv: ReducedValue) -> Expression {
|
||||
match rv {
|
||||
ReducedValue::Number(n) => Expression::Number(n),
|
||||
ReducedValue::StringLiteral(n) => Expression::StringLiteral(n),
|
||||
ReducedValue::Lambda(f) => Expression::Lambda(f),
|
||||
ReducedValue::ListLiteral(items) => Expression::ListLiteral(items),
|
||||
ReducedValue::StructLiteral(items) => Expression::StructLiteral(items),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl From<Expression> for ReducedValue {
|
||||
fn from(rv: Expression) -> ReducedValue {
|
||||
match rv {
|
||||
Expression::Number(n) => ReducedValue::Number(n),
|
||||
Expression::StringLiteral(n) => ReducedValue::StringLiteral(n),
|
||||
Expression::Lambda(f) => ReducedValue::Lambda(f),
|
||||
Expression::ListLiteral(items) => ReducedValue::ListLiteral(items),
|
||||
Expression::StructLiteral(items) => ReducedValue::StructLiteral(items),
|
||||
_ => panic!("trying to store a non-fully-reduced variable"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn get_indexer(f: f64) -> Option<usize> {
|
||||
if f.fract() == 0.0 {
|
||||
if f.trunc() >= 0.0 {
|
||||
return Some(f.trunc() as usize);
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
enum SideEffect {
|
||||
Print(String),
|
||||
AddBinding(Rc<String>, ReducedValue),
|
||||
}
|
||||
|
||||
pub struct Evaluator<'a> {
|
||||
parent: Option<&'a Evaluator<'a>>,
|
||||
variables: HashMap<String, ReducedValue>,
|
||||
stdout: BufWriter<Stdout>,
|
||||
pub trace_evaluation: bool,
|
||||
}
|
||||
|
||||
impl<'a> Evaluator<'a> {
|
||||
pub fn new(parent: Option<&'a Evaluator>) -> Evaluator<'a> {
|
||||
Evaluator {
|
||||
variables: HashMap::new(),
|
||||
parent: parent,
|
||||
stdout: BufWriter::new(::std::io::stdout()),
|
||||
trace_evaluation: parent.map_or(false, |e| e.trace_evaluation),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn run(&mut self, ast: AST) -> Vec<String> {
|
||||
ast.into_iter()
|
||||
.map(|astnode| format!("{}", self.reduction_loop(astnode)))
|
||||
.collect()
|
||||
|
||||
}
|
||||
|
||||
fn add_binding(&mut self, var: String, value: ReducedValue) {
|
||||
self.variables.insert(var, value);
|
||||
}
|
||||
|
||||
fn lookup_binding(&self, var: &str) -> Option<ReducedValue> {
|
||||
match self.variables.get(var) {
|
||||
Some(expr) => Some(expr.clone()),
|
||||
None => match self.parent {
|
||||
Some(env) => env.lookup_binding(var),
|
||||
None => None
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
trait Evaluable {
|
||||
fn is_reducible(&self) -> bool;
|
||||
}
|
||||
|
||||
impl Evaluable for Statement {
|
||||
fn is_reducible(&self) -> bool {
|
||||
match self {
|
||||
&ExprNode(ref expr) => expr.is_reducible(),
|
||||
&FuncDefNode(_) => true,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Evaluable for Expression {
|
||||
fn is_reducible(&self) -> bool {
|
||||
match *self {
|
||||
Null => false,
|
||||
StringLiteral(_) => false,
|
||||
Lambda(_) => false,
|
||||
Number(_) => false,
|
||||
ListLiteral(ref items) => {
|
||||
items.iter().any(|x| x.is_reducible())
|
||||
}
|
||||
StructLiteral(ref items) => {
|
||||
items.iter().any(|pair| pair.1.is_reducible())
|
||||
}
|
||||
_ => true,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Expression {
|
||||
fn is_truthy(&self) -> bool {
|
||||
match *self {
|
||||
Null => false,
|
||||
StringLiteral(ref s) if **s == "" => false,
|
||||
Number(n) if n == 0.0 => false,
|
||||
_ => true,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn is_assignment(op: &BinOp) -> bool {
|
||||
use self::BinOp::*;
|
||||
match *op {
|
||||
Assign | AddAssign | SubAssign |
|
||||
MulAssign | DivAssign => true,
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> Evaluator<'a> {
|
||||
fn reduction_loop(&mut self, mut node: Statement) -> Statement {
|
||||
loop {
|
||||
node = self.step(node);
|
||||
if !node.is_reducible() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
node
|
||||
}
|
||||
|
||||
fn step(&mut self, node: Statement) -> Statement {
|
||||
let mut trace = String::new();
|
||||
if self.trace_evaluation {
|
||||
trace.push_str(&format!("Step: {:?}", node));
|
||||
}
|
||||
|
||||
let (new_node, side_effect) = self.reduce_astnode(node);
|
||||
|
||||
if self.trace_evaluation {
|
||||
trace.push_str(&format!(" ➜ {:?}", new_node));
|
||||
}
|
||||
if let Some(s) = side_effect {
|
||||
if self.trace_evaluation {
|
||||
trace.push_str(&format!(" | side-effect: {:?}", s));
|
||||
}
|
||||
self.perform_side_effect(s);
|
||||
}
|
||||
if self.trace_evaluation {
|
||||
println!("{}", trace);
|
||||
}
|
||||
new_node
|
||||
}
|
||||
|
||||
fn perform_side_effect(&mut self, side_effect: SideEffect) {
|
||||
use self::SideEffect::*;
|
||||
match side_effect {
|
||||
Print(s) => {
|
||||
write!(self.stdout, "{}\n", s).unwrap();
|
||||
match self.stdout.flush() {
|
||||
Ok(_) => (),
|
||||
Err(_) => println!("Could not flush stdout"),
|
||||
};
|
||||
}
|
||||
AddBinding(var, value) => {
|
||||
self.add_binding((*var).clone(), value);
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
fn reduce_astnode(&mut self, node: Statement) -> Reduction<Statement> {
|
||||
match node {
|
||||
ExprNode(expr) => {
|
||||
if expr.is_reducible() {
|
||||
let (new_expr, side_effect) = self.reduce_expr(expr);
|
||||
(ExprNode(new_expr), side_effect)
|
||||
} else {
|
||||
(ExprNode(expr), None)
|
||||
}
|
||||
}
|
||||
FuncDefNode(func) => {
|
||||
let name = func.prototype.name.clone();
|
||||
let reduced_value = ReducedValue::Lambda(func.clone());
|
||||
let binding = Some(SideEffect::AddBinding(name, reduced_value));
|
||||
(ExprNode(Expression::Lambda(func)), binding)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
//TODO I probably want another Expression variant that holds a ReducedValue
|
||||
fn reduce_expr(&mut self, expression: Expression) -> Reduction<Expression> {
|
||||
match expression {
|
||||
Null => (Null, None),
|
||||
e @ StringLiteral(_) => (e, None),
|
||||
e @ Number(_) => (e, None),
|
||||
e @ Lambda(_) => (e, None),
|
||||
Variable(ref var) => {
|
||||
match self.lookup_binding(var).map(|x| x.into()) {
|
||||
None => (Null, None),
|
||||
Some(expr) => (expr, None),
|
||||
}
|
||||
}
|
||||
BinExp(op, mut left, mut right) => {
|
||||
if right.is_reducible() {
|
||||
let mut side_effect = None;
|
||||
take_mut::take(right.as_mut(), |expr| { let (a, b) = self.reduce_expr(expr); side_effect = b; a});
|
||||
return (BinExp(op, left, right), side_effect);
|
||||
}
|
||||
|
||||
if let BinOp::Assign = op {
|
||||
return match *left {
|
||||
Variable(var) => {
|
||||
let reduced_value: ReducedValue = ReducedValue::from(*right);
|
||||
let binding = SideEffect::AddBinding(var, reduced_value);
|
||||
(Null, Some(binding))
|
||||
},
|
||||
_ => (Null, None)
|
||||
};
|
||||
}
|
||||
|
||||
if is_assignment(&op) {
|
||||
use self::BinOp::*;
|
||||
let new_op = match op {
|
||||
AddAssign => Add,
|
||||
SubAssign => Sub,
|
||||
MulAssign => Mul,
|
||||
DivAssign => Div,
|
||||
_ => unreachable!(),
|
||||
};
|
||||
|
||||
let reduction =
|
||||
BinExp(BinOp::Assign,
|
||||
Box::new(*left.clone()),
|
||||
Box::new(BinExp(new_op, left, right))
|
||||
);
|
||||
|
||||
return (reduction, None);
|
||||
}
|
||||
|
||||
if left.is_reducible() {
|
||||
let mut side_effect = None;
|
||||
take_mut::take(left.as_mut(), |expr| { let (a, b) = self.reduce_expr(expr); side_effect = b; a});
|
||||
(BinExp(op, left, right), side_effect)
|
||||
} else {
|
||||
(self.reduce_binop(op, *left, *right), None) //can assume both arguments are maximally reduced
|
||||
}
|
||||
}
|
||||
Call(callable, mut args) => {
|
||||
let mut f = true;
|
||||
for arg in args.iter_mut() {
|
||||
if arg.is_reducible() {
|
||||
take_mut::take(arg, |arg| self.reduce_expr(arg).0);
|
||||
f = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if f {
|
||||
self.reduce_call(callable, args)
|
||||
} else {
|
||||
(Call(callable, args), None)
|
||||
}
|
||||
}
|
||||
While(test, body) => {
|
||||
let mut block = VecDeque::from(body.clone());
|
||||
block.push_back(While(test.clone(), body.clone()));
|
||||
let reduction = Conditional(test, Box::new(Block(block)), None);
|
||||
(reduction, None)
|
||||
}
|
||||
Conditional(box test, then_block, else_block) => {
|
||||
if test.is_reducible() {
|
||||
let (new_test, new_effect) = self.reduce_expr(test);
|
||||
(Conditional(Box::new(new_test), then_block, else_block), new_effect)
|
||||
} else {
|
||||
if test.is_truthy() {
|
||||
(*then_block, None)
|
||||
} else {
|
||||
match else_block {
|
||||
Some(box expr) => (expr, None),
|
||||
None => (Null, None),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Block(mut exprs) => {
|
||||
let first = exprs.pop_front();
|
||||
match first {
|
||||
None => (Null, None),
|
||||
Some(expr) => {
|
||||
if exprs.len() == 0 {
|
||||
(expr, None)
|
||||
} else {
|
||||
if expr.is_reducible() {
|
||||
let (new, side_effect) = self.reduce_expr(expr);
|
||||
exprs.push_front(new);
|
||||
(Block(exprs), side_effect)
|
||||
} else {
|
||||
(Block(exprs), None)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Index(mut expr, mut index_expr) => {
|
||||
if index_expr.is_reducible() {
|
||||
let mut side_effect = None;
|
||||
take_mut::take(index_expr.as_mut(), |expr| { let (a, b) = self.reduce_expr(expr); side_effect = b; a});
|
||||
return (Index(expr, index_expr), side_effect)
|
||||
}
|
||||
|
||||
if expr.is_reducible() {
|
||||
let mut side_effect = None;
|
||||
take_mut::take(expr.as_mut(), |expr| { let (a, b) = self.reduce_expr(expr); side_effect = b; a});
|
||||
return (Index(expr, index_expr), side_effect);
|
||||
}
|
||||
|
||||
match (*expr, *index_expr) {
|
||||
(ListLiteral(list_items), Number(n)) => {
|
||||
let indexed_expr = get_indexer(n).and_then(|i| list_items.get(i));
|
||||
if let Some(e) = indexed_expr {
|
||||
(e.clone(), None)
|
||||
} else {
|
||||
(Null, None)
|
||||
}
|
||||
}
|
||||
(StructLiteral(items), StringLiteral(s)) => {
|
||||
for item in items {
|
||||
if s == item.0 {
|
||||
return (item.1.clone(), None); //TODO this is hella inefficient
|
||||
}
|
||||
}
|
||||
(Null, None)
|
||||
},
|
||||
_ => (Null, None)
|
||||
}
|
||||
}
|
||||
ListLiteral(mut exprs) => {
|
||||
let mut side_effect = None;
|
||||
for expr in exprs.iter_mut() {
|
||||
if expr.is_reducible() {
|
||||
take_mut::take(expr, |expr| {
|
||||
let (a, b) = self.reduce_expr(expr);
|
||||
side_effect = b;
|
||||
a
|
||||
});
|
||||
break;
|
||||
}
|
||||
}
|
||||
(ListLiteral(exprs), side_effect)
|
||||
},
|
||||
|
||||
StructLiteral(mut items) => {
|
||||
let mut side_effect = None;
|
||||
for pair in items.iter_mut() {
|
||||
if pair.1.is_reducible() {
|
||||
take_mut::take(pair, |pair| {
|
||||
let (name, expr) = pair;
|
||||
let (a, b) = self.reduce_expr(expr);
|
||||
side_effect = b;
|
||||
(name, a)
|
||||
});
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
(StructLiteral(items), side_effect)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn reduce_binop(&mut self, op: BinOp, left: Expression, right: Expression) -> Expression {
|
||||
use self::BinOp::*;
|
||||
let truthy = Number(1.0);
|
||||
let falsy = Null;
|
||||
match (op, left, right) {
|
||||
(Add, Number(l), Number(r)) => Number(l + r),
|
||||
(Add, StringLiteral(s1), StringLiteral(s2)) => StringLiteral(Rc::new(format!("{}{}", *s1, *s2))),
|
||||
(Add, StringLiteral(s1), Number(r)) => StringLiteral(Rc::new(format!("{}{}", *s1, r))),
|
||||
(Add, Number(l), StringLiteral(s1)) => StringLiteral(Rc::new(format!("{}{}", l, *s1))),
|
||||
(Sub, Number(l), Number(r)) => Number(l - r),
|
||||
(Mul, Number(l), Number(r)) => Number(l * r),
|
||||
(Div, Number(l), Number(r)) if r != 0.0 => Number(l / r),
|
||||
(Mod, Number(l), Number(r)) => Number(l % r),
|
||||
(Less, Number(l), Number(r)) => if l < r { truthy } else { falsy },
|
||||
(LessEq, Number(l), Number(r)) => if l <= r { truthy } else { falsy },
|
||||
(Greater, Number(l), Number(r)) => if l > r { truthy } else { falsy },
|
||||
(GreaterEq, Number(l), Number(r)) => if l >= r { truthy } else { falsy },
|
||||
(Equal, Number(l), Number(r)) => if l == r { truthy } else { falsy },
|
||||
(Equal, Null, Null) => truthy,
|
||||
(Equal, StringLiteral(s1), StringLiteral(s2)) => if s1 == s2 { truthy } else { falsy },
|
||||
(Equal, _, _) => falsy,
|
||||
_ => falsy,
|
||||
}
|
||||
}
|
||||
|
||||
fn reduce_call(&mut self, callable: Callable, arguments: Vec<Expression>) -> Reduction<Expression> {
|
||||
if let Some(res) = handle_builtin(&callable, &arguments) {
|
||||
return res;
|
||||
}
|
||||
|
||||
let function = match callable {
|
||||
Callable::Lambda(func) => func.clone(),
|
||||
Callable::NamedFunction(name) => {
|
||||
match self.lookup_binding(&*name) {
|
||||
Some(ReducedValue::Lambda(func)) => func,
|
||||
_ => return (Null, None),
|
||||
}
|
||||
}
|
||||
};
|
||||
if function.prototype.parameters.len() != arguments.len() {
|
||||
return (Null, None);
|
||||
}
|
||||
|
||||
let mut evaluator = Evaluator::new(Some(self));
|
||||
for (binding, expr) in function.prototype.parameters.iter().zip(arguments.iter()) {
|
||||
evaluator.add_binding((**binding).clone(), expr.clone().into());
|
||||
}
|
||||
|
||||
let nodes = function.body.iter().map(|node| node.clone());
|
||||
let mut retval = ExprNode(Null);
|
||||
for n in nodes {
|
||||
retval = evaluator.reduction_loop(n);
|
||||
}
|
||||
|
||||
match retval {
|
||||
ExprNode(expr) => (expr, None),
|
||||
FuncDefNode(_) => panic!("This should never happen! A maximally-reduced node\
|
||||
should never be a function definition!")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_builtin(callable: &Callable, arguments: &Vec<Expression>) -> Option<Reduction<Expression>> {
|
||||
let name: &str = match *callable {
|
||||
Callable::NamedFunction(ref name) => *&name,
|
||||
_ => return None,
|
||||
};
|
||||
|
||||
match name {
|
||||
"print" => {
|
||||
let mut s = String::new();
|
||||
for arg in arguments {
|
||||
s.push_str(&format!("{} ", arg));
|
||||
}
|
||||
return Some((Null, Some(SideEffect::Print(s))));
|
||||
},
|
||||
_ => None
|
||||
}
|
||||
}
|
|
@ -1,99 +0,0 @@
|
|||
pub mod tokenizer;
|
||||
pub mod parser;
|
||||
pub mod eval;
|
||||
pub mod compilation;
|
||||
|
||||
use schala_lib::{ProgrammingLanguageInterface, EvalOptions, LanguageOutput, TraceArtifact, LLVMCodeString};
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct TokenError {
|
||||
pub msg: String,
|
||||
}
|
||||
|
||||
impl TokenError {
|
||||
pub fn new(msg: &str) -> TokenError {
|
||||
TokenError { msg: msg.to_string() }
|
||||
}
|
||||
}
|
||||
|
||||
pub use self::eval::Evaluator as MaaruEvaluator;
|
||||
|
||||
pub struct Maaru<'a> {
|
||||
evaluator: MaaruEvaluator<'a>
|
||||
}
|
||||
|
||||
impl<'a> Maaru<'a> {
|
||||
pub fn new() -> Maaru<'a> {
|
||||
Maaru {
|
||||
evaluator: MaaruEvaluator::new(None),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> ProgrammingLanguageInterface for Maaru<'a> {
|
||||
fn get_language_name(&self) -> String {
|
||||
"Maaru".to_string()
|
||||
}
|
||||
fn get_source_file_suffix(&self) -> String {
|
||||
format!("maaru")
|
||||
}
|
||||
|
||||
fn evaluate_in_repl(&mut self, input: &str, options: &EvalOptions) -> LanguageOutput {
|
||||
let mut output = LanguageOutput::default();
|
||||
|
||||
let tokens = match tokenizer::tokenize(input) {
|
||||
Ok(tokens) => {
|
||||
if options.debug_tokens {
|
||||
output.add_artifact(TraceArtifact::new("tokens", format!("{:?}", tokens)));
|
||||
}
|
||||
tokens
|
||||
},
|
||||
Err(err) => {
|
||||
output.add_output(format!("Tokenization error: {:?}\n", err.msg));
|
||||
return output;
|
||||
}
|
||||
};
|
||||
|
||||
let ast = match parser::parse(&tokens, &[]) {
|
||||
Ok(ast) => {
|
||||
if options.debug_parse {
|
||||
output.add_artifact(TraceArtifact::new("ast", format!("{:?}", ast)));
|
||||
}
|
||||
ast
|
||||
},
|
||||
Err(err) => {
|
||||
output.add_output(format!("Parse error: {:?}\n", err.msg));
|
||||
return output;
|
||||
}
|
||||
};
|
||||
let mut evaluation_output = String::new();
|
||||
for s in self.evaluator.run(ast).iter() {
|
||||
evaluation_output.push_str(s);
|
||||
}
|
||||
output.add_output(evaluation_output);
|
||||
return output;
|
||||
}
|
||||
|
||||
fn can_compile(&self) -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
fn compile(&mut self, input: &str) -> LLVMCodeString {
|
||||
let tokens = match tokenizer::tokenize(input) {
|
||||
Ok(tokens) => tokens,
|
||||
Err(err) => {
|
||||
let msg = format!("Tokenization error: {:?}\n", err.msg);
|
||||
panic!("{}", msg);
|
||||
}
|
||||
};
|
||||
|
||||
let ast = match parser::parse(&tokens, &[]) {
|
||||
Ok(ast) => ast,
|
||||
Err(err) => {
|
||||
let msg = format!("Parse error: {:?}\n", err.msg);
|
||||
panic!("{}", msg);
|
||||
}
|
||||
};
|
||||
compilation::compile_ast(ast)
|
||||
}
|
||||
}
|
|
@ -1,755 +0,0 @@
|
|||
use maaru_lang::tokenizer::{Token, Kw, OpTok};
|
||||
use maaru_lang::tokenizer::Token::*;
|
||||
|
||||
use std::fmt;
|
||||
use std::collections::VecDeque;
|
||||
use std::rc::Rc;
|
||||
use std::convert::From;
|
||||
|
||||
// Grammar
|
||||
// program := (statement delimiter ?)*
|
||||
// delimiter := Newline | Semicolon
|
||||
// statement := declaration | expression
|
||||
// declaration := FN prototype LCurlyBrace (statement)* RCurlyBrace
|
||||
// prototype := identifier LParen identlist RParen
|
||||
// identlist := Ident (Comma Ident)* | ε
|
||||
// exprlist := Expression (Comma Expression)* | ε
|
||||
// itemlist := Ident COLON Expression (Comma Ident COLON Expression)* | ε
|
||||
//
|
||||
// expression := postop_expression (op postop_expression)*
|
||||
// postop_expression := primary_expression postop
|
||||
// primary_expression := number_expr | String | identifier_expr | paren_expr | conditional_expr | while_expr | lambda_expr | list_expr | struct_expr
|
||||
// number_expr := (PLUS | MINUS ) number_expr | Number
|
||||
// identifier_expr := call_expression | Variable
|
||||
// list_expr := LSquareBracket exprlist RSquareBracket
|
||||
// struct_expr := LCurlyBrace itemlist RCurlyBrace
|
||||
// call_expression := Identifier LParen exprlist RParen
|
||||
// while_expr := WHILE primary_expression LCurlyBrace (expression delimiter)* RCurlyBrace
|
||||
// paren_expr := LParen expression RParen
|
||||
// conditional_expr := IF expression LCurlyBrace (expression delimiter)* RCurlyBrace (LCurlyBrace (expresion delimiter)* RCurlyBrace)?
|
||||
// lambda_expr := FN LParen identlist RParen LCurlyBrace (expression delimiter)* RCurlyBrace
|
||||
// lambda_call := | LParen exprlist RParen
|
||||
// postop := ε | LParen exprlist RParen | LBracket expression RBracket
|
||||
// op := '+', '-', etc.
|
||||
//
|
||||
|
||||
pub type AST = Vec<Statement>;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum Statement {
|
||||
ExprNode(Expression),
|
||||
FuncDefNode(Function),
|
||||
}
|
||||
|
||||
impl fmt::Display for Statement {
|
||||
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
|
||||
use self::Statement::*;
|
||||
match *self {
|
||||
ExprNode(ref expr) => write!(f, "{}", expr),
|
||||
FuncDefNode(_) => write!(f, "UNIMPLEMENTED"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Function {
|
||||
pub prototype: Prototype,
|
||||
pub body: Vec<Statement>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct Prototype {
|
||||
pub name: Rc<String>,
|
||||
pub parameters: Vec<Rc<String>>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum Expression {
|
||||
Null,
|
||||
StringLiteral(Rc<String>),
|
||||
Number(f64),
|
||||
Variable(Rc<String>),
|
||||
BinExp(BinOp, Box<Expression>, Box<Expression>),
|
||||
Call(Callable, Vec<Expression>),
|
||||
Conditional(Box<Expression>, Box<Expression>, Option<Box<Expression>>),
|
||||
Lambda(Function),
|
||||
Block(VecDeque<Expression>),
|
||||
While(Box<Expression>, Vec<Expression>),
|
||||
Index(Box<Expression>, Box<Expression>),
|
||||
ListLiteral(VecDeque<Expression>),
|
||||
StructLiteral(VecDeque<(Rc<String>, Expression)>),
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub enum Callable {
|
||||
NamedFunction(Rc<String>),
|
||||
Lambda(Function),
|
||||
}
|
||||
|
||||
//TODO this ought to be ReducedExpression
|
||||
impl fmt::Display for Expression {
|
||||
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
|
||||
use self::Expression::*;
|
||||
match *self {
|
||||
Null => write!(f, "null"),
|
||||
StringLiteral(ref s) => write!(f, "\"{}\"", s),
|
||||
Number(n) => write!(f, "{}", n),
|
||||
Lambda(Function { prototype: Prototype { ref name, ref parameters, .. }, .. }) => {
|
||||
write!(f, "«function: {}, {} arg(s)»", name, parameters.len())
|
||||
}
|
||||
ListLiteral(ref items) => {
|
||||
write!(f, "[ ")?;
|
||||
let mut iter = items.iter().peekable();
|
||||
while let Some(item) = iter.next() {
|
||||
write!(f, "{}", item)?;
|
||||
if let Some(_) = iter.peek() {
|
||||
write!(f, ", ")?;
|
||||
}
|
||||
}
|
||||
write!(f, " ]")
|
||||
}
|
||||
StructLiteral(ref items) => {
|
||||
write!(f, "{} ", "{")?;
|
||||
let mut iter = items.iter().peekable();
|
||||
while let Some(pair) = iter.next() {
|
||||
write!(f, "{}: {}", pair.0, pair.1)?;
|
||||
if let Some(_) = iter.peek() {
|
||||
write!(f, ", ")?;
|
||||
}
|
||||
}
|
||||
write!(f, "{} ", "}")
|
||||
}
|
||||
_ => write!(f, "UNIMPLEMENTED"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum BinOp {
|
||||
Add,
|
||||
AddAssign,
|
||||
Sub,
|
||||
SubAssign,
|
||||
Mul,
|
||||
MulAssign,
|
||||
Div,
|
||||
DivAssign,
|
||||
Mod,
|
||||
Less,
|
||||
LessEq,
|
||||
Greater,
|
||||
GreaterEq,
|
||||
Equal,
|
||||
Assign,
|
||||
Custom(String),
|
||||
}
|
||||
|
||||
impl From<OpTok> for BinOp {
|
||||
fn from(token: OpTok) -> BinOp {
|
||||
use self::BinOp::*;
|
||||
match &token.0[..] {
|
||||
"+" => Add,
|
||||
"+=" => AddAssign,
|
||||
"-" => Sub,
|
||||
"-=" => SubAssign,
|
||||
"*" => Mul,
|
||||
"*=" => MulAssign,
|
||||
"/" => Div,
|
||||
"/=" => DivAssign,
|
||||
"%" => Mod,
|
||||
"<" => Less,
|
||||
"<=" => LessEq,
|
||||
">" => Greater,
|
||||
">=" => GreaterEq,
|
||||
"==" => Equal,
|
||||
"=" => Assign,
|
||||
op => Custom(op.to_string()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
type Precedence = u8;
|
||||
|
||||
// TODO make this support incomplete parses
|
||||
pub type ParseResult<T> = Result<T, ParseError>;
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct ParseError {
|
||||
pub msg: String,
|
||||
pub remaining_tokens: Vec<Token>,
|
||||
}
|
||||
|
||||
impl ParseError {
|
||||
fn result_from_str<T>(msg: &str) -> ParseResult<T> {
|
||||
Err(ParseError {
|
||||
msg: msg.to_string(),
|
||||
remaining_tokens: vec![],
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
struct Parser {
|
||||
tokens: Vec<Token>,
|
||||
}
|
||||
|
||||
impl Parser {
|
||||
fn initialize(tokens: &[Token]) -> Parser {
|
||||
let mut tokens = tokens.to_vec();
|
||||
tokens.reverse();
|
||||
Parser { tokens: tokens }
|
||||
}
|
||||
|
||||
fn peek(&self) -> Option<Token> {
|
||||
self.tokens.last().map(|x| x.clone())
|
||||
}
|
||||
|
||||
fn next(&mut self) -> Option<Token> {
|
||||
self.tokens.pop()
|
||||
}
|
||||
|
||||
fn get_precedence(&self, op: &OpTok) -> Precedence {
|
||||
match &op.0[..] {
|
||||
"+" => 10,
|
||||
"-" => 10,
|
||||
"*" => 20,
|
||||
"/" => 20,
|
||||
"%" => 20,
|
||||
"==" => 40,
|
||||
"=" | "+=" | "-=" | "*=" | "/=" => 1,
|
||||
">" | ">=" | "<" | "<=" => 30,
|
||||
_ => 255,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
macro_rules! expect {
|
||||
($self_:expr, $token:pat) => {
|
||||
match $self_.peek() {
|
||||
Some($token) => {$self_.next();},
|
||||
Some(x) => {
|
||||
let err = format!("Expected `{:?}` but got `{:?}`", stringify!($token), x);
|
||||
return ParseError::result_from_str(&err)
|
||||
},
|
||||
None => {
|
||||
let err = format!("Expected `{:?}` but got end of input", stringify!($token));
|
||||
return ParseError::result_from_str(&err) //TODO make this not require 2 stringifications
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
macro_rules! expect_identifier {
|
||||
($self_:expr) => {
|
||||
match $self_.peek() {
|
||||
Some(Identifier(s)) => {$self_.next(); s},
|
||||
Some(x) => return ParseError::result_from_str(&format!("Expected identifier, but got {:?}", x)),
|
||||
None => return ParseError::result_from_str("Expected identifier, but got end of input"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
macro_rules! skip_whitespace {
|
||||
($_self: expr) => {
|
||||
loop {
|
||||
match $_self.peek() {
|
||||
Some(ref t) if is_delimiter(t) => {
|
||||
$_self.next();
|
||||
continue;
|
||||
}
|
||||
_ => break,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
macro_rules! delimiter_block {
|
||||
($_self: expr, $try_parse: ident, $($break_pattern: pat)|+) => {
|
||||
{
|
||||
let mut acc = Vec::new();
|
||||
loop {
|
||||
match $_self.peek() {
|
||||
None => break,
|
||||
Some(ref t) if is_delimiter(t) => { $_self.next(); continue; },
|
||||
$($break_pattern)|+ => break,
|
||||
_ => {
|
||||
let a = try!($_self.$try_parse());
|
||||
acc.push(a);
|
||||
}
|
||||
}
|
||||
}
|
||||
acc
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn is_delimiter(token: &Token) -> bool {
|
||||
match *token {
|
||||
Newline | Semicolon => true,
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
impl Parser {
|
||||
fn program(&mut self) -> ParseResult<AST> {
|
||||
let mut ast = Vec::new(); //TODO have this come from previously-parsed tree
|
||||
loop {
|
||||
let result: ParseResult<Statement> = match self.peek() {
|
||||
Some(ref t) if is_delimiter(t) => {
|
||||
self.next();
|
||||
continue;
|
||||
}
|
||||
Some(_) => self.statement(),
|
||||
None => break,
|
||||
};
|
||||
|
||||
match result {
|
||||
Ok(node) => ast.push(node),
|
||||
Err(mut err) => {
|
||||
err.remaining_tokens = self.tokens.clone();
|
||||
err.remaining_tokens.reverse();
|
||||
return Err(err);
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(ast)
|
||||
}
|
||||
|
||||
fn statement(&mut self) -> ParseResult<Statement> {
|
||||
let node: Statement = match self.peek() {
|
||||
Some(Keyword(Kw::Fn)) => self.declaration()?,
|
||||
Some(_) => Statement::ExprNode(self.expression()?),
|
||||
None => panic!("Unexpected end of tokens"),
|
||||
};
|
||||
Ok(node)
|
||||
}
|
||||
|
||||
fn declaration(&mut self) -> ParseResult<Statement> {
|
||||
expect!(self, Keyword(Kw::Fn));
|
||||
let prototype = self.prototype()?;
|
||||
expect!(self, LCurlyBrace);
|
||||
let body = self.body()?;
|
||||
expect!(self, RCurlyBrace);
|
||||
Ok(Statement::FuncDefNode(Function {
|
||||
prototype: prototype,
|
||||
body: body,
|
||||
}))
|
||||
}
|
||||
|
||||
fn prototype(&mut self) -> ParseResult<Prototype> {
|
||||
let name = expect_identifier!(self);
|
||||
expect!(self, LParen);
|
||||
let parameters = self.identlist()?;
|
||||
expect!(self, RParen);
|
||||
Ok(Prototype {
|
||||
name: name,
|
||||
parameters: parameters,
|
||||
})
|
||||
}
|
||||
|
||||
fn identlist(&mut self) -> ParseResult<Vec<Rc<String>>> {
|
||||
let mut args = Vec::new();
|
||||
while let Some(Identifier(name)) = self.peek() {
|
||||
args.push(name.clone());
|
||||
self.next();
|
||||
match self.peek() {
|
||||
Some(Comma) => {self.next();},
|
||||
_ => break,
|
||||
}
|
||||
}
|
||||
Ok(args)
|
||||
}
|
||||
|
||||
fn exprlist(&mut self) -> ParseResult<Vec<Expression>> {
|
||||
let mut exprs = Vec::new();
|
||||
loop {
|
||||
if let Some(RParen) = self.peek() {
|
||||
break;
|
||||
}
|
||||
let exp = self.expression()?;
|
||||
exprs.push(exp);
|
||||
match self.peek() {
|
||||
Some(Comma) => {self.next();},
|
||||
_ => break,
|
||||
}
|
||||
}
|
||||
Ok(exprs)
|
||||
}
|
||||
|
||||
fn itemlist(&mut self) -> ParseResult<VecDeque<(Rc<String>, Expression)>> {
|
||||
let mut items = VecDeque::new();
|
||||
loop {
|
||||
if let Some(RCurlyBrace) = self.peek() {
|
||||
break;
|
||||
}
|
||||
let name = expect_identifier!(self);
|
||||
expect!(self, Colon);
|
||||
let expr = self.expression()?;
|
||||
items.push_back((name, expr));
|
||||
match self.peek() {
|
||||
Some(Comma) => {self.next();},
|
||||
_ => break,
|
||||
};
|
||||
}
|
||||
Ok(items)
|
||||
}
|
||||
|
||||
fn body(&mut self) -> ParseResult<Vec<Statement>> {
|
||||
let statements = delimiter_block!(
|
||||
self,
|
||||
statement,
|
||||
Some(RCurlyBrace)
|
||||
);
|
||||
Ok(statements)
|
||||
}
|
||||
|
||||
fn expression(&mut self) -> ParseResult<Expression> {
|
||||
let lhs: Expression = self.postop_expression()?;
|
||||
self.precedence_expr(lhs, 0)
|
||||
}
|
||||
|
||||
fn precedence_expr(&mut self,
|
||||
mut lhs: Expression,
|
||||
min_precedence: u8)
|
||||
-> ParseResult<Expression> {
|
||||
while let Some(Operator(op)) = self.peek() {
|
||||
let precedence = self.get_precedence(&op);
|
||||
if precedence < min_precedence {
|
||||
break;
|
||||
}
|
||||
self.next();
|
||||
let mut rhs = self.postop_expression()?;
|
||||
while let Some(Operator(ref op)) = self.peek() {
|
||||
if self.get_precedence(op) > precedence {
|
||||
let new_prec = self.get_precedence(op);
|
||||
rhs = self.precedence_expr(rhs, new_prec)?;
|
||||
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
lhs = Expression::BinExp(op.into(), Box::new(lhs), Box::new(rhs));
|
||||
}
|
||||
Ok(lhs)
|
||||
}
|
||||
|
||||
fn postop_expression(&mut self) -> ParseResult<Expression> {
|
||||
use self::Expression::*;
|
||||
let expr = self.primary_expression()?;
|
||||
let ret = match self.peek() {
|
||||
Some(LParen) => {
|
||||
let args = self.call_expression()?;
|
||||
match expr {
|
||||
Lambda(f) => Call(Callable::Lambda(f), args),
|
||||
e => {
|
||||
let err = format!("Expected lambda expression before a call, got {:?}", e);
|
||||
return ParseError::result_from_str(&err);
|
||||
},
|
||||
}
|
||||
},
|
||||
Some(LSquareBracket) => {
|
||||
expect!(self, LSquareBracket);
|
||||
let index_expr = self.expression()?;
|
||||
expect!(self, RSquareBracket);
|
||||
Index(Box::new(expr), Box::new(index_expr))
|
||||
},
|
||||
_ => {
|
||||
expr
|
||||
}
|
||||
};
|
||||
Ok(ret)
|
||||
}
|
||||
|
||||
fn primary_expression(&mut self) -> ParseResult<Expression> {
|
||||
Ok(match self.peek() {
|
||||
Some(Keyword(Kw::Null)) => {
|
||||
self.next();
|
||||
Expression::Null
|
||||
}
|
||||
Some(NumLiteral(_)) => self.number_expression()?,
|
||||
Some(Operator(OpTok(ref a))) if **a == "+" || **a == "-" => self.number_expression()?,
|
||||
Some(StrLiteral(s)) => {
|
||||
self.next();
|
||||
Expression::StringLiteral(s)
|
||||
}
|
||||
Some(Keyword(Kw::If)) => self.conditional_expr()?,
|
||||
Some(Keyword(Kw::While)) => self.while_expr()?,
|
||||
Some(Identifier(_)) => self.identifier_expr()?,
|
||||
Some(Token::LParen) => self.paren_expr()?,
|
||||
Some(Keyword(Kw::Fn)) => self.lambda_expr()?,
|
||||
Some(Token::LSquareBracket) => self.list_expr()?,
|
||||
Some(Token::LCurlyBrace) => self.struct_expr()?,
|
||||
Some(e) => {
|
||||
return ParseError::result_from_str(&format!("Expected primary expression, got \
|
||||
{:?}",
|
||||
e));
|
||||
}
|
||||
None => return ParseError::result_from_str("Expected primary expression received EoI"),
|
||||
})
|
||||
}
|
||||
|
||||
fn list_expr(&mut self) -> ParseResult<Expression> {
|
||||
expect!(self, LSquareBracket);
|
||||
let exprlist: Vec<Expression> = self.exprlist()?;
|
||||
expect!(self, RSquareBracket);
|
||||
|
||||
Ok(Expression::ListLiteral(VecDeque::from(exprlist)))
|
||||
}
|
||||
|
||||
fn struct_expr(&mut self) -> ParseResult<Expression> {
|
||||
expect!(self, LCurlyBrace);
|
||||
let struct_items = self.itemlist()?;
|
||||
expect!(self, RCurlyBrace);
|
||||
Ok(Expression::StructLiteral(struct_items))
|
||||
}
|
||||
|
||||
fn number_expression(&mut self) -> ParseResult<Expression> {
|
||||
let mut multiplier = 1;
|
||||
loop {
|
||||
match self.peek() {
|
||||
Some(NumLiteral(n)) => {
|
||||
self.next();
|
||||
return Ok(Expression::Number(n * multiplier as f64));
|
||||
}
|
||||
Some(Operator(OpTok(ref a))) if **a == "+" => {
|
||||
self.next();
|
||||
}
|
||||
Some(Operator(OpTok(ref a))) if **a == "-" => {
|
||||
multiplier *= -1;
|
||||
self.next();
|
||||
}
|
||||
Some(e) => {
|
||||
return ParseError::result_from_str(
|
||||
&format!("Expected +, - or number, got {:?}", e));
|
||||
}
|
||||
None => {
|
||||
return ParseError::result_from_str(
|
||||
&format!("Expected +, - or number, got EoI"));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn lambda_expr(&mut self) -> ParseResult<Expression> {
|
||||
use self::Expression::*;
|
||||
expect!(self, Keyword(Kw::Fn));
|
||||
skip_whitespace!(self);
|
||||
expect!(self, LParen);
|
||||
let parameters = self.identlist()?;
|
||||
expect!(self, RParen);
|
||||
skip_whitespace!(self);
|
||||
expect!(self, LCurlyBrace);
|
||||
let body = self.body()?;
|
||||
expect!(self, RCurlyBrace);
|
||||
|
||||
let prototype = Prototype {
|
||||
name: Rc::new("a lambda yo!".to_string()),
|
||||
parameters: parameters,
|
||||
};
|
||||
|
||||
let function = Function {
|
||||
prototype: prototype,
|
||||
body: body,
|
||||
};
|
||||
|
||||
Ok(Lambda(function))
|
||||
}
|
||||
|
||||
fn while_expr(&mut self) -> ParseResult<Expression> {
|
||||
use self::Expression::*;
|
||||
expect!(self, Keyword(Kw::While));
|
||||
let test = self.expression()?;
|
||||
expect!(self, LCurlyBrace);
|
||||
let body = delimiter_block!(
|
||||
self,
|
||||
expression,
|
||||
Some(RCurlyBrace)
|
||||
);
|
||||
expect!(self, RCurlyBrace);
|
||||
Ok(While(Box::new(test), body))
|
||||
}
|
||||
|
||||
fn conditional_expr(&mut self) -> ParseResult<Expression> {
|
||||
use self::Expression::*;
|
||||
expect!(self, Keyword(Kw::If));
|
||||
let test = self.expression()?;
|
||||
skip_whitespace!(self);
|
||||
expect!(self, LCurlyBrace);
|
||||
skip_whitespace!(self);
|
||||
let then_block = delimiter_block!(
|
||||
self,
|
||||
expression,
|
||||
Some(RCurlyBrace)
|
||||
);
|
||||
expect!(self, RCurlyBrace);
|
||||
skip_whitespace!(self);
|
||||
let else_block = if let Some(Keyword(Kw::Else)) = self.peek() {
|
||||
self.next();
|
||||
skip_whitespace!(self);
|
||||
expect!(self, LCurlyBrace);
|
||||
let else_exprs = delimiter_block!(
|
||||
self,
|
||||
expression,
|
||||
Some(RCurlyBrace)
|
||||
);
|
||||
Some(else_exprs)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
expect!(self, RCurlyBrace);
|
||||
Ok(Conditional(Box::new(test),
|
||||
Box::new(Block(VecDeque::from(then_block))),
|
||||
else_block.map(|list| Box::new(Block(VecDeque::from(list))))))
|
||||
}
|
||||
|
||||
fn identifier_expr(&mut self) -> ParseResult<Expression> {
|
||||
let name = expect_identifier!(self);
|
||||
let expr = match self.peek() {
|
||||
Some(LParen) => {
|
||||
let args = self.call_expression()?;
|
||||
Expression::Call(Callable::NamedFunction(name), args)
|
||||
}
|
||||
__ => Expression::Variable(name),
|
||||
};
|
||||
Ok(expr)
|
||||
}
|
||||
|
||||
fn call_expression(&mut self) -> ParseResult<Vec<Expression>> {
|
||||
expect!(self, LParen);
|
||||
let args: Vec<Expression> = self.exprlist()?;
|
||||
expect!(self, RParen);
|
||||
Ok(args)
|
||||
}
|
||||
|
||||
fn paren_expr(&mut self) -> ParseResult<Expression> {
|
||||
expect!(self, Token::LParen);
|
||||
let expr = self.expression()?;
|
||||
expect!(self, Token::RParen);
|
||||
Ok(expr)
|
||||
}
|
||||
}
|
||||
|
||||
pub fn parse(tokens: &[Token], _parsed_tree: &[Statement]) -> ParseResult<AST> {
|
||||
let mut parser = Parser::initialize(tokens);
|
||||
parser.program()
|
||||
}
|
||||
|
||||
/*
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use schala_lang::tokenizer;
|
||||
use super::*;
|
||||
use super::Statement::*;
|
||||
use super::Expression::*;
|
||||
|
||||
macro_rules! parsetest {
|
||||
($input:expr, $output:pat, $ifexpr:expr) => {
|
||||
{
|
||||
let tokens = tokenizer::tokenize($input).unwrap();
|
||||
let ast = parse(&tokens, &[]).unwrap();
|
||||
match &ast[..] {
|
||||
$output if $ifexpr => (),
|
||||
x => panic!("Error in parse test, got {:?} instead", x)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn function_parse_test() {
|
||||
use super::Function;
|
||||
parsetest!(
|
||||
"fn a() { 1 + 2 }",
|
||||
&[FuncDefNode(Function {prototype: Prototype { ref name, ref parameters }, ref body})],
|
||||
match &body[..] { &[ExprNode(BinExp(_, box Number(1.0), box Number(2.0)))] => true, _ => false }
|
||||
&& **name == "a" && match ¶meters[..] { &[] => true, _ => false }
|
||||
);
|
||||
|
||||
parsetest!(
|
||||
"fn a(x,y){ 1 + 2 }",
|
||||
&[FuncDefNode(Function {prototype: Prototype { ref name, ref parameters }, ref body})],
|
||||
match &body[..] { &[ExprNode(BinExp(_, box Number(1.0), box Number(2.0)))] => true, _ => false }
|
||||
&& **name == "a" && *parameters[0] == "x" && *parameters[1] == "y" && parameters.len() == 2
|
||||
);
|
||||
|
||||
let t3 = "fn (x) { x + 2 }";
|
||||
let tokens3 = tokenizer::tokenize(t3).unwrap();
|
||||
assert!(parse(&tokens3, &[]).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn expression_parse_test() {
|
||||
parsetest!("a", &[ExprNode(Variable(ref s))], **s == "a");
|
||||
parsetest!("a + b",
|
||||
&[ExprNode(BinExp(BinOp::Add, box Variable(ref a), box Variable(ref b)))],
|
||||
**a == "a" && **b == "b");
|
||||
parsetest!("a + b * c",
|
||||
&[ExprNode(BinExp(BinOp::Add, box Variable(ref a), box BinExp(BinOp::Mul, box Variable(ref b), box Variable(ref c))))],
|
||||
**a == "a" && **b == "b" && **c == "c");
|
||||
parsetest!("a * b + c",
|
||||
&[ExprNode(BinExp(BinOp::Add, box BinExp(BinOp::Mul, box Variable(ref a), box Variable(ref b)), box Variable(ref c)))],
|
||||
**a == "a" && **b == "b" && **c == "c");
|
||||
parsetest!("(a + b) * c",
|
||||
&[ExprNode(BinExp(BinOp::Mul, box BinExp(BinOp::Add, box Variable(ref a), box Variable(ref b)), box Variable(ref c)))],
|
||||
**a == "a" && **b == "b" && **c == "c");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lambda_parse_test() {
|
||||
use schala_lang::tokenizer;
|
||||
let t1 = "(fn(x) { x + 2 })";
|
||||
let tokens1 = tokenizer::tokenize(t1).unwrap();
|
||||
match parse(&tokens1, &[]).unwrap()[..] {
|
||||
_ => (),
|
||||
}
|
||||
|
||||
let t2 = "fn(x) { x + 2 }";
|
||||
let tokens2 = tokenizer::tokenize(t2).unwrap();
|
||||
assert!(parse(&tokens2, &[]).is_err());
|
||||
|
||||
let t3 = "(fn(x) { x + 10 })(20)";
|
||||
let tokens3 = tokenizer::tokenize(t3).unwrap();
|
||||
match parse(&tokens3, &[]).unwrap() {
|
||||
_ => (),
|
||||
};
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn conditional_parse_test() {
|
||||
use schala_lang::tokenizer;
|
||||
let t1 = "if null { 20 } else { 40 }";
|
||||
let tokens = tokenizer::tokenize(t1).unwrap();
|
||||
match parse(&tokens, &[]).unwrap()[..] {
|
||||
[ExprNode(Conditional(box Null, box Block(_), Some(box Block(_))))] => (),
|
||||
_ => panic!(),
|
||||
}
|
||||
|
||||
let t2 = r"
|
||||
if null {
|
||||
20
|
||||
} else {
|
||||
40
|
||||
}
|
||||
";
|
||||
let tokens2 = tokenizer::tokenize(t2).unwrap();
|
||||
match parse(&tokens2, &[]).unwrap()[..] {
|
||||
[ExprNode(Conditional(box Null, box Block(_), Some(box Block(_))))] => (),
|
||||
_ => panic!(),
|
||||
}
|
||||
|
||||
let t2 = r"
|
||||
if null {
|
||||
20 } else
|
||||
{
|
||||
40
|
||||
}
|
||||
";
|
||||
let tokens3 = tokenizer::tokenize(t2).unwrap();
|
||||
match parse(&tokens3, &[]).unwrap()[..] {
|
||||
[ExprNode(Conditional(box Null, box Block(_), Some(box Block(_))))] => (),
|
||||
_ => panic!(),
|
||||
}
|
||||
}
|
||||
}
|
||||
*/
|
|
@ -1,208 +0,0 @@
|
|||
extern crate itertools;
|
||||
|
||||
use std::iter::Peekable;
|
||||
use std::str::Chars;
|
||||
use self::itertools::Itertools;
|
||||
use std::rc::Rc;
|
||||
|
||||
use maaru_lang::TokenError;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum Token {
|
||||
Newline,
|
||||
Semicolon,
|
||||
LParen,
|
||||
RParen,
|
||||
LSquareBracket,
|
||||
RSquareBracket,
|
||||
LCurlyBrace,
|
||||
RCurlyBrace,
|
||||
Comma,
|
||||
Period,
|
||||
Colon,
|
||||
NumLiteral(f64),
|
||||
StrLiteral(Rc<String>),
|
||||
Identifier(Rc<String>),
|
||||
Operator(OpTok),
|
||||
Keyword(Kw),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct OpTok(pub Rc<String>);
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum Kw {
|
||||
If,
|
||||
Else,
|
||||
While,
|
||||
Let,
|
||||
Fn,
|
||||
Null,
|
||||
}
|
||||
|
||||
pub type TokenizeResult = Result<Vec<Token>, TokenError>;
|
||||
|
||||
fn is_digit(c: &char) -> bool {
|
||||
c.is_digit(10)
|
||||
}
|
||||
|
||||
pub fn tokenize(input: &str) -> TokenizeResult {
|
||||
use self::Token::*;
|
||||
let mut tokens = Vec::new();
|
||||
let mut iter: Peekable<Chars> = input.chars().peekable();
|
||||
while let Some(c) = iter.next() {
|
||||
if c == '#' {
|
||||
while let Some(c) = iter.next() {
|
||||
if c == '\n' {
|
||||
break;
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
let cur_tok = match c {
|
||||
c if char::is_whitespace(c) && c != '\n' => continue,
|
||||
'\n' => Newline,
|
||||
';' => Semicolon,
|
||||
'(' => LParen,
|
||||
')' => RParen,
|
||||
':' => Colon,
|
||||
',' => Comma,
|
||||
'{' => LCurlyBrace,
|
||||
'}' => RCurlyBrace,
|
||||
'[' => LSquareBracket,
|
||||
']' => RSquareBracket,
|
||||
'"' => tokenize_str(&mut iter)?,
|
||||
c if !char::is_alphanumeric(c) => tokenize_operator(c, &mut iter)?,
|
||||
c @ '.' | c if is_digit(&c) => tokenize_number_or_period(c, &mut iter)?,
|
||||
c => tokenize_identifier(c, &mut iter)?,
|
||||
};
|
||||
tokens.push(cur_tok);
|
||||
}
|
||||
Ok(tokens)
|
||||
}
|
||||
|
||||
fn tokenize_str(iter: &mut Peekable<Chars>) -> Result<Token, TokenError> {
|
||||
let mut buffer = String::new();
|
||||
loop {
|
||||
// TODO handle string escapes, interpolation
|
||||
match iter.next() {
|
||||
Some(x) if x == '"' => break,
|
||||
Some(x) => buffer.push(x),
|
||||
None => return Err(TokenError::new("Unclosed quote")),
|
||||
}
|
||||
}
|
||||
Ok(Token::StrLiteral(Rc::new(buffer)))
|
||||
}
|
||||
|
||||
fn tokenize_operator(c: char, iter: &mut Peekable<Chars>) -> Result<Token, TokenError> {
|
||||
let mut buffer = String::new();
|
||||
buffer.push(c);
|
||||
buffer.extend(iter.peeking_take_while(|x| !char::is_alphanumeric(*x) && !char::is_whitespace(*x)));
|
||||
Ok(Token::Operator(OpTok(Rc::new(buffer))))
|
||||
}
|
||||
|
||||
fn tokenize_number_or_period(c: char, iter: &mut Peekable<Chars>) -> Result<Token, TokenError> {
|
||||
if c == '.' && !iter.peek().map_or(false, is_digit) {
|
||||
return Ok(Token::Period);
|
||||
}
|
||||
|
||||
let mut buffer = String::new();
|
||||
buffer.push(c);
|
||||
buffer.extend(iter.peeking_take_while(|x| is_digit(x) || *x == '.'));
|
||||
|
||||
match buffer.parse::<f64>() {
|
||||
Ok(f) => Ok(Token::NumLiteral(f)),
|
||||
Err(_) => Err(TokenError::new("Failed to parse digit")),
|
||||
}
|
||||
}
|
||||
|
||||
fn tokenize_identifier(c: char, iter: &mut Peekable<Chars>) -> Result<Token, TokenError> {
|
||||
fn ends_identifier(c: &char) -> bool {
|
||||
let c = *c;
|
||||
char::is_whitespace(c) || is_digit(&c) || c == ';' || c == '(' || c == ')' ||
|
||||
c == ',' || c == '.' || c == ',' || c == ':' || c == '[' || c == ']'
|
||||
}
|
||||
|
||||
use self::Token::*;
|
||||
let mut buffer = String::new();
|
||||
buffer.push(c);
|
||||
buffer.extend(iter.peeking_take_while(|x| !ends_identifier(x)));
|
||||
|
||||
Ok(match &buffer[..] {
|
||||
"if" => Keyword(Kw::If),
|
||||
"else" => Keyword(Kw::Else),
|
||||
"while" => Keyword(Kw::While),
|
||||
"let" => Keyword(Kw::Let),
|
||||
"fn" => Keyword(Kw::Fn),
|
||||
"null" => Keyword(Kw::Null),
|
||||
b => Identifier(Rc::new(b.to_string())),
|
||||
})
|
||||
}
|
||||
|
||||
/*
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use super::Token::*;
|
||||
|
||||
macro_rules! token_test {
|
||||
($input: expr, $output: pat, $ifexpr: expr) => {
|
||||
let tokens = tokenize($input).unwrap();
|
||||
match tokens[..] {
|
||||
$output if $ifexpr => (),
|
||||
_ => panic!("Actual output: {:?}", tokens),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn basic_tokeniziation_tests() {
|
||||
token_test!("let a = 3\n",
|
||||
[Keyword(Kw::Let), Identifier(ref a), Operator(OpTok(ref b)), NumLiteral(3.0), Newline],
|
||||
**a == "a" && **b == "=");
|
||||
|
||||
token_test!("2+1",
|
||||
[NumLiteral(2.0), Operator(OpTok(ref a)), NumLiteral(1.0)],
|
||||
**a == "+");
|
||||
|
||||
token_test!("2 + 1",
|
||||
[NumLiteral(2.0), Operator(OpTok(ref a)), NumLiteral(1.0)],
|
||||
**a == "+");
|
||||
|
||||
token_test!("2.3*49.2",
|
||||
[NumLiteral(2.3), Operator(OpTok(ref a)), NumLiteral(49.2)],
|
||||
**a == "*");
|
||||
|
||||
token_test!("a+3",
|
||||
[Identifier(ref a), NumLiteral(3.0)],
|
||||
**a == "a+");
|
||||
|
||||
assert!(tokenize("2.4.5").is_err());
|
||||
|
||||
token_test!("fn my_func(a) { a ? 3[1] }",
|
||||
[Keyword(Kw::Fn), Identifier(ref a), LParen, Identifier(ref b), RParen, LCurlyBrace, Identifier(ref c),
|
||||
Operator(OpTok(ref d)), NumLiteral(3.0), LSquareBracket, NumLiteral(1.0), RSquareBracket, RCurlyBrace],
|
||||
**a == "my_func" && **b == "a" && **c == "a" && **d == "?");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn string_test() {
|
||||
token_test!("null + \"a string\"",
|
||||
[Keyword(Kw::Null), Operator(OpTok(ref a)), StrLiteral(ref b)],
|
||||
**a == "+" && **b == "a string");
|
||||
|
||||
token_test!("\"{?'q@?\"",
|
||||
[StrLiteral(ref a)],
|
||||
**a == "{?'q@?");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn operator_test() {
|
||||
token_test!("a *> b",
|
||||
[Identifier(ref a), Operator(OpTok(ref b)), Identifier(ref c)],
|
||||
**a == "a" && **b == "*>" && **c == "b");
|
||||
|
||||
|
||||
}
|
||||
}
|
||||
*/
|
29
src/main.rs
29
src/main.rs
|
@ -1,29 +0,0 @@
|
|||
#![feature(advanced_slice_patterns, slice_patterns, box_patterns, box_syntax)]
|
||||
#![feature(plugin)]
|
||||
extern crate itertools;
|
||||
#[macro_use]
|
||||
extern crate lazy_static;
|
||||
#[macro_use]
|
||||
extern crate maplit;
|
||||
|
||||
mod schala_lang;
|
||||
mod maaru_lang;
|
||||
mod robo_lang;
|
||||
mod rukka_lang;
|
||||
|
||||
extern crate schala_lib;
|
||||
use schala_lib::{PLIGenerator, schala_main};
|
||||
|
||||
extern { }
|
||||
|
||||
fn main() {
|
||||
let generators: Vec<PLIGenerator> = vec![
|
||||
Box::new(|| { Box::new(schala_lang::Schala::new())}),
|
||||
Box::new(|| { Box::new(schala_lang::autoparser::Schala::new())}),
|
||||
Box::new(|| { Box::new(maaru_lang::Maaru::new())}),
|
||||
Box::new(|| { Box::new(robo_lang::Robo::new())}),
|
||||
Box::new(|| { Box::new(rukka_lang::Rukka::new())}),
|
||||
];
|
||||
schala_main(generators);
|
||||
}
|
||||
|
|
@ -1,167 +0,0 @@
|
|||
use itertools::Itertools;
|
||||
use schala_lib::{ProgrammingLanguageInterface, EvalOptions, LanguageOutput};
|
||||
|
||||
pub struct Robo {
|
||||
}
|
||||
|
||||
impl Robo {
|
||||
pub fn new() -> Robo {
|
||||
Robo { }
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct TokenError {
|
||||
pub msg: String,
|
||||
}
|
||||
|
||||
impl TokenError {
|
||||
pub fn new(msg: &str) -> TokenError {
|
||||
TokenError { msg: msg.to_string() }
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[derive(Debug)]
|
||||
pub enum Token {
|
||||
StrLiteral(String),
|
||||
Backtick,
|
||||
Newline,
|
||||
LParen,
|
||||
RParen,
|
||||
LBracket,
|
||||
RBracket,
|
||||
LBrace,
|
||||
RBrace,
|
||||
Period,
|
||||
Comma,
|
||||
Colon,
|
||||
Semicolon,
|
||||
SingleQuote,
|
||||
Identifier(String),
|
||||
Operator(String),
|
||||
NumLiteral(Number),
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[derive(Debug)]
|
||||
pub enum Number {
|
||||
IntegerRep(String),
|
||||
FloatRep(String)
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub type AST = Vec<ASTNode>;
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[derive(Debug)]
|
||||
pub enum ASTNode {
|
||||
FunctionDefinition(String, Expression),
|
||||
ImportStatement(String),
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[derive(Debug)]
|
||||
pub enum Expression {
|
||||
|
||||
}
|
||||
|
||||
fn tokenize(input: &str) -> Result<Vec<Token>, TokenError> {
|
||||
use self::Token::*;
|
||||
let mut tokens = Vec::new();
|
||||
let mut iter = input.chars().peekable();
|
||||
while let Some(c) = iter.next() {
|
||||
if c == ';' {
|
||||
while let Some(c) = iter.next() {
|
||||
if c == '\n' {
|
||||
break;
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
let cur_tok = match c {
|
||||
c if char::is_whitespace(c) && c != '\n' => continue,
|
||||
'\n' => Newline,
|
||||
'(' => LParen,
|
||||
')' => RParen,
|
||||
'[' => LBracket,
|
||||
']' => RBracket,
|
||||
'{' => LBrace,
|
||||
'}' => RBrace,
|
||||
',' => Comma,
|
||||
':' => Colon,
|
||||
';' => Semicolon,
|
||||
'.' => Period,
|
||||
'`' => Backtick,
|
||||
'\'' => SingleQuote,
|
||||
'"' => {
|
||||
let mut buffer = String::new();
|
||||
loop {
|
||||
match iter.next() {
|
||||
Some(x) if x == '"' => break,
|
||||
Some(x) => buffer.push(x),
|
||||
None => return Err(TokenError::new("Unclosed quote")),
|
||||
}
|
||||
}
|
||||
StrLiteral(buffer)
|
||||
}
|
||||
c if c.is_digit(10) => {
|
||||
let mut integer = true;
|
||||
let mut buffer = String::new();
|
||||
buffer.push(c);
|
||||
buffer.extend(iter.peeking_take_while(|x| x.is_digit(10)));
|
||||
if let Some(&'.') = iter.peek() {
|
||||
buffer.push(iter.next().unwrap());
|
||||
integer = false;
|
||||
}
|
||||
buffer.extend(iter.peeking_take_while(|x| x.is_digit(10)));
|
||||
let inner = if integer {
|
||||
Number::IntegerRep(buffer)
|
||||
} else {
|
||||
Number::FloatRep(buffer)
|
||||
};
|
||||
NumLiteral(inner)
|
||||
},
|
||||
c if char::is_alphanumeric(c) => {
|
||||
let mut buffer = String::new();
|
||||
buffer.push(c);
|
||||
buffer.extend(iter.peeking_take_while(|x| char::is_alphanumeric(*x)));
|
||||
Identifier(buffer)
|
||||
},
|
||||
c => {
|
||||
let mut buffer = String::new();
|
||||
buffer.push(c);
|
||||
buffer.extend(iter.peeking_take_while(|x| !char::is_whitespace(*x)));
|
||||
Operator(buffer)
|
||||
}
|
||||
};
|
||||
tokens.push(cur_tok);
|
||||
}
|
||||
|
||||
Ok(tokens)
|
||||
}
|
||||
|
||||
impl ProgrammingLanguageInterface for Robo {
|
||||
fn get_language_name(&self) -> String {
|
||||
"Robo".to_string()
|
||||
}
|
||||
|
||||
fn get_source_file_suffix(&self) -> String {
|
||||
format!("robo")
|
||||
}
|
||||
|
||||
fn evaluate_in_repl(&mut self, input: &str, _eval_options: &EvalOptions) -> LanguageOutput {
|
||||
let mut output = LanguageOutput::default();
|
||||
let tokens = match tokenize(input) {
|
||||
Ok(tokens) => tokens,
|
||||
Err(e) => {
|
||||
output.add_output(format!("Tokenize error: {:?}", e));
|
||||
return output;
|
||||
}
|
||||
};
|
||||
|
||||
output.add_output(format!("{:?}", tokens));
|
||||
output
|
||||
}
|
||||
}
|
||||
|
|
@ -1,432 +0,0 @@
|
|||
use itertools::Itertools;
|
||||
use schala_lib::{ProgrammingLanguageInterface, EvalOptions, LanguageOutput};
|
||||
use std::iter::Peekable;
|
||||
use std::vec::IntoIter;
|
||||
use std::str::Chars;
|
||||
use std::collections::HashMap;
|
||||
|
||||
pub struct EvaluatorState {
|
||||
binding_stack: Vec<HashMap<String, Sexp>>
|
||||
}
|
||||
|
||||
impl EvaluatorState {
|
||||
fn new() -> EvaluatorState {
|
||||
use self::Sexp::Primitive;
|
||||
use self::PrimitiveFn::*;
|
||||
let mut default_map = HashMap::new();
|
||||
default_map.insert(format!("+"), Primitive(Plus));
|
||||
default_map.insert(format!("-"), Primitive(Minus));
|
||||
default_map.insert(format!("*"), Primitive(Mult));
|
||||
default_map.insert(format!("/"), Primitive(Div));
|
||||
default_map.insert(format!("%"), Primitive(Mod));
|
||||
default_map.insert(format!(">"), Primitive(Greater));
|
||||
default_map.insert(format!("<"), Primitive(Less));
|
||||
default_map.insert(format!("<="), Primitive(LessThanOrEqual));
|
||||
default_map.insert(format!(">="), Primitive(GreaterThanOrEqual));
|
||||
default_map.insert(format!("display"), Primitive(Display));
|
||||
|
||||
EvaluatorState {
|
||||
binding_stack: vec![default_map],
|
||||
}
|
||||
}
|
||||
fn set_var(&mut self, var: String, value: Sexp) {
|
||||
let binding = self.binding_stack.last_mut().unwrap();
|
||||
binding.insert(var, value);
|
||||
}
|
||||
fn get_var(&self, var: &str) -> Option<&Sexp> {
|
||||
for bindings in self.binding_stack.iter().rev() {
|
||||
match bindings.get(var) {
|
||||
Some(x) => return Some(x),
|
||||
None => (),
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn push_env(&mut self) {
|
||||
self.binding_stack.push(HashMap::new());
|
||||
}
|
||||
fn pop_env(&mut self) {
|
||||
self.binding_stack.pop();
|
||||
}
|
||||
}
|
||||
|
||||
pub struct Rukka {
|
||||
state: EvaluatorState
|
||||
}
|
||||
|
||||
impl Rukka {
|
||||
pub fn new() -> Rukka { Rukka { state: EvaluatorState::new() } }
|
||||
}
|
||||
|
||||
impl ProgrammingLanguageInterface for Rukka {
|
||||
fn get_language_name(&self) -> String {
|
||||
"Rukka".to_string()
|
||||
}
|
||||
|
||||
fn get_source_file_suffix(&self) -> String {
|
||||
format!("rukka")
|
||||
}
|
||||
|
||||
fn evaluate_in_repl(&mut self, input: &str, _eval_options: &EvalOptions) -> LanguageOutput {
|
||||
let mut output = LanguageOutput::default();
|
||||
let sexps = match read(input) {
|
||||
Err(err) => {
|
||||
output.add_output(format!("Error: {}", err));
|
||||
return output;
|
||||
},
|
||||
Ok(sexps) => sexps
|
||||
};
|
||||
|
||||
let output_str: String = sexps.into_iter().enumerate().map(|(i, sexp)| {
|
||||
match self.state.eval(sexp) {
|
||||
Ok(result) => format!("{}: {}", i, result.print()),
|
||||
Err(err) => format!("{} Error: {}", i, err),
|
||||
}
|
||||
}).intersperse(format!("\n")).collect();
|
||||
output.add_output(output_str);
|
||||
output
|
||||
}
|
||||
}
|
||||
|
||||
impl EvaluatorState {
|
||||
fn eval(&mut self, expr: Sexp) -> Result<Sexp, String> {
|
||||
use self::Sexp::*;
|
||||
Ok(match expr {
|
||||
SymbolAtom(ref sym) => match self.get_var(sym) {
|
||||
Some(ref sexp) => {
|
||||
let q: &Sexp = sexp; //WTF? if I delete this line, the copy doesn't work??
|
||||
q.clone() //TODO make this not involve a clone
|
||||
},
|
||||
None => return Err(format!("Variable {} not bound", sym)),
|
||||
},
|
||||
expr @ Primitive(_) => expr,
|
||||
expr @ FnLiteral { .. } => expr,
|
||||
expr @ StringAtom(_) => expr,
|
||||
expr @ NumberAtom(_) => expr,
|
||||
expr @ BoolAtom(_) => expr,
|
||||
Cons(box operator, box operands) => match operator {
|
||||
SymbolAtom(ref sym) if match &sym[..] {
|
||||
"quote" | "eq?" | "cons" | "car" | "cdr" | "atom?" | "define" | "lambda" | "if" | "cond" => true, _ => false
|
||||
} => self.eval_special_form(sym, operands)?,
|
||||
_ => {
|
||||
let evaled = self.eval(operator)?;
|
||||
self.apply(evaled, operands)?
|
||||
}
|
||||
},
|
||||
Nil => Nil,
|
||||
})
|
||||
}
|
||||
fn eval_special_form(&mut self, form: &str, operands: Sexp) -> Result<Sexp, String> {
|
||||
use self::Sexp::*;
|
||||
Ok(match form {
|
||||
"quote" => match operands {
|
||||
Cons(box quoted, box Nil) => quoted,
|
||||
_ => return Err(format!("Bad syntax in quote")),
|
||||
},
|
||||
"eq?" => match operands {//TODO make correct
|
||||
Cons(box lhs, box Cons(box rhs, _)) => BoolAtom(lhs == rhs),
|
||||
_ => BoolAtom(true),
|
||||
},
|
||||
"cons" => match operands {
|
||||
Cons(box cadr, box Cons(box caddr, box Nil)) => {
|
||||
let newl = self.eval(cadr)?;
|
||||
let newr = self.eval(caddr)?;
|
||||
Cons(Box::new(newl), Box::new(newr))
|
||||
},
|
||||
_ => return Err(format!("Bad arguments for cons")),
|
||||
},
|
||||
"car" => match operands {
|
||||
Cons(box car, _) => car,
|
||||
_ => return Err(format!("called car with a non-pair argument")),
|
||||
},
|
||||
"cdr" => match operands {
|
||||
Cons(_, box cdr) => cdr,
|
||||
_ => return Err(format!("called cdr with a non-pair argument")),
|
||||
},
|
||||
"atom?" => match operands {
|
||||
Cons(_, _) => BoolAtom(false),
|
||||
_ => BoolAtom(true),
|
||||
},
|
||||
"define" => match operands {
|
||||
Cons(box SymbolAtom(sym), box Cons(box expr, box Nil)) => {
|
||||
let evaluated = self.eval(expr)?;
|
||||
self.set_var(sym, evaluated);
|
||||
Nil
|
||||
},
|
||||
_ => return Err(format!("Bad assignment")),
|
||||
}
|
||||
"lambda" => match operands {
|
||||
Cons(box mut paramlist, box Cons(box formalexp, box Nil)) => {
|
||||
let mut formal_params = vec![];
|
||||
{
|
||||
let mut ptr = ¶mlist;
|
||||
loop {
|
||||
match ptr {
|
||||
&Cons(ref arg, ref rest) => {
|
||||
if let SymbolAtom(ref sym) = **arg {
|
||||
formal_params.push(sym.clone());
|
||||
ptr = rest;
|
||||
} else {
|
||||
return Err(format!("Bad lambda format"));
|
||||
}
|
||||
},
|
||||
_ => break,
|
||||
}
|
||||
}
|
||||
}
|
||||
FnLiteral {
|
||||
formal_params,
|
||||
body: Box::new(formalexp)
|
||||
}
|
||||
},
|
||||
_ => return Err(format!("Bad lambda expression")),
|
||||
},
|
||||
"if" => match operands {
|
||||
Cons(box test, box body) => {
|
||||
let truth_value = test.truthy();
|
||||
match (truth_value, body) {
|
||||
(true, Cons(box consequent, _)) => consequent,
|
||||
(false, Cons(_, box Cons(box alternative, _))) => alternative,
|
||||
_ => return Err(format!("Bad if expression"))
|
||||
}
|
||||
},
|
||||
_ => return Err(format!("Bad if expression"))
|
||||
},
|
||||
s => return Err(format!("Non-existent special form {}; this should never happen", s)),
|
||||
})
|
||||
}
|
||||
|
||||
fn apply(&mut self, function: Sexp, operands: Sexp) -> Result<Sexp, String> {
|
||||
use self::Sexp::*;
|
||||
match function {
|
||||
FnLiteral { formal_params, body } => {
|
||||
self.push_env();
|
||||
|
||||
let mut cur = operands;
|
||||
for param in formal_params {
|
||||
match cur {
|
||||
Cons(box arg, box rest) => {
|
||||
cur = rest;
|
||||
self.set_var(param, arg);
|
||||
},
|
||||
_ => return Err(format!("Bad argument for function application")),
|
||||
}
|
||||
}
|
||||
let result = self.eval(*body);
|
||||
self.pop_env();
|
||||
result
|
||||
},
|
||||
Primitive(prim) => {
|
||||
let mut evaled_operands = Vec::new();
|
||||
let mut cur_operand = operands;
|
||||
loop {
|
||||
match cur_operand {
|
||||
Nil => break,
|
||||
Cons(box l, box rest) => {
|
||||
evaled_operands.push(self.eval(l)?);
|
||||
cur_operand = rest;
|
||||
},
|
||||
_ => return Err(format!("Bad operands list"))
|
||||
}
|
||||
}
|
||||
|
||||
prim.apply(evaled_operands)
|
||||
}
|
||||
_ => return Err(format!("Bad type to apply")),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn read(input: &str) -> Result<Vec<Sexp>, String> {
|
||||
let mut chars: Peekable<Chars> = input.chars().peekable();
|
||||
let mut tokens = tokenize(&mut chars).into_iter().peekable();
|
||||
let mut sexps = Vec::new();
|
||||
while let Some(_) = tokens.peek() {
|
||||
sexps.push(parse(&mut tokens)?);
|
||||
}
|
||||
Ok(sexps)
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
enum Token {
|
||||
LParen,
|
||||
RParen,
|
||||
Quote,
|
||||
Word(String),
|
||||
StringLiteral(String),
|
||||
NumLiteral(u64),
|
||||
}
|
||||
|
||||
//TODO make this notion of Eq more sophisticated
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
enum Sexp {
|
||||
SymbolAtom(String),
|
||||
StringAtom(String),
|
||||
NumberAtom(u64),
|
||||
BoolAtom(bool),
|
||||
Cons(Box<Sexp>, Box<Sexp>),
|
||||
Nil,
|
||||
FnLiteral {
|
||||
formal_params: Vec<String>,
|
||||
body: Box<Sexp>
|
||||
},
|
||||
Primitive(PrimitiveFn)
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
enum PrimitiveFn {
|
||||
Plus, Minus, Mult, Div, Mod, Greater, Less, GreaterThanOrEqual, LessThanOrEqual, Display
|
||||
}
|
||||
|
||||
impl PrimitiveFn {
|
||||
fn apply(&self, evaled_operands: Vec<Sexp>) -> Result<Sexp, String> {
|
||||
use self::Sexp::*;
|
||||
use self::PrimitiveFn::*;
|
||||
let op = self.clone();
|
||||
Ok(match op {
|
||||
Display => {
|
||||
for arg in evaled_operands {
|
||||
print!("{}\n", arg.print());
|
||||
}
|
||||
Nil
|
||||
},
|
||||
Plus | Mult => {
|
||||
let mut result = match op { Plus => 0, Mult => 1, _ => unreachable!() };
|
||||
for arg in evaled_operands {
|
||||
if let NumberAtom(n) = arg {
|
||||
if let Plus = op {
|
||||
result += n;
|
||||
} else if let Mult = op {
|
||||
result *= n;
|
||||
}
|
||||
} else {
|
||||
return Err(format!("Bad operand: {:?}", arg));
|
||||
}
|
||||
}
|
||||
NumberAtom(result)
|
||||
},
|
||||
op => return Err(format!("Primitive op {:?} not implemented", op)),
|
||||
})
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
impl Sexp {
|
||||
fn print(&self) -> String {
|
||||
use self::Sexp::*;
|
||||
match self {
|
||||
&BoolAtom(true) => format!("#t"),
|
||||
&BoolAtom(false) => format!("#f"),
|
||||
&SymbolAtom(ref sym) => format!("{}", sym),
|
||||
&StringAtom(ref s) => format!("\"{}\"", s),
|
||||
&NumberAtom(ref n) => format!("{}", n),
|
||||
&Cons(ref car, ref cdr) => format!("({} . {})", car.print(), cdr.print()),
|
||||
&Nil => format!("()"),
|
||||
&FnLiteral { ref formal_params, .. } => format!("<lambda {:?}>", formal_params),
|
||||
&Primitive(ref sym) => format!("<primitive \"{:?}\">", sym),
|
||||
}
|
||||
}
|
||||
|
||||
fn truthy(&self) -> bool {
|
||||
use self::Sexp::*;
|
||||
match self {
|
||||
&BoolAtom(false) => false,
|
||||
_ => true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn tokenize(input: &mut Peekable<Chars>) -> Vec<Token> {
|
||||
use self::Token::*;
|
||||
let mut tokens = Vec::new();
|
||||
loop {
|
||||
match input.next() {
|
||||
None => break,
|
||||
Some('(') => tokens.push(LParen),
|
||||
Some(')') => tokens.push(RParen),
|
||||
Some('\'') => tokens.push(Quote),
|
||||
Some(c) if c.is_whitespace() => continue,
|
||||
Some(c) if c.is_numeric() => {
|
||||
let tok: String = input.peeking_take_while(|next| next.is_numeric()).collect();
|
||||
let n: u64 = format!("{}{}", c, tok).parse().unwrap();
|
||||
tokens.push(NumLiteral(n));
|
||||
},
|
||||
Some('"') => {
|
||||
let string: String = input.scan(false, |escape, cur_char| {
|
||||
let seen_escape = *escape;
|
||||
*escape = cur_char == '\\' && !seen_escape;
|
||||
match (cur_char, seen_escape) {
|
||||
('"', false) => None,
|
||||
('\\', false) => Some(None),
|
||||
(c, _) => Some(Some(c))
|
||||
}
|
||||
}).filter_map(|x| x).collect();
|
||||
tokens.push(StringLiteral(string));
|
||||
}
|
||||
Some(c) => {
|
||||
let sym: String = input.peeking_take_while(|next| {
|
||||
match *next {
|
||||
'(' | ')' => false,
|
||||
c if c.is_whitespace() => false,
|
||||
_ => true
|
||||
}
|
||||
}).collect();
|
||||
tokens.push(Word(format!("{}{}", c, sym)));
|
||||
}
|
||||
}
|
||||
}
|
||||
tokens
|
||||
}
|
||||
|
||||
fn parse(tokens: &mut Peekable<IntoIter<Token>>) -> Result<Sexp, String> {
|
||||
use self::Token::*;
|
||||
use self::Sexp::*;
|
||||
match tokens.next() {
|
||||
Some(Word(ref s)) if s == "#f" => Ok(BoolAtom(false)),
|
||||
Some(Word(ref s)) if s == "#t" => Ok(BoolAtom(true)),
|
||||
Some(Word(s)) => Ok(SymbolAtom(s)),
|
||||
Some(StringLiteral(s)) => Ok(StringAtom(s)),
|
||||
Some(LParen) => parse_sexp(tokens),
|
||||
Some(RParen) => Err(format!("Unexpected ')'")),
|
||||
Some(Quote) => {
|
||||
let quoted = parse(tokens)?;
|
||||
Ok(Cons(Box::new(SymbolAtom(format!("quote"))), Box::new(Cons(Box::new(quoted), Box::new(Nil)))))
|
||||
},
|
||||
Some(NumLiteral(n)) => Ok(NumberAtom(n)),
|
||||
None => Err(format!("Unexpected end of input")),
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_sexp(tokens: &mut Peekable<IntoIter<Token>>) -> Result<Sexp, String> {
|
||||
use self::Token::*;
|
||||
use self::Sexp::*;
|
||||
let mut cell = Nil;
|
||||
{
|
||||
let mut cell_ptr = &mut cell;
|
||||
loop {
|
||||
match tokens.peek() {
|
||||
None => return Err(format!("Unexpected end of input")),
|
||||
Some(&RParen) => {
|
||||
tokens.next();
|
||||
break;
|
||||
},
|
||||
_ => {
|
||||
let current = parse(tokens)?;
|
||||
let new_cdr = Cons(Box::new(current), Box::new(Nil));
|
||||
match cell_ptr {
|
||||
&mut Cons(_, ref mut cdr) => **cdr = new_cdr,
|
||||
&mut Nil => *cell_ptr = new_cdr,
|
||||
_ => unreachable!()
|
||||
};
|
||||
|
||||
let old_ptr = cell_ptr;
|
||||
let new_ptr: &mut Sexp = match old_ptr { &mut Cons(_, ref mut cdr) => cdr, _ => unreachable!() } as &mut Sexp;
|
||||
cell_ptr = new_ptr;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(cell)
|
||||
}
|
||||
|
|
@ -1,133 +0,0 @@
|
|||
use schala_lib::{ProgrammingLanguageInterface, EvalOptions, TraceArtifact, LanguageOutput};
|
||||
use itertools::Itertools;
|
||||
|
||||
use schala_lang::{tokenizing, parsing};
|
||||
use self::tokenizing::*;
|
||||
use self::parsing::*;
|
||||
|
||||
use schala_lang::tokenizing::TokenType::*;
|
||||
|
||||
struct AutoParser {
|
||||
tokens: Vec<Token>,
|
||||
}
|
||||
|
||||
/* BNF
|
||||
* all terminals in this BNF refer to TokenType values
|
||||
|
||||
literal := Kw::True | Kw::False | StrLiteral | number_literal
|
||||
number_literal := int_literal | float_literal
|
||||
float_literal := digits float_continued
|
||||
float_continued := ε | Period digits
|
||||
int_literal := HexLiteral | nonhex_int
|
||||
nonhex_int := BinNumberSigil+ digits
|
||||
digits := (DigitGroup Underscore)+
|
||||
*/
|
||||
|
||||
impl AutoParser {
|
||||
fn new(tokens: Vec<Token>) -> AutoParser {
|
||||
AutoParser { tokens: tokens.into_iter().rev().collect() }
|
||||
}
|
||||
fn peek(&mut self) -> TokenType {
|
||||
self.tokens.last().map(|ref t| { t.token_type.clone() }).unwrap_or(TokenType::EOF)
|
||||
}
|
||||
fn next(&mut self) -> TokenType {
|
||||
self.tokens.pop().map(|t| { t.token_type }).unwrap_or(TokenType::EOF)
|
||||
}
|
||||
fn parse(&mut self) -> (Result<AST, ParseError>, Vec<String>) {
|
||||
let ast = self.program();
|
||||
(ast, vec![])
|
||||
}
|
||||
fn program(&mut self) -> ParseResult<AST> {
|
||||
let etype = self.literal()?;
|
||||
Ok(AST(vec![Statement::ExpressionStatement(Expression(etype, None))]))
|
||||
}
|
||||
}
|
||||
|
||||
macro_rules! expand_match_var {
|
||||
(($pat:pat => $e:expr)) => { $pat };
|
||||
(nonterm ($pat:pat => $e:expr)) => { $pat };
|
||||
}
|
||||
|
||||
macro_rules! expand_match_expr {
|
||||
($self:ident, ($pat:pat => $e:expr)) => {
|
||||
{ $self.next(); $e }
|
||||
};
|
||||
($self:ident, nonterm ($pat:pat => $e:expr)) => {
|
||||
{ $self.next(); $e }
|
||||
};
|
||||
}
|
||||
|
||||
macro_rules! bnf_rule {
|
||||
($self:ident, $type:ty, $rule:ident := $( $rule_clauses:tt )|*) => {
|
||||
fn $rule(&mut $self) -> ParseResult<$type> {
|
||||
Ok(match $self.peek() {
|
||||
$(
|
||||
expand_match_var!($rule_clauses) => expand_match_expr!($self, $rule_clauses),
|
||||
)*
|
||||
_ => return ParseError::new("Not found"),
|
||||
})
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
impl AutoParser {
|
||||
bnf_rule!(self, ExpressionType, literal :=
|
||||
(Keyword(Kw::True) => ExpressionType::BoolLiteral(true)) |
|
||||
(Keyword(Kw::False) => ExpressionType::BoolLiteral(false))
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
pub struct Schala { }
|
||||
|
||||
impl Schala {
|
||||
pub fn new() -> Schala {
|
||||
Schala { }
|
||||
}
|
||||
}
|
||||
|
||||
impl ProgrammingLanguageInterface for Schala {
|
||||
fn get_language_name(&self) -> String {
|
||||
"Schala-autoparser".to_string()
|
||||
}
|
||||
fn get_source_file_suffix(&self) -> String {
|
||||
format!("schala")
|
||||
}
|
||||
|
||||
fn evaluate_in_repl(&mut self, input: &str, options: &EvalOptions) -> LanguageOutput {
|
||||
let mut output = LanguageOutput::default();
|
||||
|
||||
let tokens = tokenizing::tokenize(input);
|
||||
if options.debug_tokens {
|
||||
let token_string = tokens.iter().map(|t| format!("{:?}<L:{},C:{}>", t.token_type, t.offset.0, t.offset.1)).join(", ");
|
||||
output.add_artifact(TraceArtifact::new("tokens", format!("{:?}", token_string)));
|
||||
}
|
||||
{
|
||||
let token_errors: Vec<&String> = tokens.iter().filter_map(|t| t.get_error()).collect();
|
||||
if token_errors.len() != 0 {
|
||||
output.add_output(format!("Tokenization error: {:?}\n", token_errors));
|
||||
return output;
|
||||
}
|
||||
}
|
||||
|
||||
let mut parser = AutoParser::new(tokens);
|
||||
|
||||
let ast = match parser.parse() {
|
||||
(Ok(ast), trace) => {
|
||||
if options.debug_parse {
|
||||
output.add_artifact(TraceArtifact::new_parse_trace(trace));
|
||||
output.add_artifact(TraceArtifact::new("ast", format!("{:?}", ast)));
|
||||
}
|
||||
ast
|
||||
},
|
||||
(Err(err), trace) => {
|
||||
output.add_artifact(TraceArtifact::new_parse_trace(trace));
|
||||
output.add_output(format!("Parse error: {:?}\n", err.msg));
|
||||
return output;
|
||||
}
|
||||
};
|
||||
|
||||
output.add_output(format!("{:?}", ast));
|
||||
output
|
||||
}
|
||||
}
|
|
@ -1,77 +0,0 @@
|
|||
use std::rc::Rc;
|
||||
use std::collections::HashMap;
|
||||
|
||||
use schala_lang::typechecking::{Type, TypeResult, TConst};
|
||||
use self::Type::*; use self::TConst::*;
|
||||
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
pub struct BinOp {
|
||||
sigil: Rc<String>
|
||||
}
|
||||
|
||||
impl BinOp {
|
||||
pub fn from_sigil(sigil: &str) -> BinOp {
|
||||
BinOp { sigil: Rc::new(sigil.to_string()) }
|
||||
}
|
||||
pub fn sigil(&self) -> &Rc<String> {
|
||||
&self.sigil
|
||||
}
|
||||
pub fn get_type(&self) -> TypeResult<Type> {
|
||||
let s = self.sigil.as_str();
|
||||
BINOPS.get(s).map(|x| x.0.clone()).ok_or(format!("Binop {} not found", s))
|
||||
}
|
||||
pub fn min_precedence() -> i32 {
|
||||
i32::min_value()
|
||||
}
|
||||
pub fn get_precedence(op: &str) -> i32 {
|
||||
let default = 10_000_000;
|
||||
BINOPS.get(op).map(|x| x.2.clone()).unwrap_or(default)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
pub struct PrefixOp {
|
||||
sigil: Rc<String>
|
||||
}
|
||||
|
||||
impl PrefixOp {
|
||||
pub fn from_sigil(sigil: &str) -> PrefixOp {
|
||||
PrefixOp { sigil: Rc::new(sigil.to_string()) }
|
||||
}
|
||||
pub fn sigil(&self) -> &Rc<String> {
|
||||
&self.sigil
|
||||
}
|
||||
pub fn is_prefix(op: &str) -> bool {
|
||||
PREFIX_OPS.get(op).is_some()
|
||||
}
|
||||
pub fn get_type(&self) -> TypeResult<Type> {
|
||||
let s = self.sigil.as_str();
|
||||
PREFIX_OPS.get(s).map(|x| x.0.clone()).ok_or(format!("Prefix op {} not found", s))
|
||||
}
|
||||
}
|
||||
lazy_static! {
|
||||
static ref PREFIX_OPS: HashMap<&'static str, (Type, ())> =
|
||||
hashmap! {
|
||||
"+" => (Func(bx!(Const(Int)), bx!(Const(Int))), ()),
|
||||
"-" => (Func(bx!(Const(Int)), bx!(Const(Int))), ()),
|
||||
"!" => (Func(bx!(Const(Bool)), bx!(Const(Bool))), ()),
|
||||
};
|
||||
}
|
||||
|
||||
/* the second tuple member is a placeholder for when I want to make evaluation rules tied to the
|
||||
* binop definition */
|
||||
lazy_static! {
|
||||
static ref BINOPS: HashMap<&'static str, (Type, (), i32)> =
|
||||
hashmap! {
|
||||
"+" => (Func(bx!(Const(Int)), bx!(Func(bx!(Const(Int)), bx!(Const(Int))))), (), 10),
|
||||
"-" => (Func(bx!(Const(Int)), bx!(Func(bx!(Const(Int)), bx!(Const(Int))))), (), 10),
|
||||
"*" => (Func(bx!(Const(Int)), bx!(Func(bx!(Const(Int)), bx!(Const(Int))))), (), 20),
|
||||
"/" => (Func(bx!(Const(Int)), bx!(Func(bx!(Const(Int)), bx!(Const(Float))))), (), 20),
|
||||
"//" => (Func(bx!(Const(Int)), bx!(Func(bx!(Const(Int)), bx!(Const(Int))))), (), 20),
|
||||
"%" => (Func(bx!(Const(Int)), bx!(Func(bx!(Const(Int)), bx!(Const(Int))))), (), 20),
|
||||
"++" => (Func(bx!(Const(StringT)), bx!(Func(bx!(Const(StringT)), bx!(Const(StringT))))), (), 30),
|
||||
"^" => (Func(bx!(Const(Int)), bx!(Func(bx!(Const(Int)), bx!(Const(Int))))), (), 20),
|
||||
"&" => (Func(bx!(Const(Int)), bx!(Func(bx!(Const(Int)), bx!(Const(Int))))), (), 20),
|
||||
"|" => (Func(bx!(Const(Int)), bx!(Func(bx!(Const(Int)), bx!(Const(Int))))), (), 20),
|
||||
};
|
||||
}
|
|
@ -1,317 +0,0 @@
|
|||
use std::collections::HashMap;
|
||||
use std::rc::Rc;
|
||||
use std::fmt::Write;
|
||||
|
||||
use itertools::Itertools;
|
||||
|
||||
use schala_lang::parsing::{AST, Statement, Declaration, Expression, Variant, ExpressionType};
|
||||
use schala_lang::builtin::{BinOp, PrefixOp};
|
||||
|
||||
pub struct State<'a> {
|
||||
parent_frame: Option<&'a State<'a>>,
|
||||
values: HashMap<Rc<String>, ValueEntry>,
|
||||
}
|
||||
|
||||
impl<'a> State<'a> {
|
||||
|
||||
fn insert(&mut self, name: Rc<String>, value: ValueEntry) {
|
||||
self.values.insert(name, value);
|
||||
}
|
||||
fn lookup(&self, name: &Rc<String>) -> Option<&ValueEntry> {
|
||||
match (self.values.get(name), self.parent_frame) {
|
||||
(None, None) => None,
|
||||
(None, Some(parent)) => parent.lookup(name),
|
||||
(Some(value), _) => Some(value),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
enum ValueEntry {
|
||||
Binding {
|
||||
val: FullyEvaluatedExpr,
|
||||
},
|
||||
Function {
|
||||
param_names: Vec<Rc<String>>,
|
||||
body: Vec<Statement>,
|
||||
}
|
||||
}
|
||||
|
||||
type EvalResult<T> = Result<T, String>;
|
||||
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
enum FullyEvaluatedExpr {
|
||||
UnsignedInt(u64),
|
||||
SignedInt(i64),
|
||||
Float(f64),
|
||||
Str(String),
|
||||
Bool(bool),
|
||||
FuncLit(Rc<String>),
|
||||
Custom {
|
||||
string_rep: Rc<String>,
|
||||
},
|
||||
Tuple(Vec<FullyEvaluatedExpr>),
|
||||
List(Vec<FullyEvaluatedExpr>)
|
||||
}
|
||||
|
||||
impl FullyEvaluatedExpr {
|
||||
fn to_string(&self) -> String {
|
||||
use self::FullyEvaluatedExpr::*;
|
||||
match self {
|
||||
&UnsignedInt(ref n) => format!("{}", n),
|
||||
&SignedInt(ref n) => format!("{}", n),
|
||||
&Float(ref f) => format!("{}", f),
|
||||
&Str(ref s) => format!("\"{}\"", s),
|
||||
&Bool(ref b) => format!("{}", b),
|
||||
&Custom { ref string_rep } => format!("{}", string_rep),
|
||||
&Tuple(ref items) => {
|
||||
let mut buf = String::new();
|
||||
write!(buf, "(").unwrap();
|
||||
for term in items.iter().map(|e| Some(e)).intersperse(None) {
|
||||
match term {
|
||||
Some(e) => write!(buf, "{}", e.to_string()).unwrap(),
|
||||
None => write!(buf, ", ").unwrap(),
|
||||
};
|
||||
}
|
||||
write!(buf, ")").unwrap();
|
||||
buf
|
||||
},
|
||||
&FuncLit(ref name) => format!("<function {}>", name),
|
||||
&List(ref items) => {
|
||||
let mut buf = String::new();
|
||||
write!(buf, "[").unwrap();
|
||||
for term in items.iter().map(|e| Some(e)).intersperse(None) {
|
||||
match term {
|
||||
Some(e) => write!(buf, "{}", e.to_string()).unwrap(),
|
||||
None => write!(buf, ", ").unwrap()
|
||||
}
|
||||
}
|
||||
write!(buf, "]").unwrap();
|
||||
buf
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> State<'a> {
|
||||
pub fn new() -> State<'a> {
|
||||
State { parent_frame: None, values: HashMap::new() }
|
||||
}
|
||||
|
||||
pub fn new_with_parent(parent: &'a State<'a>) -> State<'a> {
|
||||
State { parent_frame: Some(parent), values: HashMap::new() }
|
||||
}
|
||||
|
||||
pub fn evaluate(&mut self, ast: AST) -> Vec<String> {
|
||||
let mut acc = vec![];
|
||||
for statement in ast.0 {
|
||||
match self.eval_statement(statement) {
|
||||
Ok(output) => {
|
||||
if let Some(fully_evaluated) = output {
|
||||
acc.push(fully_evaluated.to_string());
|
||||
}
|
||||
},
|
||||
Err(error) => {
|
||||
acc.push(format!("Eval error: {}", error));
|
||||
return acc;
|
||||
},
|
||||
}
|
||||
}
|
||||
acc
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> State<'a> {
|
||||
fn eval_statement(&mut self, statement: Statement) -> EvalResult<Option<FullyEvaluatedExpr>> {
|
||||
Ok(match statement {
|
||||
Statement::ExpressionStatement(expr) => Some(self.eval_expr(expr)?),
|
||||
Statement::Declaration(decl) => { self.eval_decl(decl)?; None }
|
||||
})
|
||||
}
|
||||
|
||||
fn eval_decl(&mut self, decl: Declaration) -> EvalResult<()> {
|
||||
use self::Declaration::*;
|
||||
use self::Variant::*;
|
||||
|
||||
match decl {
|
||||
FuncDecl(signature, statements) => {
|
||||
let name = signature.name;
|
||||
let param_names: Vec<Rc<String>> = signature.params.iter().map(|fp| fp.0.clone()).collect();
|
||||
self.insert(name, ValueEntry::Function { body: statements.clone(), param_names });
|
||||
},
|
||||
TypeDecl(_name, body) => {
|
||||
for variant in body.0.iter() {
|
||||
match variant {
|
||||
&UnitStruct(ref name) => self.insert(name.clone(),
|
||||
ValueEntry::Binding { val: FullyEvaluatedExpr::Custom { string_rep: name.clone() } }),
|
||||
&TupleStruct(ref _name, ref _args) => unimplemented!(),
|
||||
&Record(ref _name, ref _fields) => unimplemented!(),
|
||||
};
|
||||
}
|
||||
},
|
||||
Binding { name, expr, ..} => {
|
||||
let val = self.eval_expr(expr)?;
|
||||
self.insert(name.clone(), ValueEntry::Binding { val });
|
||||
},
|
||||
_ => return Err(format!("Declaration evaluation not yet implemented"))
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn eval_expr(&mut self, expr: Expression) -> EvalResult<FullyEvaluatedExpr> {
|
||||
use self::ExpressionType::*;
|
||||
use self::FullyEvaluatedExpr::*;
|
||||
|
||||
let expr_type = expr.0;
|
||||
match expr_type {
|
||||
IntLiteral(n) => Ok(UnsignedInt(n)),
|
||||
FloatLiteral(f) => Ok(Float(f)),
|
||||
StringLiteral(s) => Ok(Str(s.to_string())),
|
||||
BoolLiteral(b) => Ok(Bool(b)),
|
||||
PrefixExp(op, expr) => self.eval_prefix_exp(op, expr),
|
||||
BinExp(op, lhs, rhs) => self.eval_binexp(op, lhs, rhs),
|
||||
Value(name) => self.eval_value(name),
|
||||
TupleLiteral(expressions) => {
|
||||
let mut evals = Vec::new();
|
||||
for expr in expressions {
|
||||
match self.eval_expr(expr) {
|
||||
Ok(fully_evaluated) => evals.push(fully_evaluated),
|
||||
error => return error,
|
||||
}
|
||||
}
|
||||
Ok(Tuple(evals))
|
||||
}
|
||||
Call { f, arguments } => {
|
||||
let mut evaled_arguments = Vec::new();
|
||||
for arg in arguments.into_iter() {
|
||||
evaled_arguments.push(self.eval_expr(arg)?);
|
||||
}
|
||||
self.eval_application(*f, evaled_arguments)
|
||||
},
|
||||
Index { box indexee, indexers } => {
|
||||
let evaled = self.eval_expr(indexee)?;
|
||||
match evaled {
|
||||
Tuple(mut exprs) => {
|
||||
let len = indexers.len();
|
||||
if len == 1 {
|
||||
let idx = indexers.into_iter().nth(0).unwrap();
|
||||
match self.eval_expr(idx)? {
|
||||
UnsignedInt(n) if (n as usize) < exprs.len() => Ok(exprs.drain(n as usize..).next().unwrap()),
|
||||
UnsignedInt(n) => Err(format!("Index {} out of range", n)),
|
||||
other => Err(format!("{:?} is not an unsigned integer", other)),
|
||||
}
|
||||
} else {
|
||||
Err(format!("Tuple index must be one integer"))
|
||||
}
|
||||
},
|
||||
_ => Err(format!("Bad index expression"))
|
||||
}
|
||||
},
|
||||
ListLiteral(items) => Ok(List(items.into_iter().map(|item| self.eval_expr(item)).collect::<Result<Vec<_>,_>>()?)),
|
||||
x => Err(format!("Unimplemented thing {:?}", x)),
|
||||
}
|
||||
}
|
||||
|
||||
fn eval_application(&mut self, f: Expression, arguments: Vec<FullyEvaluatedExpr>) -> EvalResult<FullyEvaluatedExpr> {
|
||||
use self::ExpressionType::*;
|
||||
match f {
|
||||
Expression(Value(ref identifier), _) if self.is_builtin(identifier) => self.eval_builtin(identifier, arguments),
|
||||
Expression(Value(identifier), _) => {
|
||||
match self.lookup(&identifier) {
|
||||
Some(&ValueEntry::Function { ref body, ref param_names }) => {
|
||||
if arguments.len() != param_names.len() {
|
||||
return Err(format!("Wrong number of arguments for the function"));
|
||||
}
|
||||
let mut new_state = State::new_with_parent(self);
|
||||
let sub_ast = body.clone();
|
||||
for (param, val) in param_names.iter().zip(arguments.into_iter()) {
|
||||
new_state.insert(param.clone(), ValueEntry::Binding { val });
|
||||
}
|
||||
let mut ret: Option<FullyEvaluatedExpr> = None;
|
||||
for statement in sub_ast.into_iter() {
|
||||
ret = new_state.eval_statement(statement)?;
|
||||
}
|
||||
Ok(ret.unwrap_or(FullyEvaluatedExpr::Custom { string_rep: Rc::new("()".to_string()) }))
|
||||
},
|
||||
_ => Err(format!("Function {} not found", identifier)),
|
||||
}
|
||||
},
|
||||
x => Err(format!("Trying to apply {:?} which is not a function", x)),
|
||||
}
|
||||
}
|
||||
fn is_builtin(&self, name: &Rc<String>) -> bool {
|
||||
match &name.as_ref()[..] {
|
||||
"print" | "println" => true,
|
||||
_ => false
|
||||
}
|
||||
}
|
||||
fn eval_builtin(&mut self, name: &Rc<String>, args: Vec<FullyEvaluatedExpr>) -> EvalResult<FullyEvaluatedExpr> {
|
||||
use self::FullyEvaluatedExpr::*;
|
||||
match &name.as_ref()[..] {
|
||||
"print" => {
|
||||
for arg in args {
|
||||
print!("{}", arg.to_string());
|
||||
}
|
||||
Ok(Tuple(vec![]))
|
||||
},
|
||||
"println" => {
|
||||
for arg in args {
|
||||
println!("{}", arg.to_string());
|
||||
}
|
||||
Ok(Tuple(vec![]))
|
||||
},
|
||||
_ => unreachable!()
|
||||
}
|
||||
}
|
||||
fn eval_value(&mut self, name: Rc<String>) -> EvalResult<FullyEvaluatedExpr> {
|
||||
use self::ValueEntry::*;
|
||||
match self.lookup(&name) {
|
||||
None => return Err(format!("Value {} not found", *name)),
|
||||
Some(lookup) => match lookup {
|
||||
&Binding { ref val } => Ok(val.clone()),
|
||||
&Function { .. } => Ok(FullyEvaluatedExpr::FuncLit(name.clone()))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn eval_binexp(&mut self, op: BinOp, lhs: Box<Expression>, rhs: Box<Expression>) -> EvalResult<FullyEvaluatedExpr> {
|
||||
use self::FullyEvaluatedExpr::*;
|
||||
let evaled_lhs = self.eval_expr(*lhs)?;
|
||||
let evaled_rhs = self.eval_expr(*rhs)?;
|
||||
let sigil = op.sigil();
|
||||
//let sigil: &str = op.sigil().as_ref().as_str();
|
||||
Ok(match (sigil.as_str(), evaled_lhs, evaled_rhs) {
|
||||
("+", UnsignedInt(l), UnsignedInt(r)) => UnsignedInt(l + r),
|
||||
("++", Str(s1), Str(s2)) => Str(format!("{}{}", s1, s2)),
|
||||
("-", UnsignedInt(l), UnsignedInt(r)) => UnsignedInt(l - r),
|
||||
("*", UnsignedInt(l), UnsignedInt(r)) => UnsignedInt(l * r),
|
||||
("/", UnsignedInt(l), UnsignedInt(r)) => Float((l as f64)/ (r as f64)),
|
||||
("//", UnsignedInt(l), UnsignedInt(r)) => if r == 0 {
|
||||
return Err(format!("Runtime error: divide by zero"));
|
||||
} else {
|
||||
UnsignedInt(l / r)
|
||||
},
|
||||
("%", UnsignedInt(l), UnsignedInt(r)) => UnsignedInt(l % r),
|
||||
("^", UnsignedInt(l), UnsignedInt(r)) => UnsignedInt(l ^ r),
|
||||
("&", UnsignedInt(l), UnsignedInt(r)) => UnsignedInt(l & r),
|
||||
("|", UnsignedInt(l), UnsignedInt(r)) => UnsignedInt(l | r),
|
||||
_ => return Err(format!("Runtime error: not yet implemented")),
|
||||
})
|
||||
}
|
||||
|
||||
fn eval_prefix_exp(&mut self, op: PrefixOp, expr: Box<Expression>) -> EvalResult<FullyEvaluatedExpr> {
|
||||
use self::FullyEvaluatedExpr::*;
|
||||
let evaled_expr = self.eval_expr(*expr)?;
|
||||
let sigil = op.sigil();
|
||||
|
||||
Ok(match (sigil.as_str(), evaled_expr) {
|
||||
("!", Bool(true)) => Bool(false),
|
||||
("!", Bool(false)) => Bool(true),
|
||||
("-", UnsignedInt(n)) => SignedInt(-1*(n as i64)),
|
||||
("-", SignedInt(n)) => SignedInt(-1*(n as i64)),
|
||||
("+", SignedInt(n)) => SignedInt(n),
|
||||
("+", UnsignedInt(n)) => UnsignedInt(n),
|
||||
_ => return Err(format!("Runtime error: not yet implemented")),
|
||||
})
|
||||
}
|
||||
}
|
|
@ -1,104 +0,0 @@
|
|||
use itertools::Itertools;
|
||||
use schala_lib::{ProgrammingLanguageInterface, EvalOptions, TraceArtifact, LanguageOutput};
|
||||
|
||||
macro_rules! bx {
|
||||
($e:expr) => { Box::new($e) }
|
||||
}
|
||||
|
||||
pub mod autoparser;
|
||||
|
||||
mod builtin;
|
||||
|
||||
mod tokenizing;
|
||||
mod parsing;
|
||||
mod typechecking;
|
||||
mod eval;
|
||||
|
||||
use self::typechecking::{TypeContext};
|
||||
|
||||
pub struct Schala {
|
||||
state: eval::State<'static>,
|
||||
type_context: TypeContext
|
||||
}
|
||||
|
||||
impl Schala {
|
||||
pub fn new() -> Schala {
|
||||
Schala {
|
||||
state: eval::State::new(),
|
||||
type_context: TypeContext::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl ProgrammingLanguageInterface for Schala {
|
||||
fn get_language_name(&self) -> String {
|
||||
"Schala".to_string()
|
||||
}
|
||||
|
||||
fn get_source_file_suffix(&self) -> String {
|
||||
format!("schala")
|
||||
}
|
||||
|
||||
fn evaluate_in_repl(&mut self, input: &str, options: &EvalOptions) -> LanguageOutput {
|
||||
let mut output = LanguageOutput::default();
|
||||
let tokens = tokenizing::tokenize(input);
|
||||
if options.debug_tokens {
|
||||
let token_string = tokens.iter().map(|t| format!("{:?}<L:{},C:{}>", t.token_type, t.offset.0, t.offset.1)).join(", ");
|
||||
output.add_artifact(TraceArtifact::new("tokens", format!("{:?}", token_string)));
|
||||
}
|
||||
|
||||
{
|
||||
let token_errors: Vec<&String> = tokens.iter().filter_map(|t| t.get_error()).collect();
|
||||
if token_errors.len() != 0 {
|
||||
output.add_output(format!("Tokenization error: {:?}\n", token_errors));
|
||||
return output;
|
||||
}
|
||||
}
|
||||
|
||||
let ast = match parsing::parse(tokens) {
|
||||
(Ok(ast), trace) => {
|
||||
if options.debug_parse {
|
||||
output.add_artifact(TraceArtifact::new_parse_trace(trace));
|
||||
output.add_artifact(TraceArtifact::new("ast", format!("{:?}", ast)));
|
||||
}
|
||||
ast
|
||||
},
|
||||
(Err(err), trace) => {
|
||||
output.add_artifact(TraceArtifact::new_parse_trace(trace));
|
||||
output.add_output(format!("Parse error: {:?}\n", err.msg));
|
||||
return output;
|
||||
}
|
||||
};
|
||||
|
||||
match self.type_context.add_top_level_types(&ast) {
|
||||
Ok(()) => (),
|
||||
Err(msg) => {
|
||||
output.add_artifact(TraceArtifact::new("type_check", msg));
|
||||
//return output
|
||||
}
|
||||
};
|
||||
|
||||
if options.debug_symbol_table {
|
||||
let text = self.type_context.debug_symbol_table();
|
||||
output.add_artifact(TraceArtifact::new("symbol_table", text));
|
||||
}
|
||||
|
||||
match self.type_context.type_check_ast(&ast) {
|
||||
Ok(ty) => {
|
||||
output.add_artifact(TraceArtifact::new("type_check", format!("{:?}", ty)));
|
||||
},
|
||||
Err(msg) => {
|
||||
output.add_artifact(TraceArtifact::new("type_check", msg));
|
||||
/*
|
||||
output.add_output(format!("Type error"));
|
||||
return output;
|
||||
*/
|
||||
}
|
||||
}
|
||||
|
||||
let evaluation_outputs = self.state.evaluate(ast);
|
||||
let text_output: String = evaluation_outputs.into_iter().intersperse(format!("\n")).collect();
|
||||
output.add_output(text_output);
|
||||
return output;
|
||||
}
|
||||
}
|
File diff suppressed because it is too large
Load Diff
|
@ -1,264 +0,0 @@
|
|||
use itertools::Itertools;
|
||||
use std::collections::HashMap;
|
||||
use std::rc::Rc;
|
||||
use std::iter::{Iterator, Peekable};
|
||||
use std::fmt;
|
||||
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
pub enum TokenType {
|
||||
Newline, Semicolon,
|
||||
|
||||
LParen, RParen,
|
||||
LSquareBracket, RSquareBracket,
|
||||
LAngleBracket, RAngleBracket,
|
||||
LCurlyBrace, RCurlyBrace,
|
||||
Pipe,
|
||||
|
||||
Comma, Period, Colon, Underscore,
|
||||
|
||||
Operator(Rc<String>),
|
||||
DigitGroup(Rc<String>), HexLiteral(Rc<String>), BinNumberSigil,
|
||||
StrLiteral(Rc<String>),
|
||||
Identifier(Rc<String>),
|
||||
Keyword(Kw),
|
||||
|
||||
EOF,
|
||||
|
||||
Error(String),
|
||||
}
|
||||
use self::TokenType::*;
|
||||
|
||||
impl fmt::Display for TokenType {
|
||||
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
|
||||
match self {
|
||||
&Operator(ref s) => write!(f, "Operator({})", **s),
|
||||
&DigitGroup(ref s) => write!(f, "DigitGroup({})", s),
|
||||
&HexLiteral(ref s) => write!(f, "HexLiteral({})", s),
|
||||
&StrLiteral(ref s) => write!(f, "StrLiteral({})", s),
|
||||
&Identifier(ref s) => write!(f, "Identifier({})", s),
|
||||
&Error(ref s) => write!(f, "Error({})", s),
|
||||
other => write!(f, "{:?}", other),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq)]
|
||||
pub enum Kw {
|
||||
If, Else,
|
||||
Func,
|
||||
For,
|
||||
Match,
|
||||
Var, Const, Let, In,
|
||||
Return,
|
||||
Alias, Type, SelfType, SelfIdent,
|
||||
Trait, Impl,
|
||||
True, False,
|
||||
Module
|
||||
}
|
||||
|
||||
lazy_static! {
|
||||
static ref KEYWORDS: HashMap<&'static str, Kw> =
|
||||
hashmap! {
|
||||
"if" => Kw::If,
|
||||
"else" => Kw::Else,
|
||||
"fn" => Kw::Func,
|
||||
"for" => Kw::For,
|
||||
"match" => Kw::Match,
|
||||
"var" => Kw::Var,
|
||||
"const" => Kw::Const,
|
||||
"let" => Kw::Let,
|
||||
"in" => Kw::In,
|
||||
"return" => Kw::Return,
|
||||
"alias" => Kw::Alias,
|
||||
"type" => Kw::Type,
|
||||
"Self" => Kw::SelfType,
|
||||
"self" => Kw::SelfIdent,
|
||||
"trait" => Kw::Trait,
|
||||
"impl" => Kw::Impl,
|
||||
"true" => Kw::True,
|
||||
"false" => Kw::False,
|
||||
"module" => Kw::Module,
|
||||
};
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Token {
|
||||
pub token_type: TokenType,
|
||||
pub offset: (usize, usize),
|
||||
}
|
||||
|
||||
impl Token {
|
||||
pub fn get_error(&self) -> Option<&String> {
|
||||
match self.token_type {
|
||||
TokenType::Error(ref s) => Some(s),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
pub fn to_string_with_metadata(&self) -> String {
|
||||
format!("{}(L:{},c:{})", self.token_type, self.offset.0, self.offset.1)
|
||||
}
|
||||
}
|
||||
|
||||
const OPERATOR_CHARS: [char; 19] = ['!', '$', '%', '&', '*', '+', '-', '.', '/', ':', '<', '>', '=', '?', '@', '^', '|', '~', '`'];
|
||||
fn is_operator(c: &char) -> bool {
|
||||
OPERATOR_CHARS.iter().any(|x| x == c)
|
||||
}
|
||||
|
||||
type CharIter<I: Iterator<Item=(usize,usize,char)>> = Peekable<I>;
|
||||
|
||||
pub fn tokenize(input: &str) -> Vec<Token> {
|
||||
let mut tokens: Vec<Token> = Vec::new();
|
||||
|
||||
let mut input = input.lines().enumerate()
|
||||
.flat_map(|(line_idx, ref line)| {
|
||||
line.chars().enumerate().map(move |(ch_idx, ch)| (line_idx, ch_idx, ch))
|
||||
}).peekable();
|
||||
|
||||
while let Some((line_idx, ch_idx, c)) = input.next() {
|
||||
let cur_tok_type = match c {
|
||||
'#' => {
|
||||
if let Some(&(_, _, '{')) = input.peek() {
|
||||
} else {
|
||||
while let Some((_, _, c)) = input.next() {
|
||||
if c == '\n' {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
continue;
|
||||
},
|
||||
c if c.is_whitespace() && c != '\n' => continue,
|
||||
'\n' => Newline, ';' => Semicolon,
|
||||
':' => Colon, ',' => Comma,
|
||||
'(' => LParen, ')' => RParen,
|
||||
'{' => LCurlyBrace, '}' => RCurlyBrace,
|
||||
'[' => LSquareBracket, ']' => RSquareBracket,
|
||||
'"' => handle_quote(&mut input),
|
||||
c if c.is_digit(10) => handle_digit(c, &mut input),
|
||||
c if c.is_alphabetic() || c == '_' => handle_alphabetic(c, &mut input), //TODO I'll probably have to rewrite this if I care about types being uppercase, also type parameterization
|
||||
c if is_operator(&c) => handle_operator(c, &mut input),
|
||||
unknown => Error(format!("Unexpected character: {}", unknown)),
|
||||
};
|
||||
tokens.push(Token { token_type: cur_tok_type, offset: (line_idx, ch_idx) });
|
||||
}
|
||||
tokens
|
||||
}
|
||||
|
||||
fn handle_digit<I: Iterator<Item=(usize,usize,char)>>(c: char, input: &mut CharIter<I>) -> TokenType {
|
||||
if c == '0' && input.peek().map_or(false, |&(_, _, c)| { c == 'x' }) {
|
||||
input.next();
|
||||
let rest: String = input.peeking_take_while(|&(_, _, ref c)| c.is_digit(16) || *c == '_').map(|(_, _, c)| { c }).collect();
|
||||
HexLiteral(Rc::new(rest))
|
||||
} else if c == '0' && input.peek().map_or(false, |&(_, _, c)| { c == 'b' }) {
|
||||
input.next();
|
||||
BinNumberSigil
|
||||
} else {
|
||||
let mut buf = c.to_string();
|
||||
buf.extend(input.peeking_take_while(|&(_, _, ref c)| c.is_digit(10)).map(|(_, _, c)| { c }));
|
||||
DigitGroup(Rc::new(buf))
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_quote<I: Iterator<Item=(usize,usize,char)>>(input: &mut CharIter<I>) -> TokenType {
|
||||
let mut buf = String::new();
|
||||
loop {
|
||||
match input.next().map(|(_, _, c)| { c }) {
|
||||
Some('"') => break,
|
||||
Some('\\') => {
|
||||
let next = input.peek().map(|&(_, _, c)| { c });
|
||||
if next == Some('n') {
|
||||
input.next();
|
||||
buf.push('\n')
|
||||
} else if next == Some('"') {
|
||||
input.next();
|
||||
buf.push('"');
|
||||
} else if next == Some('t') {
|
||||
input.next();
|
||||
buf.push('\t');
|
||||
}
|
||||
},
|
||||
Some(c) => buf.push(c),
|
||||
None => return TokenType::Error(format!("Unclosed string")),
|
||||
}
|
||||
}
|
||||
TokenType::StrLiteral(Rc::new(buf))
|
||||
}
|
||||
|
||||
fn handle_alphabetic<I: Iterator<Item=(usize,usize,char)>>(c: char, input: &mut CharIter<I>) -> TokenType {
|
||||
let mut buf = String::new();
|
||||
buf.push(c);
|
||||
if c == '_' && input.peek().map(|&(_, _, c)| { !c.is_alphabetic() }).unwrap_or(true) {
|
||||
return TokenType::Underscore
|
||||
}
|
||||
|
||||
loop {
|
||||
match input.peek().map(|&(_, _, c)| { c }) {
|
||||
Some(c) if c.is_alphanumeric() => {
|
||||
input.next();
|
||||
buf.push(c);
|
||||
},
|
||||
_ => break,
|
||||
}
|
||||
}
|
||||
|
||||
match KEYWORDS.get(buf.as_str()) {
|
||||
Some(kw) => TokenType::Keyword(*kw),
|
||||
None => TokenType::Identifier(Rc::new(buf)),
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_operator<I: Iterator<Item=(usize,usize,char)>>(c: char, input: &mut CharIter<I>) -> TokenType {
|
||||
match c {
|
||||
'<' | '>' | '|' | '.' => {
|
||||
let ref next = input.peek().map(|&(_, _, c)| { c });
|
||||
if !next.map(|n| { is_operator(&n) }).unwrap_or(false) {
|
||||
return match c {
|
||||
'<' => LAngleBracket,
|
||||
'>' => RAngleBracket,
|
||||
'|' => Pipe,
|
||||
'.' => Period,
|
||||
_ => unreachable!(),
|
||||
}
|
||||
}
|
||||
},
|
||||
_ => (),
|
||||
};
|
||||
|
||||
let mut buf = String::new();
|
||||
buf.push(c);
|
||||
loop {
|
||||
match input.peek().map(|&(_, _, c)| { c }) {
|
||||
Some(c) if is_operator(&c) => {
|
||||
input.next();
|
||||
buf.push(c);
|
||||
},
|
||||
_ => break
|
||||
}
|
||||
}
|
||||
TokenType::Operator(Rc::new(buf))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod schala_tokenizer_tests {
|
||||
use super::*;
|
||||
use super::Kw::*;
|
||||
|
||||
macro_rules! digit { ($ident:expr) => { DigitGroup(Rc::new($ident.to_string())) } }
|
||||
macro_rules! ident { ($ident:expr) => { Identifier(Rc::new($ident.to_string())) } }
|
||||
macro_rules! op { ($ident:expr) => { Operator(Rc::new($ident.to_string())) } }
|
||||
|
||||
|
||||
#[test]
|
||||
fn tokens() {
|
||||
let a = tokenize("let a: A<B> = c ++ d");
|
||||
let token_types: Vec<TokenType> = a.into_iter().map(move |t| t.token_type).collect();
|
||||
assert_eq!(token_types, vec![Keyword(Let), ident!("a"), Colon, ident!("A"),
|
||||
LAngleBracket, ident!("B"), RAngleBracket, op!("="), ident!("c"), op!("++"), ident!("d")]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn underscores() {
|
||||
let token_types: Vec<TokenType> = tokenize("4_8").into_iter().map(move |t| t.token_type).collect();
|
||||
assert_eq!(token_types, vec![digit!("4"), Underscore, digit!("8")]);
|
||||
}
|
||||
}
|
|
@ -1,445 +0,0 @@
|
|||
use std::collections::HashMap;
|
||||
use std::rc::Rc;
|
||||
|
||||
|
||||
use schala_lang::parsing::{AST, Statement, Declaration, Signature, Expression, ExpressionType, Operation, Variant, TypeName, TypeSingletonName};
|
||||
|
||||
// from Niko's talk
|
||||
/* fn type_check(expression, expected_ty) -> Ty {
|
||||
let ty = bare_type_check(expression, expected_type);
|
||||
if ty icompatible with expected_ty {
|
||||
try_coerce(expression, ty, expected_ty)
|
||||
} else {
|
||||
ty
|
||||
}
|
||||
}
|
||||
|
||||
fn bare_type_check(exprssion, expected_type) -> Ty { ... }
|
||||
*/
|
||||
|
||||
/* H-M ALGO NOTES
|
||||
from https://www.youtube.com/watch?v=il3gD7XMdmA
|
||||
(also check out http://dev.stephendiehl.com/fun/006_hindley_milner.html)
|
||||
|
||||
typeInfer :: Expr a -> Matching (Type a)
|
||||
unify :: Type a -> Type b -> Matching (Type c)
|
||||
|
||||
(Matching a) is a monad in which unification is done
|
||||
|
||||
ex:
|
||||
|
||||
typeInfer (If e1 e2 e3) = do
|
||||
t1 <- typeInfer e1
|
||||
t2 <- typeInfer e2
|
||||
t3 <- typeInfer e3
|
||||
_ <- unify t1 BoolType
|
||||
unify t2 t3 -- b/c t2 and t3 have to be the same type
|
||||
|
||||
typeInfer (Const (ConstInt _)) = IntType -- same for other literals
|
||||
|
||||
--function application
|
||||
typeInfer (Apply f x) = do
|
||||
tf <- typeInfer f
|
||||
tx <- typeInfer x
|
||||
case tf of
|
||||
FunctionType t1 t2 -> do
|
||||
_ <- unify t1 tx
|
||||
return t2
|
||||
_ -> fail "Not a function"
|
||||
|
||||
--type annotation
|
||||
typeInfer (Typed x t) = do
|
||||
tx <- typeInfer x
|
||||
unify tx t
|
||||
|
||||
--variable and let expressions - need to pass around a map of variable names to types here
|
||||
typeInfer :: [ (Var, Type Var) ] -> Expr Var -> Matching (Type Var)
|
||||
|
||||
typeInfer ctx (Var x) = case (lookup x ctx) of
|
||||
Just t -> return t
|
||||
Nothing -> fail "Unknown variable"
|
||||
|
||||
--let x = e1 in e2
|
||||
typeInfer ctx (Let x e1 e2) = do
|
||||
t1 <- typeInfer ctx e1
|
||||
typeInfer ((x, t1) :: ctx) e2
|
||||
|
||||
--lambdas are complicated (this represents ʎx.e)
|
||||
typeInfer ctx (Lambda x e) = do
|
||||
t1 <- allocExistentialVariable
|
||||
t2 <- typeInfer ((x, t1) :: ctx) e
|
||||
return $ FunctionType t1 t2 -- ie. t1 -> t2
|
||||
|
||||
|
||||
--to solve the problem of map :: (a -> b) -> [a] -> [b]
|
||||
when we use a variable whose type has universal tvars, convert those universal
|
||||
tvars to existential ones
|
||||
-and each distinct universal tvar needs to map to the same existential type
|
||||
|
||||
-so we change typeinfer:
|
||||
typeInfer ctx (Var x) = do
|
||||
case (lookup x ctx) of
|
||||
Nothing -> ...
|
||||
Just t -> do
|
||||
let uvars = nub (toList t) -- nub removes duplicates, so this gets unique universally quantified variables
|
||||
evars <- mapM (const allocExistentialVariable) uvars
|
||||
let varMap = zip uvars evars
|
||||
let vixVar varMap v = fromJust $ lookup v varMap
|
||||
return (fmap (fixVar varMap) t)
|
||||
|
||||
--how do we define unify??
|
||||
|
||||
-recall, type signature is:
|
||||
unify :: Type a -> Type b -> Matching (Type c)
|
||||
unify BoolType BoolType = BoolType --easy, same for all constants
|
||||
unify (FunctionType t1 t2) (FunctionType t3 t4) = do
|
||||
t5 <- unify t1 t3
|
||||
t6 <- unify t2 t4
|
||||
return $ FunctionType t5 t6
|
||||
unify (TVar a) (TVar b) = if a == b then TVar a else fail
|
||||
--existential types can be assigned another type at most once
|
||||
--some complicated stuff about hanlding existential types
|
||||
--everything else is a type error
|
||||
unify a b = fail
|
||||
|
||||
|
||||
SKOLEMIZATION - how you prevent an unassigned existential type variable from leaking!
|
||||
-before a type gets to global scope, replace all unassigned existential vars w/ new unique universal
|
||||
type variables
|
||||
|
||||
*/
|
||||
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
pub enum Type {
|
||||
TVar(TypeVar),
|
||||
TConst(TypeConst),
|
||||
TFunc(Box<Type>, Box<Type>),
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
pub enum TypeVar {
|
||||
Univ(Rc<String>),
|
||||
Exist(u64),
|
||||
}
|
||||
impl TypeVar {
|
||||
fn univ(label: &str) -> TypeVar {
|
||||
TypeVar::Univ(Rc::new(label.to_string()))
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
pub enum TypeConst {
|
||||
UserT(Rc<String>),
|
||||
Integer,
|
||||
Float,
|
||||
StringT,
|
||||
Boolean,
|
||||
Unit,
|
||||
Bottom,
|
||||
}
|
||||
|
||||
type TypeCheckResult = Result<Type, String>;
|
||||
|
||||
#[derive(Debug, PartialEq, Eq, Hash)]
|
||||
struct PathSpecifier(Rc<String>);
|
||||
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
struct TypeContextEntry {
|
||||
ty: Type,
|
||||
constant: bool
|
||||
}
|
||||
|
||||
pub struct TypeContext {
|
||||
symbol_table: HashMap<PathSpecifier, TypeContextEntry>,
|
||||
evar_table: HashMap<u64, Type>,
|
||||
existential_type_label_count: u64
|
||||
}
|
||||
|
||||
impl TypeContext {
|
||||
pub fn new() -> TypeContext {
|
||||
TypeContext {
|
||||
symbol_table: HashMap::new(),
|
||||
evar_table: HashMap::new(),
|
||||
existential_type_label_count: 0,
|
||||
}
|
||||
}
|
||||
pub fn add_symbols(&mut self, ast: &AST) {
|
||||
use self::Declaration::*;
|
||||
use self::Type::*;
|
||||
use self::TypeConst::*;
|
||||
|
||||
for statement in ast.0.iter() {
|
||||
match *statement {
|
||||
Statement::ExpressionStatement(_) => (),
|
||||
Statement::Declaration(ref decl) => match *decl {
|
||||
FuncSig(_) => (),
|
||||
Impl { .. } => (),
|
||||
TypeDecl(ref type_constructor, ref body) => {
|
||||
for variant in body.0.iter() {
|
||||
let (spec, ty) = match variant {
|
||||
&Variant::UnitStruct(ref data_constructor) => {
|
||||
let spec = PathSpecifier(data_constructor.clone());
|
||||
let ty = TConst(UserT(type_constructor.name.clone()));
|
||||
(spec, ty)
|
||||
},
|
||||
&Variant::TupleStruct(ref data_construcor, ref args) => {
|
||||
//TODO fix
|
||||
let arg = args.get(0).unwrap();
|
||||
let type_arg = self.from_anno(arg);
|
||||
let spec = PathSpecifier(data_construcor.clone());
|
||||
let ty = TFunc(Box::new(type_arg), Box::new(TConst(UserT(type_constructor.name.clone()))));
|
||||
(spec, ty)
|
||||
},
|
||||
&Variant::Record(_, _) => unimplemented!(),
|
||||
};
|
||||
let entry = TypeContextEntry { ty, constant: true };
|
||||
self.symbol_table.insert(spec, entry);
|
||||
}
|
||||
},
|
||||
TypeAlias { .. } => (),
|
||||
Binding {ref name, ref constant, ref expr} => {
|
||||
let spec = PathSpecifier(name.clone());
|
||||
let ty = expr.1.as_ref()
|
||||
.map(|ty| self.from_anno(ty))
|
||||
.unwrap_or_else(|| { self.alloc_existential_type() }); // this call to alloc_existential is OK b/c a binding only ever has one type, so if the annotation is absent, it's fine to just make one de novo
|
||||
let entry = TypeContextEntry { ty, constant: *constant };
|
||||
self.symbol_table.insert(spec, entry);
|
||||
},
|
||||
FuncDecl(ref signature, _) => {
|
||||
let spec = PathSpecifier(signature.name.clone());
|
||||
let ty = self.from_signature(signature);
|
||||
let entry = TypeContextEntry { ty, constant: true };
|
||||
self.symbol_table.insert(spec, entry);
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
fn lookup(&mut self, binding: &Rc<String>) -> Option<TypeContextEntry> {
|
||||
let key = PathSpecifier(binding.clone());
|
||||
self.symbol_table.get(&key).map(|entry| entry.clone())
|
||||
}
|
||||
pub fn debug_symbol_table(&self) -> String {
|
||||
format!("Symbol table:\n {:?}\nEvar table:\n{:?}", self.symbol_table, self.evar_table)
|
||||
}
|
||||
fn alloc_existential_type(&mut self) -> Type {
|
||||
let ret = Type::TVar(TypeVar::Exist(self.existential_type_label_count));
|
||||
self.existential_type_label_count += 1;
|
||||
ret
|
||||
}
|
||||
|
||||
fn from_anno(&mut self, anno: &TypeName) -> Type {
|
||||
use self::Type::*;
|
||||
use self::TypeConst::*;
|
||||
|
||||
match anno {
|
||||
&TypeName::Singleton(TypeSingletonName { ref name, .. }) => {
|
||||
match name.as_ref().as_ref() {
|
||||
"Int" => TConst(Integer),
|
||||
"Float" => TConst(Float),
|
||||
"Bool" => TConst(Boolean),
|
||||
"String" => TConst(StringT),
|
||||
s => TVar(TypeVar::Univ(Rc::new(format!("{}",s)))),
|
||||
}
|
||||
},
|
||||
&TypeName::Tuple(ref items) => {
|
||||
if items.len() == 1 {
|
||||
TConst(Unit)
|
||||
} else {
|
||||
TConst(Bottom)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn from_signature(&mut self, sig: &Signature) -> Type {
|
||||
use self::Type::*;
|
||||
use self::TypeConst::*;
|
||||
|
||||
//TODO this won't work properly until you make sure that all (universal) type vars in the function have the same existential type var
|
||||
// actually this should never even put existential types into the symbol table at all
|
||||
|
||||
//this will crash if more than 5 arg function is used
|
||||
let names = vec!["a", "b", "c", "d", "e", "f"];
|
||||
let mut idx = 0;
|
||||
|
||||
let mut get_type = || { let q = TVar(TypeVar::Univ(Rc::new(format!("{}", names.get(idx).unwrap())))); idx += 1; q };
|
||||
|
||||
let return_type = sig.type_anno.as_ref().map(|anno| self.from_anno(&anno)).unwrap_or_else(|| { get_type() });
|
||||
if sig.params.len() == 0 {
|
||||
TFunc(Box::new(TConst(Unit)), Box::new(return_type))
|
||||
} else {
|
||||
let mut output_type = return_type;
|
||||
for p in sig.params.iter() {
|
||||
let p_type = p.1.as_ref().map(|anno| self.from_anno(anno)).unwrap_or_else(|| { get_type() });
|
||||
output_type = TFunc(Box::new(p_type), Box::new(output_type));
|
||||
}
|
||||
output_type
|
||||
}
|
||||
}
|
||||
|
||||
pub fn type_check(&mut self, ast: &AST) -> TypeCheckResult {
|
||||
use self::Type::*;
|
||||
use self::TypeConst::*;
|
||||
|
||||
let mut last = TConst(Unit);
|
||||
|
||||
for statement in ast.0.iter() {
|
||||
match statement {
|
||||
&Statement::Declaration(ref _decl) => {
|
||||
//return Err(format!("Declarations not supported"));
|
||||
},
|
||||
&Statement::ExpressionStatement(ref expr) => {
|
||||
last = self.infer(expr)?;
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(last)
|
||||
}
|
||||
fn infer(&mut self, expr: &Expression) -> TypeCheckResult {
|
||||
match (&expr.0, &expr.1) {
|
||||
(exprtype, &Some(ref anno)) => {
|
||||
let tx = self.infer_no_anno(exprtype)?;
|
||||
let ty = self.from_anno(anno);
|
||||
self.unify(tx, ty)
|
||||
},
|
||||
(exprtype, &None) => self.infer_no_anno(exprtype),
|
||||
}
|
||||
}
|
||||
|
||||
fn infer_no_anno(&mut self, ex: &ExpressionType) -> TypeCheckResult {
|
||||
use self::ExpressionType::*;
|
||||
use self::Type::*;
|
||||
use self::TypeConst::*;
|
||||
|
||||
Ok(match ex {
|
||||
&IntLiteral(_) => TConst(Integer),
|
||||
&FloatLiteral(_) => TConst(Float),
|
||||
&StringLiteral(_) => TConst(StringT),
|
||||
&BoolLiteral(_) => TConst(Boolean),
|
||||
&Value(ref name, _) => {
|
||||
self.lookup(name)
|
||||
.map(|entry| entry.ty)
|
||||
.ok_or(format!("Couldn't find {}", name))?
|
||||
},
|
||||
&BinExp(ref op, ref lhs, ref rhs) => {
|
||||
let t_lhs = self.infer(lhs)?;
|
||||
match self.infer_op(op)? {
|
||||
TFunc(t1, t2) => {
|
||||
let _ = self.unify(t_lhs, *t1)?;
|
||||
let t_rhs = self.infer(rhs)?;
|
||||
let x = *t2;
|
||||
match x {
|
||||
TFunc(t3, t4) => {
|
||||
let _ = self.unify(t_rhs, *t3)?;
|
||||
*t4
|
||||
},
|
||||
_ => return Err(format!("Not a function type either")),
|
||||
}
|
||||
},
|
||||
_ => return Err(format!("Op {:?} is not a function type", op)),
|
||||
}
|
||||
},
|
||||
&Call { ref f, ref arguments } => {
|
||||
let tf = self.infer(f)?;
|
||||
let targ = self.infer(arguments.get(0).unwrap())?;
|
||||
match tf {
|
||||
TFunc(box t1, box t2) => {
|
||||
let _ = self.unify(t1, targ)?;
|
||||
t2
|
||||
},
|
||||
_ => return Err(format!("Not a function!")),
|
||||
}
|
||||
},
|
||||
_ => TConst(Bottom),
|
||||
})
|
||||
}
|
||||
|
||||
fn infer_op(&mut self, op: &Operation) -> TypeCheckResult {
|
||||
use self::Type::*;
|
||||
use self::TypeConst::*;
|
||||
macro_rules! binoptype {
|
||||
($lhs:expr, $rhs:expr, $out:expr) => { TFunc(Box::new($lhs), Box::new(TFunc(Box::new($rhs), Box::new($out)))) };
|
||||
}
|
||||
|
||||
Ok(match (*op.0).as_ref() {
|
||||
"+" => binoptype!(TConst(Integer), TConst(Integer), TConst(Integer)),
|
||||
"++" => binoptype!(TConst(StringT), TConst(StringT), TConst(StringT)),
|
||||
"-" => binoptype!(TConst(Integer), TConst(Integer), TConst(Integer)),
|
||||
"*" => binoptype!(TConst(Integer), TConst(Integer), TConst(Integer)),
|
||||
"/" => binoptype!(TConst(Integer), TConst(Integer), TConst(Integer)),
|
||||
"%" => binoptype!(TConst(Integer), TConst(Integer), TConst(Integer)),
|
||||
_ => TConst(Bottom)
|
||||
})
|
||||
}
|
||||
|
||||
fn unify(&mut self, t1: Type, t2: Type) -> TypeCheckResult {
|
||||
use self::Type::*;
|
||||
use self::TypeVar::*;
|
||||
|
||||
println!("Calling unify with `{:?}` and `{:?}`", t1, t2);
|
||||
|
||||
match (&t1, &t2) {
|
||||
(&TConst(ref c1), &TConst(ref c2)) if c1 == c2 => Ok(TConst(c1.clone())),
|
||||
(&TFunc(ref t1, ref t2), &TFunc(ref t3, ref t4)) => {
|
||||
let t5 = self.unify(*t1.clone().clone(), *t3.clone().clone())?;
|
||||
let t6 = self.unify(*t2.clone().clone(), *t4.clone().clone())?;
|
||||
Ok(TFunc(Box::new(t5), Box::new(t6)))
|
||||
},
|
||||
(&TVar(Univ(ref a)), &TVar(Univ(ref b))) => {
|
||||
if a == b {
|
||||
Ok(TVar(Univ(a.clone())))
|
||||
} else {
|
||||
Err(format!("Couldn't unify universal types {} and {}", a, b))
|
||||
}
|
||||
},
|
||||
//the interesting case!!
|
||||
(&TVar(Exist(ref a)), ref t2) => {
|
||||
let x = self.evar_table.get(a).map(|x| x.clone());
|
||||
match x {
|
||||
Some(ref t1) => self.unify(t1.clone().clone(), t2.clone().clone()),
|
||||
None => {
|
||||
self.evar_table.insert(*a, t2.clone().clone());
|
||||
Ok(t2.clone().clone())
|
||||
}
|
||||
}
|
||||
},
|
||||
(ref t1, &TVar(Exist(ref a))) => {
|
||||
let x = self.evar_table.get(a).map(|x| x.clone());
|
||||
match x {
|
||||
Some(ref t2) => self.unify(t2.clone().clone(), t1.clone().clone()),
|
||||
None => {
|
||||
self.evar_table.insert(*a, t1.clone().clone());
|
||||
Ok(t1.clone().clone())
|
||||
}
|
||||
}
|
||||
},
|
||||
_ => Err(format!("Types {:?} and {:?} don't unify", t1, t2))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{Type, TypeVar, TypeConst, TypeContext};
|
||||
use super::Type::*;
|
||||
use super::TypeConst::*;
|
||||
use schala_lang::parsing::{parse, tokenize};
|
||||
|
||||
macro_rules! type_test {
|
||||
($input:expr, $correct:expr) => {
|
||||
{
|
||||
let mut tc = TypeContext::new();
|
||||
let ast = parse(tokenize($input)).0.unwrap() ;
|
||||
tc.add_symbols(&ast);
|
||||
assert_eq!($correct, tc.type_check(&ast).unwrap())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn basic_inference() {
|
||||
type_test!("30", TConst(Integer));
|
||||
type_test!("fn x(a: Int): Bool {}; x(1)", TConst(Boolean));
|
||||
}
|
||||
}
|
|
@ -1,254 +0,0 @@
|
|||
use std::rc::Rc;
|
||||
use std::collections::HashMap;
|
||||
use std::char;
|
||||
use std::fmt;
|
||||
use std::fmt::Write;
|
||||
|
||||
use itertools::Itertools;
|
||||
|
||||
use schala_lang::parsing;
|
||||
|
||||
pub struct TypeContext {
|
||||
type_var_count: u64,
|
||||
bindings: HashMap<Rc<String>, Type>,
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
pub enum Type {
|
||||
Const(TConst),
|
||||
Sum(Vec<Type>),
|
||||
Func(Box<Type>, Box<Type>),
|
||||
UVar(String),
|
||||
EVar(u64),
|
||||
Void
|
||||
}
|
||||
|
||||
impl fmt::Display for Type {
|
||||
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
|
||||
use self::Type::*;
|
||||
match self {
|
||||
&Const(ref c) => write!(f, "{:?}", c),
|
||||
&Sum(ref types) => {
|
||||
write!(f, "(")?;
|
||||
for item in types.iter().map(|ty| Some(ty)).intersperse(None) {
|
||||
match item {
|
||||
Some(ty) => write!(f, "{}", ty)?,
|
||||
None => write!(f, ",")?,
|
||||
};
|
||||
}
|
||||
write!(f, ")")
|
||||
},
|
||||
&Func(ref a, ref b) => write!(f, "{} -> {}", a, b),
|
||||
&UVar(ref s) => write!(f, "{}_u", s),
|
||||
&EVar(ref n) => write!(f, "{}_e", n),
|
||||
&Void => write!(f, "Void")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct UVarGenerator {
|
||||
n: u32,
|
||||
}
|
||||
impl UVarGenerator {
|
||||
fn new() -> UVarGenerator {
|
||||
UVarGenerator::default()
|
||||
}
|
||||
fn next(&mut self) -> Type {
|
||||
//TODO handle this in the case where someone wants to make a function with more than 26 variables
|
||||
let s = format!("{}", unsafe { char::from_u32_unchecked(self.n + ('a' as u32)) });
|
||||
self.n += 1;
|
||||
Type::UVar(s)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Clone)]
|
||||
pub enum TConst {
|
||||
Unit,
|
||||
Int,
|
||||
Float,
|
||||
StringT,
|
||||
Bool,
|
||||
Custom(String),
|
||||
}
|
||||
|
||||
impl parsing::TypeName {
|
||||
fn to_type(&self) -> TypeResult<Type> {
|
||||
use self::parsing::TypeSingletonName;
|
||||
use self::parsing::TypeName::*;
|
||||
use self::Type::*; use self::TConst::*;
|
||||
Ok(match self {
|
||||
&Tuple(_) => return Err(format!("Tuples not yet implemented")),
|
||||
&Singleton(ref name) => match name {
|
||||
&TypeSingletonName { ref name, .. } => match &name[..] {
|
||||
"Int" => Const(Int),
|
||||
"Float" => Const(Float),
|
||||
"Bool" => Const(Bool),
|
||||
"String" => Const(StringT),
|
||||
n => Const(Custom(n.to_string()))
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
pub type TypeResult<T> = Result<T, String>;
|
||||
|
||||
impl TypeContext {
|
||||
pub fn new() -> TypeContext {
|
||||
TypeContext { bindings: HashMap::new(), type_var_count: 0 }
|
||||
}
|
||||
pub fn fresh(&mut self) -> Type {
|
||||
let ret = self.type_var_count;
|
||||
self.type_var_count += 1;
|
||||
Type::EVar(ret)
|
||||
}
|
||||
}
|
||||
|
||||
impl TypeContext {
|
||||
pub fn add_top_level_types(&mut self, ast: &parsing::AST) -> TypeResult<()> {
|
||||
use self::parsing::TypeName;
|
||||
use self::parsing::Declaration::*;
|
||||
use self::Type::*;
|
||||
for statement in ast.0.iter() {
|
||||
if let &self::parsing::Statement::Declaration(ref decl) = statement {
|
||||
match decl {
|
||||
&FuncSig(ref signature) | &FuncDecl(ref signature, _) => {
|
||||
let mut uvar_gen = UVarGenerator::new();
|
||||
let mut ty: Type = signature.type_anno.as_ref().map(|name: &TypeName| name.to_type()).unwrap_or_else(|| {Ok(uvar_gen.next())} )?;
|
||||
for &(_, ref type_name) in signature.params.iter().rev() {
|
||||
let arg_type = type_name.as_ref().map(|name| name.to_type()).unwrap_or_else(|| {Ok(uvar_gen.next())} )?;
|
||||
ty = Func(bx!(arg_type), bx!(ty));
|
||||
}
|
||||
self.bindings.insert(signature.name.clone(), ty);
|
||||
},
|
||||
_ => ()
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
pub fn debug_symbol_table(&self) -> String {
|
||||
let mut output = format!("Symbols\n");
|
||||
for (sym, ty) in &self.bindings {
|
||||
write!(output, "{} : {}\n", sym, ty).unwrap();
|
||||
}
|
||||
output
|
||||
}
|
||||
}
|
||||
|
||||
impl TypeContext {
|
||||
pub fn type_check_ast(&mut self, ast: &parsing::AST) -> TypeResult<Type> {
|
||||
use self::Type::*; use self::TConst::*;
|
||||
let mut ret_type = Const(Unit);
|
||||
for statement in ast.0.iter() {
|
||||
ret_type = self.type_check_statement(statement)?;
|
||||
}
|
||||
Ok(ret_type)
|
||||
}
|
||||
fn type_check_statement(&mut self, statement: &parsing::Statement) -> TypeResult<Type> {
|
||||
use self::parsing::Statement::*;
|
||||
match statement {
|
||||
&ExpressionStatement(ref expr) => self.infer(expr),
|
||||
&Declaration(ref decl) => self.add_declaration(decl),
|
||||
}
|
||||
}
|
||||
fn add_declaration(&mut self, decl: &parsing::Declaration) -> TypeResult<Type> {
|
||||
use self::parsing::Declaration::*;
|
||||
use self::Type::*;
|
||||
match decl {
|
||||
&Binding { ref name, ref expr, .. } => {
|
||||
let ty = self.infer(expr)?;
|
||||
self.bindings.insert(name.clone(), ty);
|
||||
},
|
||||
_ => return Err(format!("other formats not done"))
|
||||
}
|
||||
Ok(Void)
|
||||
}
|
||||
fn infer(&mut self, expr: &parsing::Expression) -> TypeResult<Type> {
|
||||
use self::parsing::Expression;
|
||||
match expr {
|
||||
&Expression(ref e, Some(ref anno)) => {
|
||||
let anno_ty = anno.to_type()?;
|
||||
let ty = self.infer_exprtype(&e)?;
|
||||
self.unify(ty, anno_ty)
|
||||
},
|
||||
&Expression(ref e, None) => self.infer_exprtype(e)
|
||||
}
|
||||
}
|
||||
fn infer_exprtype(&mut self, expr: &parsing::ExpressionType) -> TypeResult<Type> {
|
||||
use self::parsing::ExpressionType::*;
|
||||
use self::Type::*; use self::TConst::*;
|
||||
match expr {
|
||||
&IntLiteral(_) => Ok(Const(Int)),
|
||||
&FloatLiteral(_) => Ok(Const(Float)),
|
||||
&StringLiteral(_) => Ok(Const(StringT)),
|
||||
&BoolLiteral(_) => Ok(Const(Bool)),
|
||||
&BinExp(ref op, ref lhs, ref rhs) => { /* remember there are both the haskell convention talk and the write you a haskell ways to do this! */
|
||||
match op.get_type()? {
|
||||
Func(box t1, box Func(box t2, box t3)) => {
|
||||
let lhs_ty = self.infer(lhs)?;
|
||||
let rhs_ty = self.infer(rhs)?;
|
||||
self.unify(t1, lhs_ty)?;
|
||||
self.unify(t2, rhs_ty)?;
|
||||
Ok(t3)
|
||||
},
|
||||
other => Err(format!("{:?} is not a binary function type", other))
|
||||
}
|
||||
},
|
||||
&PrefixExp(ref op, ref expr) => match op.get_type()? {
|
||||
Func(box t1, box t2) => {
|
||||
let expr_ty = self.infer(expr)?;
|
||||
self.unify(t1, expr_ty)?;
|
||||
Ok(t2)
|
||||
},
|
||||
other => Err(format!("{:?} is not a prefix op function type", other))
|
||||
},
|
||||
&Value(ref name) => {
|
||||
match self.bindings.get(name) {
|
||||
Some(ty) => Ok(ty.clone()),
|
||||
None => Err(format!("No binding found for variable: {}", name)),
|
||||
}
|
||||
},
|
||||
&Call { ref f, ref arguments } => {
|
||||
let mut tf = self.infer(f)?;
|
||||
for arg in arguments.iter() {
|
||||
match tf {
|
||||
Func(box t, box rest) => {
|
||||
let t_arg = self.infer(arg)?;
|
||||
self.unify(t, t_arg)?;
|
||||
tf = rest;
|
||||
},
|
||||
other => return Err(format!("Function call failed to unify; last type: {:?}", other)),
|
||||
}
|
||||
}
|
||||
Ok(tf)
|
||||
},
|
||||
&TupleLiteral(ref expressions) => {
|
||||
let mut types = vec![];
|
||||
for expr in expressions {
|
||||
types.push(self.infer(expr)?);
|
||||
}
|
||||
Ok(Sum(types))
|
||||
},
|
||||
/*
|
||||
Index {
|
||||
indexee: Box<Expression>,
|
||||
indexers: Vec<Expression>,
|
||||
},
|
||||
IfExpression(Box<Expression>, Vec<Statement>, Option<Vec<Statement>>),
|
||||
MatchExpression(Box<Expression>, Vec<MatchArm>),
|
||||
ForExpression
|
||||
*/
|
||||
_ => Err(format!("Type not yet implemented"))
|
||||
}
|
||||
}
|
||||
fn unify(&mut self, t1: Type, t2: Type) -> TypeResult<Type> {
|
||||
use self::Type::*;// use self::TConst::*;
|
||||
match (t1, t2) {
|
||||
(Const(ref a), Const(ref b)) if a == b => Ok(Const(a.clone())),
|
||||
(a, b) => Err(format!("Types {:?} and {:?} don't unify", a, b))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1,17 +0,0 @@
|
|||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Schala Metainterpreter Web Evaluator</title>
|
||||
<style>
|
||||
.CodeArea {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="main">
|
||||
</div>
|
||||
<script src="bundle.js"></script>
|
||||
</body>
|
||||
</html>
|
|
@ -1,64 +0,0 @@
|
|||
const React = require("react");
|
||||
const ReactDOM = require("react-dom");
|
||||
const superagent = require("superagent");
|
||||
|
||||
const serverAddress = "http://localhost:8000";
|
||||
|
||||
class CodeArea extends React.Component {
|
||||
constructor(props) {
|
||||
super(props);
|
||||
this.state = {value: "", lastOutput: null};
|
||||
this.handleChange = this.handleChange.bind(this);
|
||||
this.submit = this.submit.bind(this);
|
||||
}
|
||||
|
||||
handleChange(event) {
|
||||
this.setState({value: event.target.value});
|
||||
}
|
||||
|
||||
submit(event) {
|
||||
console.log("Event", this.state.value);
|
||||
const source = this.state.value;
|
||||
|
||||
superagent.post(`${serverAddress}/input`)
|
||||
.send({ source })
|
||||
.set("accept", "json")
|
||||
.end((error, response) => {
|
||||
if (response) {
|
||||
console.log("Resp", response);
|
||||
this.setState({lastOutput: response.body.text})
|
||||
} else {
|
||||
console.error("Error: ", error);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
renderOutput() {
|
||||
if (!this.state.lastOutput) {
|
||||
return null;
|
||||
}
|
||||
return <textarea readOnly value={ this.state.lastOutput } />;
|
||||
}
|
||||
|
||||
render() {
|
||||
return (<div className="CodeArea">
|
||||
<div className="input">
|
||||
<textarea value={ this.state.value } onChange={this.handleChange}>
|
||||
</textarea>
|
||||
<button onClick={ this.submit }>Run!</button>
|
||||
</div>
|
||||
<div className="output">
|
||||
{ this.renderOutput() }
|
||||
</div>
|
||||
</div>);
|
||||
}
|
||||
}
|
||||
|
||||
const main = (<div>
|
||||
<h1>Schala web input</h1>
|
||||
<p>Write your source code here</p>
|
||||
<CodeArea/>
|
||||
</div>);
|
||||
|
||||
const rootDom = document.getElementById("main");
|
||||
ReactDOM.render(main, rootDom);
|
|
@ -1,27 +0,0 @@
|
|||
{
|
||||
"name": "static",
|
||||
"version": "1.0.0",
|
||||
"main": "index.js",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"babel": "^6.23.0",
|
||||
"babel-preset-es2015": "^6.24.1",
|
||||
"babel-preset-react": "^6.24.1",
|
||||
"babelify": "^7.3.0",
|
||||
"browserify": "^14.4.0",
|
||||
"react": "^15.6.1",
|
||||
"react-dom": "^15.6.1",
|
||||
"superagent": "^3.6.3",
|
||||
"uglify-js": "^3.1.1"
|
||||
},
|
||||
"babel": {
|
||||
"presets": [
|
||||
"babel-preset-react",
|
||||
"babel-preset-es2015"
|
||||
]
|
||||
},
|
||||
"scripts": {
|
||||
"build": "browserify main.jsx -t babelify -o bundle.js",
|
||||
"build-minify": "browserify main.jsx -t babelify | uglifyjs > bundle.js"
|
||||
}
|
||||
}
|
1728
static/yarn.lock
1728
static/yarn.lock
File diff suppressed because it is too large
Load Diff
|
@ -0,0 +1,2 @@
|
|||
/target
|
||||
/Cargo.lock
|
|
@ -0,0 +1,13 @@
|
|||
[package]
|
||||
name = "parser-combinator"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
|
||||
|
||||
[dependencies]
|
||||
arbitrary = "1.2.0"
|
||||
proptest = "1.0.0"
|
||||
|
||||
[dev-dependencies]
|
||||
rstest = "0.16.0"
|
|
@ -0,0 +1,10 @@
|
|||
# Rust Parser Combinator
|
||||
|
||||
This is a super-basic Rust parser combinator library I wrote mostly
|
||||
as an exercise for myself. Inspired by [nom](https://github.com/rust-bakery/nom)
|
||||
and [chumsky](https://github.com/zesterer/chumsky)
|
||||
|
||||
## Ideas for future work
|
||||
|
||||
* See if some of the ideas in [Efficient Parsing with Parser Combinators](https://research.rug.nl/en/publications/efficient-parsing-with-parser-combinators)
|
||||
can be incorporated here.
|
|
@ -0,0 +1,198 @@
|
|||
use crate::parser::{ParseResult, Parser, ParserInput, Representation};
|
||||
|
||||
pub fn choice2<P1, P2, I, O, E>(parser1: P1, parser2: P2) -> impl Parser<I, O, E>
|
||||
where
|
||||
P1: Parser<I, O, E>,
|
||||
P2: Parser<I, O, E>,
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
choice((parser1, parser2))
|
||||
}
|
||||
|
||||
pub fn choice<C, I, O, E>(choices: C) -> impl Parser<I, O, E>
|
||||
where
|
||||
C: Choice<I, O, E>,
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
let rep = choices.representation();
|
||||
(move |input| choices.parse(input), rep)
|
||||
}
|
||||
|
||||
pub trait Choice<I: Clone, O, E> {
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E>;
|
||||
fn representation(&self) -> Representation;
|
||||
}
|
||||
|
||||
impl<I, O, E, P1, P2> Choice<I, O, E> for (P1, P2)
|
||||
where
|
||||
P1: Parser<I, O, E>,
|
||||
P2: Parser<I, O, E>,
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E> {
|
||||
let parsers = vec![&self.0 as &dyn Parser<I, O, E>, &self.1];
|
||||
choice_loop(input, parsers)
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
let parsers = vec![&self.0 as &dyn Parser<I, O, E>, &self.1];
|
||||
repr_loop(parsers)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, O, E, P1, P2, P3> Choice<I, O, E> for (P1, P2, P3)
|
||||
where
|
||||
P1: Parser<I, O, E>,
|
||||
P2: Parser<I, O, E>,
|
||||
P3: Parser<I, O, E>,
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E> {
|
||||
let parsers = vec![&self.0 as &dyn Parser<I, O, E>, &self.1, &self.2];
|
||||
choice_loop(input, parsers)
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
let parsers = vec![&self.0 as &dyn Parser<I, O, E>, &self.1, &self.2];
|
||||
repr_loop(parsers)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, O, E, P1, P2, P3, P4> Choice<I, O, E> for (P1, P2, P3, P4)
|
||||
where
|
||||
P1: Parser<I, O, E>,
|
||||
P2: Parser<I, O, E>,
|
||||
P3: Parser<I, O, E>,
|
||||
P4: Parser<I, O, E>,
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E> {
|
||||
let parsers = vec![&self.0 as &dyn Parser<I, O, E>, &self.1, &self.2, &self.3];
|
||||
choice_loop(input, parsers)
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
let parsers = vec![&self.0 as &dyn Parser<I, O, E>, &self.1, &self.2, &self.3];
|
||||
repr_loop(parsers)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, O, E, P1, P2, P3, P4, P5> Choice<I, O, E> for (P1, P2, P3, P4, P5)
|
||||
where
|
||||
P1: Parser<I, O, E>,
|
||||
P2: Parser<I, O, E>,
|
||||
P3: Parser<I, O, E>,
|
||||
P4: Parser<I, O, E>,
|
||||
P5: Parser<I, O, E>,
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E> {
|
||||
let parsers = vec![
|
||||
&self.0 as &dyn Parser<I, O, E>,
|
||||
&self.1,
|
||||
&self.2,
|
||||
&self.3,
|
||||
&self.4,
|
||||
];
|
||||
choice_loop(input, parsers)
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
let parsers = vec![
|
||||
&self.0 as &dyn Parser<I, O, E>,
|
||||
&self.1,
|
||||
&self.2,
|
||||
&self.3,
|
||||
&self.4,
|
||||
];
|
||||
repr_loop(parsers)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, O, E, P1, P2, P3, P4, P5, P6> Choice<I, O, E> for (P1, P2, P3, P4, P5, P6)
|
||||
where
|
||||
P1: Parser<I, O, E>,
|
||||
P2: Parser<I, O, E>,
|
||||
P3: Parser<I, O, E>,
|
||||
P4: Parser<I, O, E>,
|
||||
P5: Parser<I, O, E>,
|
||||
P6: Parser<I, O, E>,
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E> {
|
||||
let parsers = vec![
|
||||
&self.0 as &dyn Parser<I, O, E>,
|
||||
&self.1,
|
||||
&self.2,
|
||||
&self.3,
|
||||
&self.4,
|
||||
&self.5,
|
||||
];
|
||||
choice_loop(input, parsers)
|
||||
}
|
||||
fn representation(&self) -> Representation {
|
||||
let parsers = vec![
|
||||
&self.0 as &dyn Parser<I, O, E>,
|
||||
&self.1,
|
||||
&self.2,
|
||||
&self.3,
|
||||
&self.4,
|
||||
&self.5,
|
||||
];
|
||||
repr_loop(parsers)
|
||||
}
|
||||
}
|
||||
|
||||
fn choice_loop<I, O, E>(input: I, parsers: Vec<&dyn Parser<I, O, E>>) -> ParseResult<I, O, E>
|
||||
where
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
//TODO need a more principled way to return an error when no choices work
|
||||
let mut err = None;
|
||||
|
||||
for parser in parsers.iter() {
|
||||
match parser.parse(input.clone()) {
|
||||
Ok(result) => return Ok(result),
|
||||
Err(e) => {
|
||||
err = Some(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(err.unwrap())
|
||||
}
|
||||
|
||||
fn repr_loop<I, O, E>(parsers: Vec<&dyn Parser<I, O, E>>) -> Representation
|
||||
where
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
let mut iter = parsers.iter().map(|p| p.representation());
|
||||
Representation::from_choice(&mut iter)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::combinators::repeated;
|
||||
use crate::primitives::literal;
|
||||
|
||||
#[test]
|
||||
fn test_choice() {
|
||||
let p = choice2(
|
||||
literal("gnostika").to(1),
|
||||
repeated(literal(" ")).at_least(1).to(2),
|
||||
);
|
||||
assert_eq!(p.parse("gnostika twentynine"), Ok((1, " twentynine")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_several_choices() {
|
||||
let p = choice((
|
||||
literal("a").to(1),
|
||||
literal("q").to(10),
|
||||
repeated(literal("chutney")).to(200),
|
||||
literal("banana").to(10000),
|
||||
));
|
||||
|
||||
assert_eq!(p.parse("q drugs").unwrap(), (10, " drugs"));
|
||||
}
|
||||
}
|
|
@ -0,0 +1,16 @@
|
|||
use crate::parser::{Parser, ParserInput};
|
||||
|
||||
pub fn map<P, F, I, O1, O2, E>(parser: P, map_fn: F) -> impl Parser<I, O2, E>
|
||||
where
|
||||
I: ParserInput,
|
||||
P: Parser<I, O1, E>,
|
||||
F: Fn(O1) -> O2,
|
||||
{
|
||||
let rep = parser.representation();
|
||||
let p = move |input| {
|
||||
parser
|
||||
.parse(input)
|
||||
.map(|(result, rest)| (map_fn(result), rest))
|
||||
};
|
||||
(p, rep)
|
||||
}
|
|
@ -0,0 +1,66 @@
|
|||
mod map;
|
||||
mod optional;
|
||||
mod repeated;
|
||||
mod separated_by;
|
||||
|
||||
pub use map::map;
|
||||
pub use optional::optional;
|
||||
pub use repeated::repeated;
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::parser::Parser;
|
||||
use crate::primitives::literal;
|
||||
|
||||
#[test]
|
||||
fn test_map() {
|
||||
let lit_a = literal("a");
|
||||
let output = lit_a.map(|s| s.to_uppercase()).parse("a yolo");
|
||||
assert_eq!(output.unwrap(), ("A".to_string(), " yolo"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_one_or_more() {
|
||||
let p = repeated(literal("bongo ")).at_least(1);
|
||||
let input = "bongo bongo bongo bongo bongo ";
|
||||
|
||||
let (output, rest) = p.parse(input).unwrap();
|
||||
assert_eq!(rest, "");
|
||||
assert_eq!(output.len(), 5);
|
||||
|
||||
let (output, rest) = p.parse("bongo ecks").unwrap();
|
||||
assert_eq!(output.len(), 1);
|
||||
assert_eq!(rest, "ecks");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_separated_by() {
|
||||
let p = repeated(literal("garb").to(20))
|
||||
.separated_by(repeated(literal(" ")).at_least(1), false);
|
||||
|
||||
assert_eq!(
|
||||
p.parse("garb garb garb garb").unwrap(),
|
||||
(vec![20, 20, 20, 20], "")
|
||||
);
|
||||
|
||||
assert!(p.parse("garb garb garb garb ").is_err());
|
||||
|
||||
let p =
|
||||
repeated(literal("garb").to(20)).separated_by(repeated(literal(" ")).at_least(1), true);
|
||||
|
||||
assert_eq!(
|
||||
p.parse("garb garb garb garb").unwrap(),
|
||||
(vec![20, 20, 20, 20], "")
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
p.parse("garb garb garb garb ").unwrap(),
|
||||
(vec![20, 20, 20, 20], "")
|
||||
);
|
||||
assert_eq!(
|
||||
p.parse("garb garb garb garb q").unwrap(),
|
||||
(vec![20, 20, 20, 20], "q")
|
||||
);
|
||||
}
|
||||
}
|
|
@ -0,0 +1,17 @@
|
|||
use crate::parser::{Parser, ParserInput, Representation};
|
||||
|
||||
pub fn optional<P, I, O, E>(parser: P) -> impl Parser<I, Option<O>, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
let rep = Representation::from_choice(
|
||||
&mut [parser.representation(), Representation::new("ε")].into_iter(),
|
||||
);
|
||||
let p = move |input: I| match parser.parse(input.clone()) {
|
||||
Ok((output, rest)) => Ok((Some(output), rest)),
|
||||
Err(_e) => Ok((None, input)),
|
||||
};
|
||||
|
||||
(p, rep)
|
||||
}
|
|
@ -0,0 +1,94 @@
|
|||
use crate::combinators::separated_by::SeparatedBy;
|
||||
use crate::parser::{BoxedParser, ParseResult, Parser, ParserInput, Representation};
|
||||
|
||||
pub fn repeated<'a, P, I, O>(parser: P) -> Repeated<'a, I, O>
|
||||
where
|
||||
P: Parser<I, O, I> + 'a,
|
||||
I: ParserInput + Clone + 'a,
|
||||
{
|
||||
Repeated {
|
||||
inner_parser: BoxedParser::new(parser),
|
||||
at_least: None,
|
||||
at_most: None,
|
||||
}
|
||||
}
|
||||
|
||||
pub struct Repeated<'a, I, O>
|
||||
where
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
pub(super) inner_parser: BoxedParser<'a, I, O, I>,
|
||||
pub(super) at_least: Option<u16>,
|
||||
pub(super) at_most: Option<u16>,
|
||||
}
|
||||
|
||||
impl<'a, I, O> Repeated<'a, I, O>
|
||||
where
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
pub fn at_least(self, n: u16) -> Self {
|
||||
Self {
|
||||
at_least: Some(n),
|
||||
..self
|
||||
}
|
||||
}
|
||||
pub fn at_most(self, n: u16) -> Self {
|
||||
Self {
|
||||
at_most: Some(n),
|
||||
..self
|
||||
}
|
||||
}
|
||||
|
||||
pub fn separated_by<D, O2>(self, delimiter: D, allow_trailing: bool) -> SeparatedBy<'a, I, O>
|
||||
where
|
||||
D: Parser<I, O2, I> + 'a,
|
||||
O2: 'a,
|
||||
I: 'a,
|
||||
{
|
||||
SeparatedBy {
|
||||
inner_repeated: self,
|
||||
delimiter: BoxedParser::new(delimiter.to(())),
|
||||
allow_trailing,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a, I, O> Parser<I, Vec<O>, I> for Repeated<'a, I, O>
|
||||
where
|
||||
I: ParserInput + Clone + 'a,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, Vec<O>, I> {
|
||||
let at_least = self.at_least.unwrap_or(0);
|
||||
let at_most = self.at_most.unwrap_or(u16::MAX);
|
||||
|
||||
if at_most == 0 {
|
||||
return Ok((vec![], input));
|
||||
}
|
||||
|
||||
let mut results = Vec::new();
|
||||
let mut count: u16 = 0;
|
||||
let mut further_input = input.clone();
|
||||
|
||||
while let Ok((item, rest)) = self.inner_parser.parse(further_input.clone()) {
|
||||
results.push(item);
|
||||
further_input = rest;
|
||||
count += 1;
|
||||
if count >= at_most {
|
||||
break;
|
||||
}
|
||||
}
|
||||
if count < at_least {
|
||||
return Err(input);
|
||||
}
|
||||
|
||||
Ok((results, further_input))
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
Representation::repeated(
|
||||
self.inner_parser.representation(),
|
||||
self.at_least.unwrap_or(0),
|
||||
self.at_most.unwrap_or(u16::MAX),
|
||||
)
|
||||
}
|
||||
}
|
|
@ -0,0 +1,84 @@
|
|||
use crate::combinators::repeated::Repeated;
|
||||
use crate::parser::{BoxedParser, ParseResult, Parser, ParserInput, Representation};
|
||||
|
||||
pub struct SeparatedBy<'a, I, O>
|
||||
where
|
||||
I: ParserInput + Clone,
|
||||
{
|
||||
pub(super) inner_repeated: Repeated<'a, I, O>,
|
||||
pub(super) delimiter: BoxedParser<'a, I, (), I>,
|
||||
pub(super) allow_trailing: bool,
|
||||
}
|
||||
|
||||
impl<'a, I, O> Parser<I, Vec<O>, I> for SeparatedBy<'a, I, O>
|
||||
where
|
||||
I: ParserInput + Clone + 'a,
|
||||
{
|
||||
fn representation(&self) -> Representation {
|
||||
Representation::new("sepby")
|
||||
}
|
||||
|
||||
fn parse(&self, input: I) -> ParseResult<I, Vec<O>, I> {
|
||||
let at_least = self.inner_repeated.at_least.unwrap_or(0);
|
||||
let at_most = self.inner_repeated.at_most.unwrap_or(u16::MAX);
|
||||
let parser = &self.inner_repeated.inner_parser;
|
||||
let delimiter = &self.delimiter;
|
||||
|
||||
if at_most == 0 {
|
||||
return Ok((vec![], input));
|
||||
}
|
||||
|
||||
let mut results = Vec::new();
|
||||
let mut count: u16 = 0;
|
||||
let mut further_input;
|
||||
|
||||
match parser.parse(input.clone()) {
|
||||
Ok((item, rest)) => {
|
||||
results.push(item);
|
||||
further_input = rest;
|
||||
}
|
||||
Err(_e) => {
|
||||
if at_least > 0 {
|
||||
return Err(input);
|
||||
} else {
|
||||
return Ok((vec![], input));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
loop {
|
||||
match delimiter.parse(further_input.clone()) {
|
||||
Ok(((), rest)) => {
|
||||
further_input = rest;
|
||||
}
|
||||
Err(_e) => {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
match parser.parse(further_input.clone()) {
|
||||
Ok((item, rest)) => {
|
||||
results.push(item);
|
||||
further_input = rest;
|
||||
count += 1;
|
||||
}
|
||||
Err(_e) if self.allow_trailing => {
|
||||
break;
|
||||
}
|
||||
Err(e) => {
|
||||
return Err(e);
|
||||
}
|
||||
}
|
||||
|
||||
if count >= at_most {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if count < at_least {
|
||||
return Err(input);
|
||||
}
|
||||
|
||||
Ok((results, further_input))
|
||||
}
|
||||
}
|
|
@ -0,0 +1,7 @@
|
|||
pub mod choice;
|
||||
pub mod combinators;
|
||||
mod parser;
|
||||
pub mod primitives;
|
||||
pub mod sequence;
|
||||
|
||||
pub use parser::{ParseResult, Parser, ParserInput, Representation};
|
|
@ -0,0 +1,38 @@
|
|||
use crate::parser::{ParseResult, Parser, ParserInput, Representation};
|
||||
|
||||
pub struct BoxedParser<'a, I, O, E>
|
||||
where
|
||||
I: ParserInput,
|
||||
{
|
||||
inner: Box<dyn Parser<I, O, E> + 'a>,
|
||||
}
|
||||
|
||||
impl<'a, I, O, E> BoxedParser<'a, I, O, E>
|
||||
where
|
||||
I: ParserInput,
|
||||
{
|
||||
pub(crate) fn new<P>(inner: P) -> Self
|
||||
where
|
||||
P: Parser<I, O, E> + 'a,
|
||||
{
|
||||
BoxedParser {
|
||||
inner: Box::new(inner),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a, I: ParserInput, O, E> Parser<I, O, E> for BoxedParser<'a, I, O, E> {
|
||||
fn representation(&self) -> Representation {
|
||||
self.inner.representation()
|
||||
}
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E> {
|
||||
self.inner.parse(input)
|
||||
}
|
||||
|
||||
fn boxed<'b>(self) -> BoxedParser<'b, I, O, E>
|
||||
where
|
||||
Self: Sized + 'b,
|
||||
{
|
||||
self
|
||||
}
|
||||
}
|
|
@ -0,0 +1,179 @@
|
|||
mod boxed_parser;
|
||||
mod named_parser;
|
||||
mod parser_input;
|
||||
mod representation;
|
||||
|
||||
use std::rc::Rc;
|
||||
|
||||
pub use boxed_parser::BoxedParser;
|
||||
pub use named_parser::NamedParser;
|
||||
pub use parser_input::ParserInput;
|
||||
pub use representation::Representation;
|
||||
|
||||
pub type ParseResult<I, O, E> = Result<(O, I), E>;
|
||||
|
||||
pub trait Parser<I, O, E>
|
||||
where
|
||||
I: ParserInput,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E>;
|
||||
|
||||
fn representation(&self) -> Representation;
|
||||
|
||||
fn boxed<'a>(self) -> BoxedParser<'a, I, O, E>
|
||||
where
|
||||
Self: Sized + 'a,
|
||||
{
|
||||
BoxedParser::new(self)
|
||||
}
|
||||
|
||||
fn map<'a, F, O2>(self, map_fn: F) -> BoxedParser<'a, I, O2, E>
|
||||
where
|
||||
Self: Sized + 'a,
|
||||
I: 'a,
|
||||
E: 'a,
|
||||
O: 'a,
|
||||
O2: 'a,
|
||||
F: Fn(O) -> O2 + 'a,
|
||||
{
|
||||
crate::combinators::map(self, map_fn).boxed()
|
||||
}
|
||||
|
||||
fn to<'a, O2>(self, item: O2) -> BoxedParser<'a, I, O2, E>
|
||||
where
|
||||
Self: Sized + 'a,
|
||||
I: 'a,
|
||||
O: 'a,
|
||||
O2: Clone + 'a,
|
||||
E: 'a,
|
||||
{
|
||||
self.map(move |_| item.clone())
|
||||
}
|
||||
|
||||
fn then<'a, P, O2>(self, next_parser: P) -> BoxedParser<'a, I, (O, O2), E>
|
||||
where
|
||||
Self: Sized + 'a,
|
||||
I: 'a,
|
||||
O: 'a,
|
||||
O2: 'a,
|
||||
E: 'a,
|
||||
P: Parser<I, O2, E> + 'a,
|
||||
{
|
||||
crate::sequence::tuple2(self, next_parser).boxed()
|
||||
}
|
||||
|
||||
fn ignore_then<'a, P, O2>(self, next_parser: P) -> BoxedParser<'a, I, O2, E>
|
||||
where
|
||||
Self: Sized + 'a,
|
||||
I: 'a,
|
||||
O: 'a,
|
||||
O2: 'a,
|
||||
E: 'a,
|
||||
P: Parser<I, O2, E> + 'a,
|
||||
{
|
||||
crate::sequence::tuple2(self, next_parser).map(|(_, next_output)| next_output)
|
||||
}
|
||||
|
||||
fn then_ignore<'a, P, O2>(self, next_parser: P) -> BoxedParser<'a, I, O, E>
|
||||
where
|
||||
Self: Sized + 'a,
|
||||
I: 'a,
|
||||
O: 'a,
|
||||
O2: 'a,
|
||||
E: 'a,
|
||||
P: Parser<I, O2, E> + 'a,
|
||||
{
|
||||
crate::sequence::tuple2(self, next_parser).map(|(this_output, _)| this_output)
|
||||
}
|
||||
|
||||
fn delimited<'a, P1, O1, P2, O2>(self, left: P1, right: P2) -> BoxedParser<'a, I, O, E>
|
||||
where
|
||||
Self: Sized + 'a,
|
||||
I: 'a,
|
||||
O1: 'a,
|
||||
O2: 'a,
|
||||
O: 'a,
|
||||
E: 'a,
|
||||
P1: Parser<I, O1, E> + 'a,
|
||||
P2: Parser<I, O2, E> + 'a,
|
||||
{
|
||||
crate::sequence::seq((left, self, right)).map(|(_, output, _)| output)
|
||||
}
|
||||
|
||||
fn surrounded_by<'a, P, O1>(self, surrounding: P) -> BoxedParser<'a, I, O, E>
|
||||
where
|
||||
Self: Sized + 'a,
|
||||
I: 'a,
|
||||
O1: 'a,
|
||||
O: 'a,
|
||||
E: 'a,
|
||||
P: Parser<I, O1, E> + 'a,
|
||||
{
|
||||
BoxedParser::new(move |input| {
|
||||
let p1 = |i| surrounding.parse(i);
|
||||
let p2 = |i| surrounding.parse(i);
|
||||
let main = |i| self.parse(i);
|
||||
crate::sequence::seq((p1, main, p2))
|
||||
.map(|(_, output, _)| output)
|
||||
.parse(input)
|
||||
})
|
||||
}
|
||||
|
||||
fn optional<'a>(self) -> BoxedParser<'a, I, Option<O>, E>
|
||||
where
|
||||
I: Clone + 'a,
|
||||
O: 'a,
|
||||
E: 'a,
|
||||
Self: Sized + 'a,
|
||||
{
|
||||
crate::combinators::optional(self).boxed()
|
||||
}
|
||||
|
||||
fn named<'a>(self, parser_name: &str) -> NamedParser<'a, I, O, E>
|
||||
where
|
||||
Self: Sized + 'a,
|
||||
I: 'a,
|
||||
{
|
||||
NamedParser::new(self.boxed(), parser_name.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: ParserInput, O, E, F> Parser<I, O, E> for F
|
||||
where
|
||||
F: Fn(I) -> ParseResult<I, O, E>,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E> {
|
||||
self(input)
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
Representation::new("NOT IMPL'D")
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: ParserInput, O, E, F> Parser<I, O, E> for (F, Representation)
|
||||
where
|
||||
F: Fn(I) -> ParseResult<I, O, E>,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E> {
|
||||
self.0(input)
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
self.1.clone()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, O, E, T> Parser<I, O, E> for Rc<T>
|
||||
where
|
||||
I: ParserInput,
|
||||
T: Parser<I, O, E>,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E> {
|
||||
self.as_ref().parse(input)
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
self.as_ref().representation()
|
||||
}
|
||||
}
|
|
@ -0,0 +1,36 @@
|
|||
use super::boxed_parser::BoxedParser;
|
||||
use crate::parser::{ParseResult, Parser, ParserInput, Representation};
|
||||
|
||||
pub struct NamedParser<'a, I, O, E>
|
||||
where
|
||||
I: ParserInput,
|
||||
{
|
||||
inner_parser: BoxedParser<'a, I, O, E>,
|
||||
name: String,
|
||||
}
|
||||
|
||||
impl<'a, I, O, E> NamedParser<'a, I, O, E>
|
||||
where
|
||||
I: ParserInput,
|
||||
{
|
||||
pub(super) fn new(inner_parser: BoxedParser<'a, I, O, E>, name: String) -> Self
|
||||
where
|
||||
I: 'a,
|
||||
{
|
||||
NamedParser { inner_parser, name }
|
||||
}
|
||||
|
||||
pub fn get_name(&'a self) -> &'a str {
|
||||
self.name.as_ref()
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a, I: ParserInput, O, E> Parser<I, O, E> for NamedParser<'a, I, O, E> {
|
||||
fn representation(&self) -> Representation {
|
||||
self.inner_parser.representation()
|
||||
}
|
||||
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E> {
|
||||
self.inner_parser.parse(input)
|
||||
}
|
||||
}
|
|
@ -0,0 +1,11 @@
|
|||
pub trait ParserInput: std::fmt::Debug {
|
||||
type Output;
|
||||
fn next_token() -> Self::Output;
|
||||
}
|
||||
|
||||
impl ParserInput for &str {
|
||||
type Output = ();
|
||||
fn next_token() -> Self::Output {
|
||||
()
|
||||
}
|
||||
}
|
|
@ -0,0 +1,66 @@
|
|||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct Representation {
|
||||
val: String,
|
||||
}
|
||||
|
||||
impl Representation {
|
||||
pub fn new(from: &str) -> Self {
|
||||
Self {
|
||||
val: from.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn from_choice(
|
||||
choice_parser_reps: &mut impl Iterator<Item = Representation>,
|
||||
) -> Self {
|
||||
let mut buf = String::new();
|
||||
let mut iter = choice_parser_reps.peekable();
|
||||
loop {
|
||||
let rep = match iter.next() {
|
||||
Some(r) => r,
|
||||
None => break,
|
||||
};
|
||||
buf.push_str(&rep.val);
|
||||
match iter.peek() {
|
||||
Some(_) => {
|
||||
buf.push_str(" | ");
|
||||
}
|
||||
None => {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Representation::new(&buf)
|
||||
}
|
||||
|
||||
pub(crate) fn from_sequence(
|
||||
sequence_representations: &mut impl Iterator<Item = Representation>,
|
||||
) -> Self {
|
||||
let mut buf = String::new();
|
||||
let mut iter = sequence_representations.peekable();
|
||||
loop {
|
||||
let rep = match iter.next() {
|
||||
Some(r) => r,
|
||||
None => break,
|
||||
};
|
||||
buf.push_str(&rep.val);
|
||||
match iter.peek() {
|
||||
Some(_) => {
|
||||
buf.push_str(" ");
|
||||
}
|
||||
None => {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Representation::new(&buf)
|
||||
}
|
||||
|
||||
// TODO use at_least, at_most
|
||||
pub(crate) fn repeated(underlying: Representation, at_least: u16, _at_most: u16) -> Self {
|
||||
let sigil = if at_least == 0 { "*" } else { "+" };
|
||||
Representation::new(&format!("({}){}", underlying.val, sigil))
|
||||
}
|
||||
}
|
|
@ -0,0 +1,108 @@
|
|||
use crate::parser::{ParseResult, Parser, ParserInput, Representation};
|
||||
|
||||
pub fn literal_char(expected: char) -> impl Fn(&str) -> ParseResult<&str, char, &str> {
|
||||
move |input| match input.chars().next() {
|
||||
Some(ch) if ch == expected => Ok((expected, &input[ch.len_utf8()..])),
|
||||
_ => Err(input),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn literal<'a>(expected: &'static str) -> impl Parser<&'a str, &'a str, &'a str> {
|
||||
println!("literal call expected: {}", expected);
|
||||
let rep = Representation::new(expected);
|
||||
let p = move |input: &'a str| match input.get(0..expected.len()) {
|
||||
Some(next) if next == expected => Ok((expected, &input[expected.len()..])),
|
||||
_ => Err(input),
|
||||
};
|
||||
(p, rep)
|
||||
}
|
||||
|
||||
pub fn any_char(input: &str) -> ParseResult<&str, char, &str> {
|
||||
match input.chars().next() {
|
||||
Some(ch) => Ok((ch, &input[ch.len_utf8()..])),
|
||||
None => Err(input),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn one_of<'a>(items: &'static str) -> impl Parser<&'a str, &'a str, &'a str> {
|
||||
let p = move |input: &'a str| {
|
||||
if let Some(ch) = input.chars().next() {
|
||||
if items.contains(ch) {
|
||||
let (first, rest) = input.split_at(1);
|
||||
return Ok((first, rest));
|
||||
}
|
||||
}
|
||||
Err(input)
|
||||
};
|
||||
|
||||
let mut s = String::new();
|
||||
for ch in items.chars() {
|
||||
s.push(ch);
|
||||
s.push_str(" | ");
|
||||
}
|
||||
let rep = Representation::new(&s);
|
||||
(p, rep)
|
||||
}
|
||||
|
||||
pub fn pred<P, F, I, O>(parser: P, pred_fn: F) -> impl Parser<I, O, I>
|
||||
where
|
||||
I: ParserInput,
|
||||
P: Parser<I, O, I>,
|
||||
F: Fn(&O) -> bool,
|
||||
{
|
||||
let orig_rep = parser.representation();
|
||||
(
|
||||
move |input| {
|
||||
parser.parse(input).and_then(|(result, rest)| {
|
||||
if pred_fn(&result) {
|
||||
Ok((result, rest))
|
||||
} else {
|
||||
Err(rest)
|
||||
}
|
||||
})
|
||||
},
|
||||
Representation::new(&format!("{:?} if <PREDICATE>", orig_rep)),
|
||||
)
|
||||
}
|
||||
|
||||
/// Parses a standard identifier in a programming language
|
||||
pub fn identifier(input: &str) -> ParseResult<&str, String, &str> {
|
||||
let mut chars = input.chars();
|
||||
let mut buf = String::new();
|
||||
|
||||
match chars.next() {
|
||||
Some(ch) if ch.is_alphabetic() => buf.push(ch),
|
||||
_ => return Err(input),
|
||||
}
|
||||
|
||||
for next in chars {
|
||||
if next.is_alphanumeric() {
|
||||
buf.push(next);
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
let next_index = buf.len();
|
||||
Ok((buf, &input[next_index..]))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_identifier() {
|
||||
assert_eq!(
|
||||
identifier("bongo1beans").unwrap(),
|
||||
(("bongo1beans".to_string(), ""))
|
||||
);
|
||||
assert_eq!(identifier("2bongo1beans"), Err("2bongo1beans"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_pred() {
|
||||
let p = pred(any_char, |c| *c == 'f');
|
||||
assert_eq!(p.parse("frog"), Ok(('f', "rog")));
|
||||
}
|
||||
}
|
|
@ -0,0 +1,195 @@
|
|||
use crate::parser::{ParseResult, Parser, ParserInput, Representation};
|
||||
|
||||
pub fn tuple2<P1, P2, I, O1, O2, E>(parser1: P1, parser2: P2) -> impl Parser<I, (O1, O2), E>
|
||||
where
|
||||
I: ParserInput,
|
||||
P1: Parser<I, O1, E>,
|
||||
P2: Parser<I, O2, E>,
|
||||
{
|
||||
seq((parser1, parser2))
|
||||
}
|
||||
|
||||
pub fn seq<T, I, O, E>(sequence: T) -> impl Parser<I, O, E>
|
||||
where
|
||||
I: ParserInput,
|
||||
T: Sequence<I, O, E>,
|
||||
{
|
||||
let rep = sequence.representation();
|
||||
let p = move |input| sequence.parse(input);
|
||||
(p, rep)
|
||||
}
|
||||
|
||||
/* TODO - eventually rewrite this parser combinator in Schala. Seeing what this
|
||||
* code that makes heavy use of type variables and abstraction over types looks like
|
||||
* in Schala's type system should be educational
|
||||
*/
|
||||
|
||||
pub trait Sequence<I, O, E> {
|
||||
fn parse(&self, input: I) -> ParseResult<I, O, E>;
|
||||
fn representation(&self) -> Representation;
|
||||
}
|
||||
|
||||
impl<I, O1, O2, E, P1, P2> Sequence<I, (O1, O2), E> for (P1, P2)
|
||||
where
|
||||
I: ParserInput,
|
||||
P1: Parser<I, O1, E>,
|
||||
P2: Parser<I, O2, E>,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, (O1, O2), E> {
|
||||
let parser1 = &self.0;
|
||||
let parser2 = &self.1;
|
||||
parser1.parse(input).and_then(|(result1, rest1)| {
|
||||
parser2
|
||||
.parse(rest1)
|
||||
.map(|(result2, rest2)| ((result1, result2), rest2))
|
||||
})
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
let mut iter = [self.0.representation(), self.1.representation()].into_iter();
|
||||
Representation::from_sequence(&mut iter)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, O1, O2, O3, E, P1, P2, P3> Sequence<I, (O1, O2, O3), E> for (P1, P2, P3)
|
||||
where
|
||||
I: ParserInput,
|
||||
P1: Parser<I, O1, E>,
|
||||
P2: Parser<I, O2, E>,
|
||||
P3: Parser<I, O3, E>,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, (O1, O2, O3), E> {
|
||||
let parser1 = &self.0;
|
||||
let parser2 = &self.1;
|
||||
let parser3 = &self.2;
|
||||
|
||||
let (result1, rest1) = parser1.parse(input)?;
|
||||
let (result2, rest2) = parser2.parse(rest1)?;
|
||||
let (result3, rest3) = parser3.parse(rest2)?;
|
||||
|
||||
Ok(((result1, result2, result3), rest3))
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
let mut iter = [
|
||||
self.0.representation(),
|
||||
self.1.representation(),
|
||||
self.2.representation(),
|
||||
]
|
||||
.into_iter();
|
||||
Representation::from_sequence(&mut iter)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, O1, O2, O3, O4, E, P1, P2, P3, P4> Sequence<I, (O1, O2, O3, O4), E> for (P1, P2, P3, P4)
|
||||
where
|
||||
I: ParserInput,
|
||||
P1: Parser<I, O1, E>,
|
||||
P2: Parser<I, O2, E>,
|
||||
P3: Parser<I, O3, E>,
|
||||
P4: Parser<I, O4, E>,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, (O1, O2, O3, O4), E> {
|
||||
let parser1 = &self.0;
|
||||
let parser2 = &self.1;
|
||||
let parser3 = &self.2;
|
||||
let parser4 = &self.3;
|
||||
|
||||
let (result1, rest1) = parser1.parse(input)?;
|
||||
let (result2, rest2) = parser2.parse(rest1)?;
|
||||
let (result3, rest3) = parser3.parse(rest2)?;
|
||||
let (result4, rest4) = parser4.parse(rest3)?;
|
||||
|
||||
Ok(((result1, result2, result3, result4), rest4))
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
let mut iter = [
|
||||
self.0.representation(),
|
||||
self.1.representation(),
|
||||
self.2.representation(),
|
||||
self.3.representation(),
|
||||
]
|
||||
.into_iter();
|
||||
Representation::from_sequence(&mut iter)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, O1, O2, O3, O4, O5, E, P1, P2, P3, P4, P5> Sequence<I, (O1, O2, O3, O4, O5), E>
|
||||
for (P1, P2, P3, P4, P5)
|
||||
where
|
||||
I: ParserInput,
|
||||
P1: Parser<I, O1, E>,
|
||||
P2: Parser<I, O2, E>,
|
||||
P3: Parser<I, O3, E>,
|
||||
P4: Parser<I, O4, E>,
|
||||
P5: Parser<I, O5, E>,
|
||||
{
|
||||
fn parse(&self, input: I) -> ParseResult<I, (O1, O2, O3, O4, O5), E> {
|
||||
let parser1 = &self.0;
|
||||
let parser2 = &self.1;
|
||||
let parser3 = &self.2;
|
||||
let parser4 = &self.3;
|
||||
let parser5 = &self.4;
|
||||
|
||||
let (result1, rest1) = parser1.parse(input)?;
|
||||
let (result2, rest2) = parser2.parse(rest1)?;
|
||||
let (result3, rest3) = parser3.parse(rest2)?;
|
||||
let (result4, rest4) = parser4.parse(rest3)?;
|
||||
let (result5, rest5) = parser5.parse(rest4)?;
|
||||
|
||||
Ok(((result1, result2, result3, result4, result5), rest5))
|
||||
}
|
||||
|
||||
fn representation(&self) -> Representation {
|
||||
let mut iter = [
|
||||
self.0.representation(),
|
||||
self.1.representation(),
|
||||
self.2.representation(),
|
||||
self.3.representation(),
|
||||
self.4.representation(),
|
||||
]
|
||||
.into_iter();
|
||||
Representation::from_sequence(&mut iter)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod test {
|
||||
use super::*;
|
||||
use crate::combinators::repeated;
|
||||
use crate::primitives::{identifier, literal};
|
||||
|
||||
#[test]
|
||||
fn test_tuple2() {
|
||||
let p = tuple2(identifier, tuple2(literal(" "), literal("ruts")));
|
||||
let (output, _rest) = p.parse("fort1 ruts").unwrap();
|
||||
assert_eq!(output, ("fort1".into(), (" ", "ruts")));
|
||||
|
||||
let p = identifier.then(literal(" ")).then(literal("ruts"));
|
||||
let (output, _rest) = p.parse("fort1 ruts").unwrap();
|
||||
assert_eq!(output, (("fort1".into(), " "), "ruts"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_seq() {
|
||||
let p = seq((
|
||||
literal("bong").to(10),
|
||||
repeated(literal(" ")).to(()),
|
||||
literal("hits").to(20),
|
||||
));
|
||||
assert_eq!(p.parse("bong hits").unwrap(), ((10, (), 20), ""));
|
||||
|
||||
let p = seq((
|
||||
literal("alpha").to(10),
|
||||
repeated(literal(" ")).to(()),
|
||||
repeated(literal("-")).to(()),
|
||||
repeated(literal(" ")),
|
||||
literal("beta"),
|
||||
));
|
||||
assert_eq!(
|
||||
p.parse("alpha ------ beta gamma").unwrap(),
|
||||
((10, (), (), vec![" ", " ", " "], "beta"), " gamma")
|
||||
);
|
||||
}
|
||||
}
|
|
@ -0,0 +1,49 @@
|
|||
{
|
||||
"$schema": "https://joplinapp.org/schema/settings.json",
|
||||
"locale": "en_GB",
|
||||
"sync.target": 6,
|
||||
"markdown.plugin.softbreaks": false,
|
||||
"markdown.plugin.typographer": false,
|
||||
"spellChecker.language": "en-US",
|
||||
"ui.layout": {
|
||||
"key": "root",
|
||||
"children": [
|
||||
{
|
||||
"key": "sideBar",
|
||||
"width": 250,
|
||||
"visible": true
|
||||
},
|
||||
{
|
||||
"key": "noteList",
|
||||
"width": 250,
|
||||
"visible": true
|
||||
},
|
||||
{
|
||||
"key": "editor",
|
||||
"visible": true,
|
||||
"width": 1493
|
||||
},
|
||||
{
|
||||
"key": "plugin-view-joplin.plugin.note.tabs-note.tabs.panel",
|
||||
"context": {
|
||||
"pluginId": "joplin.plugin.note.tabs"
|
||||
},
|
||||
"visible": true
|
||||
}
|
||||
],
|
||||
"visible": true
|
||||
},
|
||||
"noteVisiblePanes": [
|
||||
"editor",
|
||||
"viewer"
|
||||
],
|
||||
"theme": 4,
|
||||
"sync.6.username": "webdav",
|
||||
"net.ignoreTlsErrors": true,
|
||||
"style.editor.contentMaxWidth": 600,
|
||||
"editor.codeView": true,
|
||||
"markdown.plugin.sub": true,
|
||||
"markdown.plugin.sup": true,
|
||||
"markdown.plugin.multitable": true
|
||||
}
|
||||
|
|
@ -0,0 +1,248 @@
|
|||
use parser_combinator::choice::choice;
|
||||
use parser_combinator::combinators::repeated;
|
||||
use parser_combinator::primitives::{any_char, literal, literal_char, one_of, pred};
|
||||
use parser_combinator::sequence::seq;
|
||||
use parser_combinator::Parser;
|
||||
use parser_combinator::Representation;
|
||||
|
||||
use proptest::prelude::*;
|
||||
|
||||
use rstest::*;
|
||||
|
||||
proptest! {
|
||||
#[test]
|
||||
fn doesnt_crash(s in "\\PC*") {
|
||||
let _output = json_object().parse(&s);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_string(s in r#"[^"]+"#) {
|
||||
let input = format!("\"{}\"", s);
|
||||
let output = json_string().parse(&input).unwrap();
|
||||
match output {
|
||||
(JsonValue::Str(output_s), "") if output_s == s => (),
|
||||
_ => panic!(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_parsing() {
|
||||
let output = literal("a").parse("a yolo");
|
||||
assert_eq!(output.unwrap(), ("a", " yolo"));
|
||||
}
|
||||
|
||||
/*
|
||||
* JSON BNF
|
||||
* <JSON> ::= <value>
|
||||
<value> ::= <object> | <array> | <boolean> | <string> | <number> | <null>
|
||||
<array> ::= "[" [<value>] {"," <value>}* "]"
|
||||
<object> ::= "{" [<property>] {"," <property>}* "}"
|
||||
<property> ::= <string> ":" <value>
|
||||
*/
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
enum JsonValue {
|
||||
Null,
|
||||
Bool(bool),
|
||||
Str(String),
|
||||
Num(f64),
|
||||
Array(Vec<JsonValue>),
|
||||
Object(Vec<(String, JsonValue)>),
|
||||
}
|
||||
|
||||
trait JsonParser<'a, T>: Parser<&'a str, T, &'a str> {}
|
||||
impl<'a, T, P> JsonParser<'a, T> for P where P: Parser<&'a str, T, &'a str> {}
|
||||
|
||||
fn json_null<'a>() -> impl JsonParser<'a, JsonValue> {
|
||||
literal("null").to(JsonValue::Null)
|
||||
}
|
||||
|
||||
fn json_bool<'a>() -> impl JsonParser<'a, JsonValue> {
|
||||
choice((
|
||||
literal("true").to(JsonValue::Bool(true)),
|
||||
literal("false").to(JsonValue::Bool(false)),
|
||||
))
|
||||
}
|
||||
|
||||
fn json_number<'a>() -> impl JsonParser<'a, JsonValue> {
|
||||
fn digit<'a>() -> impl JsonParser<'a, &'a str> {
|
||||
one_of("1234567890")
|
||||
}
|
||||
|
||||
fn digits<'a>() -> impl JsonParser<'a, Vec<&'a str>> {
|
||||
repeated(digit()).at_least(1)
|
||||
}
|
||||
|
||||
let json_number_inner = choice((
|
||||
seq((digits(), literal(".").ignore_then(digits()).optional())).map(
|
||||
|(mut digits, maybe_decimal)| {
|
||||
if let Some(decimal_digits) = maybe_decimal {
|
||||
digits.push(".");
|
||||
digits.extend(decimal_digits.into_iter());
|
||||
}
|
||||
digits.into_iter().collect::<String>()
|
||||
},
|
||||
),
|
||||
literal(".").ignore_then(digits()).map(|decimal_digits| {
|
||||
let mut d = vec!["."];
|
||||
d.extend(decimal_digits.into_iter());
|
||||
d.into_iter().collect::<String>()
|
||||
}),
|
||||
))
|
||||
.map(|digits| digits.parse::<f64>().unwrap());
|
||||
|
||||
literal("-")
|
||||
.optional()
|
||||
.then(json_number_inner)
|
||||
.map(|(maybe_sign, mut val)| {
|
||||
if maybe_sign.is_some() {
|
||||
val *= -1.0;
|
||||
}
|
||||
JsonValue::Num(val)
|
||||
})
|
||||
}
|
||||
|
||||
fn json_string_raw<'a>() -> impl JsonParser<'a, String> {
|
||||
seq((
|
||||
literal_char('"'),
|
||||
repeated(pred(any_char, |ch| *ch != '"')),
|
||||
literal_char('"'),
|
||||
))
|
||||
.map(|(_, s, _)| s.iter().cloned().collect::<String>())
|
||||
}
|
||||
|
||||
fn json_string<'a>() -> impl JsonParser<'a, JsonValue> {
|
||||
json_string_raw().map(JsonValue::Str)
|
||||
}
|
||||
|
||||
fn whitespace<'a>() -> impl JsonParser<'a, ()> {
|
||||
repeated(choice((
|
||||
literal_char('\t'),
|
||||
literal_char('\n'),
|
||||
literal_char(' '),
|
||||
)))
|
||||
.to(())
|
||||
}
|
||||
|
||||
fn json_array<'a>() -> impl JsonParser<'a, JsonValue> {
|
||||
move |input| {
|
||||
let val = json_value().surrounded_by(whitespace());
|
||||
|
||||
repeated(val)
|
||||
.separated_by(literal(","), false)
|
||||
.delimited(literal_char('['), literal_char(']'))
|
||||
.map(JsonValue::Array)
|
||||
.parse(input)
|
||||
}
|
||||
}
|
||||
|
||||
fn json_object<'a>() -> impl JsonParser<'a, JsonValue> {
|
||||
move |input| {
|
||||
let kv = json_string_raw()
|
||||
.surrounded_by(whitespace())
|
||||
.then_ignore(literal_char(':'))
|
||||
.then(json_value().surrounded_by(whitespace()));
|
||||
|
||||
repeated(kv)
|
||||
.separated_by(literal_char(','), false)
|
||||
.delimited(literal_char('{'), literal_char('}'))
|
||||
.map(JsonValue::Object)
|
||||
.parse(input)
|
||||
}
|
||||
}
|
||||
|
||||
fn json_value<'a>() -> impl JsonParser<'a, JsonValue> {
|
||||
choice((
|
||||
json_null(),
|
||||
json_bool(),
|
||||
json_number(),
|
||||
json_string(),
|
||||
json_array(),
|
||||
json_object(),
|
||||
))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_json_primitives() {
|
||||
assert_eq!(
|
||||
json_string().parse(r#""yolo swagg""#).unwrap(),
|
||||
(JsonValue::Str("yolo swagg".into()), "")
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
json_number().parse("-383").unwrap().0,
|
||||
JsonValue::Num(-383f64)
|
||||
);
|
||||
assert_eq!(
|
||||
json_number().parse("-.383").unwrap().0,
|
||||
JsonValue::Num(-0.383)
|
||||
);
|
||||
assert_eq!(
|
||||
json_number().parse(".383").unwrap().0,
|
||||
JsonValue::Num(0.383)
|
||||
);
|
||||
assert_eq!(
|
||||
json_number().parse("-1.383").unwrap().0,
|
||||
JsonValue::Num(-1.383)
|
||||
);
|
||||
}
|
||||
|
||||
#[rstest]
|
||||
#[case(r#"[ 4, 9, "ara",]"#)]
|
||||
fn parse_json_array_err(#[case] input: &str) {
|
||||
assert!(json_array().parse(input).is_err());
|
||||
}
|
||||
|
||||
#[rstest]
|
||||
#[case("[[],[]]", (JsonValue::Array(vec![JsonValue::Array(vec![]), JsonValue::Array(vec![])]), ""))]
|
||||
#[case(r#"[ 4, 9, "foo" ]"#, (
|
||||
JsonValue::Array(vec![
|
||||
JsonValue::Num(4.),
|
||||
JsonValue::Num(9.0),
|
||||
JsonValue::Str("foo".to_string())
|
||||
]),
|
||||
""
|
||||
))]
|
||||
#[case(r#"[8,null,[],5],{}"#,
|
||||
(
|
||||
JsonValue::Array(vec![
|
||||
JsonValue::Num(8.),
|
||||
JsonValue::Null,
|
||||
JsonValue::Array(vec![]),
|
||||
JsonValue::Num(5.),
|
||||
]),
|
||||
",{}"
|
||||
))]
|
||||
fn parse_json_array(#[case] input: &str, #[case] expected: (JsonValue, &str)) {
|
||||
assert_eq!(json_array().parse(input).unwrap(), expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_json_object() {
|
||||
assert_eq!(
|
||||
json_object().parse(r#"{ "a": 23}"#).unwrap().0,
|
||||
JsonValue::Object(vec![("a".into(), JsonValue::Num(23.))])
|
||||
);
|
||||
assert_eq!(
|
||||
json_object().parse(r#"{}"#).unwrap().0,
|
||||
JsonValue::Object(vec![])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_json_document() {
|
||||
let test_json = include_str!("joplin-cfg.json");
|
||||
let parsed_json = json_object().parse(test_json);
|
||||
assert!(parsed_json.is_ok());
|
||||
}
|
||||
|
||||
#[rstest]
|
||||
#[case(json_null().representation(), Representation::new("null"))]
|
||||
#[case(json_bool().representation(), Representation::new("true | false"))]
|
||||
#[case(json_number().representation(), Representation::new("- | ε (1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 0 | )+ . (1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 0 | )+ | ε | . (1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 0 | )+"))]
|
||||
fn representations_test(
|
||||
#[case] parser_representation: Representation,
|
||||
#[case] expected: Representation,
|
||||
) {
|
||||
assert_eq!(parser_representation, expected);
|
||||
}
|
Loading…
Reference in New Issue