greg/schala
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
523 Commits 33 Branches 0 Tags 4.4 MiB
3a97401f61
Commit Graph

492 Commits

Author SHA1 Message Date
greg
f53c14535b Made error! macro more programtic 
TODO implement Display on Token so we're not just displaying the debug
name of the token enum variants
 2017-09-19 22:10:55 -07:00
greg
4f96abd7d9 Changes to make the code more concise 2017-09-19 22:10:55 -07:00
greg
fdaf4c302c Fix all compiler warnings 2017-09-19 22:10:55 -07:00
greg
8ce53d7c72 Fix bind error 2017-09-19 22:10:55 -07:00
greg
428d560e2a Add tests for call expr parsing 2017-09-19 22:10:55 -07:00
greg
80bc7ec089 Proper call expression parsing 2017-09-19 22:10:55 -07:00
greg
e6591b80d9 Add paren test 2017-09-19 22:10:55 -07:00
greg
e099f713ad Add binop parsing test 2017-09-19 22:10:55 -07:00
greg
9b54256521 Import types for breivty 
and rename function to be explicit
 2017-09-19 22:10:55 -07:00
greg
087402ece6 Add more tests 
Need to use box patterns
 2017-09-19 22:10:55 -07:00
greg
252b6e8bd9 Okay, this strategy makes the test work 2017-09-19 22:10:55 -07:00
greg
b1163e2ae4 Operator-precedence parsing + tests 
The tests are crippled now, because it's hard to write a test macro that
can also match on Strings
 2017-09-19 22:10:55 -07:00
greg
032d01c9f5 Fix tokenization bug 2017-09-19 22:10:55 -07:00
greg
c4ab1ed105 Fix tokenizer tests 2017-09-19 22:10:55 -07:00
greg
9a257f08d7 Introduce Op type 
For operator parsing
 2017-09-19 22:10:55 -07:00
greg
47d56a7b44 fix operator parsing 2017-09-19 22:10:55 -07:00
greg
1f7ae2e30f Paren expression 2017-09-19 22:10:55 -07:00
greg
e3c8753a4d Expression parsing 2017-09-19 22:10:55 -07:00
greg
e1aa7ecb17 Finish tokenizing Op separately 2017-09-19 22:10:55 -07:00
greg
f09a6e14ba Tokenizer work to support operators 
work in progress but committing to transfer
 2017-09-19 22:10:55 -07:00
greg
31da25a66e Expression parsing work 2017-09-19 22:10:55 -07:00
greg
fff9cb7d25 Fix functin parsing 2017-09-19 22:10:55 -07:00
greg
db1e188fdb Move grammar to top of file 2017-09-19 22:10:55 -07:00
greg
935185ed92 more parsing 2017-09-19 22:10:55 -07:00
greg
0999cbe28e More parsing work 2017-09-19 22:10:55 -07:00
greg
674f70a428 Convert parsing to method-based 2017-09-19 22:10:54 -07:00
greg
b1b6672399 Implement function parsing 
With a lot of dummy code, especially around expression parsing
 2017-09-19 22:10:54 -07:00
greg
bd1c455dc8 Basic infrastructure parses 
Also got rid of EOF, don't need it
 2017-09-19 22:10:54 -07:00
greg
b62ef43f07 Add basic BNF grammar 2017-09-19 22:10:54 -07:00
greg
09b67dc3f7 Change error message 2017-09-19 22:10:54 -07:00
greg
a613fa73e5 Basic parsing framework 2017-09-19 22:10:54 -07:00
greg
8c473c554e Fix bug 2017-09-19 22:10:54 -07:00
greg
3e04cbfa29 Add comma tokenization 2017-09-19 22:10:54 -07:00
greg
570650cbfa Finish keyword tokenization 2017-09-19 22:10:54 -07:00
greg
49be163181 Add test to ignore 
For better handing of user-defined operators, which I will do in the
future
 2017-09-19 22:10:54 -07:00
greg
b4f93acbd8 Couple more tests 2017-09-19 22:10:54 -07:00
greg
8c65ae3214 Macro-ize token tests 2017-09-19 22:10:54 -07:00
greg
e436533638 Passing test 2017-09-19 22:10:54 -07:00
greg
4601a56867 Start working on tokenization tests 2017-09-19 22:10:54 -07:00
greg
2f7a1850db Finish tokenizing 2017-09-19 22:10:54 -07:00
greg
71aef379d3 Tokenize number literals 
TODO: expand this bit of code to handle 0x12, etc. syntax
 2017-09-19 22:10:54 -07:00
greg
8662a3ba0e Make tokenize error-able 2017-09-19 22:10:54 -07:00
greg
5ca98c7d77 Print tokeniziation 2017-09-19 22:10:54 -07:00
greg
13cde3106c Start making tokenizer changes 
Hopefully this time iron out all the bugs from the last implementation
 2017-09-19 22:10:54 -07:00
greg
09d524c74a Changing how parsing works again 
Someone wrote a port of the LLVM kaleidoscope tutorial to rust, namely
https://github.com/jauhien/iron-kaleidoscope

I'm just gonna follow this along
 2017-09-19 22:10:54 -07:00
greg
61c36c4def Fix assign parsing 
= is a keyword not an identifier
 2017-09-19 22:10:54 -07:00
greg
bc4fbe4276 Start implementing definition 
WIP, doesn't work
 2017-09-19 22:10:54 -07:00
greg
fc11ee753d Add block parsing 
Right now evaluating a block reduces it to just the last AST in it, will
fix later with environments
 2017-09-19 22:10:54 -07:00
greg
dd2b4893a4 Get rid of Separator token 
Have separate newline and semicolon tokens
 2017-09-19 22:10:54 -07:00
greg
51745fd800 Make convenience macro for parse errors 2017-09-19 22:10:54 -07:00
greg
8c0ac19fa8 Add full test 
Test evaluate, tokenize, parser all at once
 2017-09-19 22:10:54 -07:00
greg
971ab9ba21 Implement paren expressions 2017-09-19 22:10:54 -07:00
greg
819fb3f58f Basic evaluator functionality 
Interpreter now works for simple arithmetic expressions
 2017-09-19 22:10:54 -07:00
greg
1c23329656 Add boilerplate for evaluation 
Just wires everything up, doesn't actually evaluate yet
 2017-09-19 22:10:54 -07:00
greg
6da20cbfaf Address compiler warnings 2017-09-19 22:10:54 -07:00
greg
f48451125e Parsing arithmetic expressions works 
At the expense of an unnecessary move in lookahead()
 2017-09-19 22:10:54 -07:00
greg
06c3999430 Add expect_identifier function 
For utility
 2017-09-19 22:10:54 -07:00
greg
1af1589550 Make Parser class internally use IntoIterator 
And wrap the next() and peek() methods
 2017-09-19 22:10:54 -07:00
greg
32a90b8103 write expect 
and also make the ParseResult type more general so it can be used for
more things.
 2017-09-19 22:10:54 -07:00
greg
186c900920 Implement expect macro 
Seems like there's no way around passing in self manually
 2017-09-19 22:10:54 -07:00
greg
509ab80b9c I can now parse one thing 2017-09-19 22:10:54 -07:00
greg
247638c4db Get parsing REPL working 2017-09-19 22:10:54 -07:00
greg
be98f8387e Add another test 2017-09-19 22:10:54 -07:00
greg
841b38d5b1 Add tokenization test 2017-09-19 22:10:53 -07:00
greg
16dfbb27d5 Use state features from simplerepl 2017-09-19 22:10:53 -07:00
greg
3af7e6a409 Refactoring Schala 
Gonna work on Schala in earnest now! Using the simplerepl crate instead
of a build-in REPL, temporarily dropping parsing and evaluation code.
 2017-09-19 22:10:53 -07:00
greg
123f388711 Rename language to "Schala" 2017-09-19 22:10:53 -07:00
greg
bb349cda5f Fix newline tokenizing bug 
Still need to fix <statements> parsing because of the final newline
 2017-09-19 22:10:53 -07:00
greg
caa331ecdc Read from file as well as repl 2017-09-19 22:10:53 -07:00
greg
ae3a030ad8 Use iterative semantics for reduction 
Becuase if you try to make reduce() recursive you blow out the stack.

Incidentally, "let a = 0; while a < 999999; let a = a + 1 end" is a neat
thing to try at this stage of the game
 2017-09-19 22:10:53 -07:00
greg
ddb09b453d Implement booleans 2017-09-19 22:10:53 -07:00
greg
9a4760d44f Implement while-reduction 2017-09-19 22:10:53 -07:00
greg
eb5ce2ef9e Don't print newline for empty REPL result 2017-09-19 22:10:53 -07:00
greg
b080ea7c81 Add if-statement evaluation 2017-09-19 22:10:53 -07:00
greg
dffab8ae94 Fixed arg list parsing 2017-09-19 22:10:53 -07:00
greg
48ee6c9a75 Add function name to parse 2017-09-19 22:10:53 -07:00
greg
58d399dace More function parsing 2017-09-19 22:10:53 -07:00
greg
57ea1bae30 Preliminary addition of function blocks 2017-09-19 22:10:53 -07:00
greg
e870d8172a Make = a keyword 2017-09-19 22:10:53 -07:00
greg
72b26755a7 Make ParseResult just a normal Result type 
No reason for it to be different
 2017-09-19 22:10:53 -07:00
greg
1416c9d444 Parse null as simple_expression 2017-09-19 22:10:53 -07:00
greg
5c79563e53 Reduce binop args before reducing full expr 2017-09-19 22:10:53 -07:00
greg
c27c900e7f Add operators % and ** 
semantics for these are hard with floats handle later
 2017-09-19 22:10:53 -07:00
greg
50c5dbe96d Evaluate simple binops 
Woo this is kind of a neat milestone! I can now compute with this
language!
 2017-09-19 22:10:53 -07:00
greg
be8c8b3343 Add paren test 2017-09-19 22:10:53 -07:00
greg
b856023072 Fixed paren parsing 
also made error reporting a bit nicer
 2017-09-19 22:10:53 -07:00
greg
868373f409 use match syntax in simple_parse 2017-09-19 22:10:53 -07:00
greg
582a7fd6dc Make parse and tokens optional 2017-09-19 22:10:53 -07:00
greg
a947ec3cb2 Add simple parsing test 2017-09-19 22:10:53 -07:00
greg
79619025ea Add directive to print precedence chart 2017-09-19 22:10:53 -07:00
greg
56b338a6a8 Move to global precedence table 2017-09-19 22:10:53 -07:00
greg
7d6f946e22 Fill out a few more precedences 2017-09-19 22:10:53 -07:00
greg
0da7f7e3a1 Fully fixed binop parsing 2017-09-19 22:10:53 -07:00
greg
19a344fa77 Fixed precedent-less binop parsing 2017-09-19 22:10:53 -07:00
greg
626a7f3861 Working on binop parsing 2017-09-19 22:10:53 -07:00
greg
8e3a571d67 .env dirctive to display environment 2017-09-19 22:10:53 -07:00
greg
f88f115567 Environment persistent across repl loop 2017-09-19 22:10:53 -07:00
greg
08f1092b69 Variable lookup works 
Note this introduces a panic - if the AST node inserted into the
environment is not reduced, it throws an error when trying to look it up
 2017-09-19 22:10:53 -07:00
greg
6ddea790c0 Beginning of variable lookup 
everything is null
 2017-09-19 22:10:53 -07:00
greg
6897eb1283 Implemented variable binding 2017-09-19 22:10:53 -07:00
greg
34fdf2be00 Add machinery for evaluation environments 2017-09-19 22:10:53 -07:00
greg
4ef93fafb5 Cause tokenize error for unclosed strings 2017-09-19 22:10:53 -07:00
greg
d2108f0f97 First pass at evaluation 
Very incomplete
 2017-09-19 22:10:52 -07:00
greg
2989ac338c Implemented binop parsing 
Uses Operator-precedence parsing algorithm, which I don't fully
understand.
 2017-09-19 22:10:52 -07:00
greg
8b6d54aec2 Fix let clause parsing 
let a = x, x should be expression not just simple expression
 2017-09-19 22:10:52 -07:00
greg
cdb47bb3b9 Add paren parsing 2017-09-19 22:10:52 -07:00
greg
9d6dc5a5f2 Tokenize periods separately 2017-09-19 22:10:52 -07:00
greg
8f9bfbc5bd Rename rhs to simple_expression 2017-09-19 22:10:52 -07:00
greg
c9fdd5e83c Simplified statements-parsing 
Still a little wonky wrt extraneous Separators, need to adjust grammar
to fix I think
 2017-09-19 22:10:52 -07:00
greg
30eddf7737 while statements 2017-09-19 22:10:52 -07:00
greg
42719dc2f2 Change 'input' to 'tokens' 
just to be consistent
 2017-09-19 22:10:52 -07:00
greg
8fcc850d77 Added else clause to if parsing 2017-09-19 22:10:52 -07:00
greg
f421918945 basic if expression 2017-09-19 22:10:52 -07:00
greg
0e4469fa58 Filled out keyword tokenizing 2017-09-19 22:10:52 -07:00
greg
edf100b583 Starting to do if statement parsing 2017-09-19 22:10:52 -07:00
greg
169e662049 Collapse Separator tokens 
only ever gonna be one in a row
 2017-09-19 22:10:52 -07:00
greg
46999beabf Added skeleton of expression() parser 2017-09-19 22:10:52 -07:00
greg
1342a76786 Added support for interpreter directives 2017-09-19 22:10:52 -07:00
greg
05238bced3 rhs production 2017-09-19 22:10:52 -07:00
greg
a97cce184c Empty program is valid too 2017-09-19 22:10:52 -07:00
greg
329c521964 Parsing statement blocks works 2017-09-19 22:10:52 -07:00
greg
bfa16fd6fb Added Keyword lexical class 2017-09-19 22:10:52 -07:00
greg
25f5188d8c Move definition around 2017-09-19 22:10:52 -07:00
greg
5213dd327f Change type to peekable 2017-09-19 22:10:52 -07:00
greg
cea29094cd type alias for Tokens 2017-09-19 22:10:52 -07:00
greg
67eafba97a Put expect into early return macro 2017-09-19 22:10:52 -07:00
greg
1059a88ee6 Separate parsing into module 2017-09-19 22:10:52 -07:00
greg
429ace73bd Move tokenizing into separate module 2017-09-19 22:10:52 -07:00
greg
044e7a6a26 Rename ASTNode -> AST 
saves typing
 2017-09-19 22:10:52 -07:00
greg
dbdae42c1b Add string to AST 2017-09-19 22:10:52 -07:00
greg
fc3dcf792d Start writing recursive descent parser 
I think I get the idea now
 2017-09-19 22:10:52 -07:00
greg
02b34ca105 Wrote expect() 
Hopefully correctly?
 2017-09-19 22:10:52 -07:00
greg
9f4330889a Starting parsing work 2017-09-19 22:10:52 -07:00
greg
3058af4f05 Break on ctrl-D 2017-09-19 22:10:52 -07:00
greg
4f17d5a0dc Add number tokenizing 2017-09-19 22:10:52 -07:00
greg
8e3774ffca Comma as separate token 2017-09-19 22:10:52 -07:00
greg
c6059ada7d Separators and parens 
Separator = ; or \n, they are equivalent
 2017-09-19 22:10:52 -07:00
greg
2aaa600d53 More tokenizer stuff 2017-09-19 22:10:51 -07:00
greg
c6a92728ee Scaffolding for evaluation function 2017-09-19 22:10:51 -07:00
greg
b2e23bed86 Print tokens and parse 2017-09-19 22:10:51 -07:00
greg
3fdacf018e Basic repl 2017-09-19 22:10:51 -07:00
greg
6098bf84d6 Initial commit 2017-09-19 22:10:51 -07:00