greg/schala
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
718 Commits 33 Branches 0 Tags 4.4 MiB
d9e96398a4
Commit Graph

675 Commits

Author SHA1 Message Date
greg
e3c8753a4d Expression parsing 2017-09-19 22:10:55 -07:00
greg
e1aa7ecb17 Finish tokenizing Op separately 2017-09-19 22:10:55 -07:00
greg
f09a6e14ba Tokenizer work to support operators 
work in progress but committing to transfer
 2017-09-19 22:10:55 -07:00
greg
31da25a66e Expression parsing work 2017-09-19 22:10:55 -07:00
greg
fff9cb7d25 Fix functin parsing 2017-09-19 22:10:55 -07:00
greg
db1e188fdb Move grammar to top of file 2017-09-19 22:10:55 -07:00
greg
935185ed92 more parsing 2017-09-19 22:10:55 -07:00
greg
0999cbe28e More parsing work 2017-09-19 22:10:55 -07:00
greg
674f70a428 Convert parsing to method-based 2017-09-19 22:10:54 -07:00
greg
b1b6672399 Implement function parsing 
With a lot of dummy code, especially around expression parsing
 2017-09-19 22:10:54 -07:00
greg
bd1c455dc8 Basic infrastructure parses 
Also got rid of EOF, don't need it
 2017-09-19 22:10:54 -07:00
greg
b62ef43f07 Add basic BNF grammar 2017-09-19 22:10:54 -07:00
greg
09b67dc3f7 Change error message 2017-09-19 22:10:54 -07:00
greg
a613fa73e5 Basic parsing framework 2017-09-19 22:10:54 -07:00
greg
8c473c554e Fix bug 2017-09-19 22:10:54 -07:00
greg
3e04cbfa29 Add comma tokenization 2017-09-19 22:10:54 -07:00
greg
570650cbfa Finish keyword tokenization 2017-09-19 22:10:54 -07:00
greg
49be163181 Add test to ignore 
For better handing of user-defined operators, which I will do in the
future
 2017-09-19 22:10:54 -07:00
greg
b4f93acbd8 Couple more tests 2017-09-19 22:10:54 -07:00
greg
8c65ae3214 Macro-ize token tests 2017-09-19 22:10:54 -07:00
greg
e436533638 Passing test 2017-09-19 22:10:54 -07:00
greg
4601a56867 Start working on tokenization tests 2017-09-19 22:10:54 -07:00
greg
2f7a1850db Finish tokenizing 2017-09-19 22:10:54 -07:00
greg
71aef379d3 Tokenize number literals 
TODO: expand this bit of code to handle 0x12, etc. syntax
 2017-09-19 22:10:54 -07:00
greg
8662a3ba0e Make tokenize error-able 2017-09-19 22:10:54 -07:00
greg
5ca98c7d77 Print tokeniziation 2017-09-19 22:10:54 -07:00
greg
13cde3106c Start making tokenizer changes 
Hopefully this time iron out all the bugs from the last implementation
 2017-09-19 22:10:54 -07:00
greg
09d524c74a Changing how parsing works again 
Someone wrote a port of the LLVM kaleidoscope tutorial to rust, namely
https://github.com/jauhien/iron-kaleidoscope

I'm just gonna follow this along
 2017-09-19 22:10:54 -07:00
greg
61c36c4def Fix assign parsing 
= is a keyword not an identifier
 2017-09-19 22:10:54 -07:00
greg
bc4fbe4276 Start implementing definition 
WIP, doesn't work
 2017-09-19 22:10:54 -07:00
greg
fc11ee753d Add block parsing 
Right now evaluating a block reduces it to just the last AST in it, will
fix later with environments
 2017-09-19 22:10:54 -07:00
greg
dd2b4893a4 Get rid of Separator token 
Have separate newline and semicolon tokens
 2017-09-19 22:10:54 -07:00
greg
51745fd800 Make convenience macro for parse errors 2017-09-19 22:10:54 -07:00
greg
8c0ac19fa8 Add full test 
Test evaluate, tokenize, parser all at once
 2017-09-19 22:10:54 -07:00
greg
971ab9ba21 Implement paren expressions 2017-09-19 22:10:54 -07:00
greg
819fb3f58f Basic evaluator functionality 
Interpreter now works for simple arithmetic expressions
 2017-09-19 22:10:54 -07:00
greg
1c23329656 Add boilerplate for evaluation 
Just wires everything up, doesn't actually evaluate yet
 2017-09-19 22:10:54 -07:00
greg
6da20cbfaf Address compiler warnings 2017-09-19 22:10:54 -07:00
greg
f48451125e Parsing arithmetic expressions works 
At the expense of an unnecessary move in lookahead()
 2017-09-19 22:10:54 -07:00
greg
06c3999430 Add expect_identifier function 
For utility
 2017-09-19 22:10:54 -07:00
greg
1af1589550 Make Parser class internally use IntoIterator 
And wrap the next() and peek() methods
 2017-09-19 22:10:54 -07:00
greg
32a90b8103 write expect 
and also make the ParseResult type more general so it can be used for
more things.
 2017-09-19 22:10:54 -07:00
greg
186c900920 Implement expect macro 
Seems like there's no way around passing in self manually
 2017-09-19 22:10:54 -07:00
greg
509ab80b9c I can now parse one thing 2017-09-19 22:10:54 -07:00
greg
247638c4db Get parsing REPL working 2017-09-19 22:10:54 -07:00
greg
be98f8387e Add another test 2017-09-19 22:10:54 -07:00
greg
841b38d5b1 Add tokenization test 2017-09-19 22:10:53 -07:00
greg
16dfbb27d5 Use state features from simplerepl 2017-09-19 22:10:53 -07:00
greg
3af7e6a409 Refactoring Schala 
Gonna work on Schala in earnest now! Using the simplerepl crate instead
of a build-in REPL, temporarily dropping parsing and evaluation code.
 2017-09-19 22:10:53 -07:00
greg
123f388711 Rename language to "Schala" 2017-09-19 22:10:53 -07:00
greg
bb349cda5f Fix newline tokenizing bug 
Still need to fix <statements> parsing because of the final newline
 2017-09-19 22:10:53 -07:00
greg
caa331ecdc Read from file as well as repl 2017-09-19 22:10:53 -07:00
greg
ae3a030ad8 Use iterative semantics for reduction 
Becuase if you try to make reduce() recursive you blow out the stack.

Incidentally, "let a = 0; while a < 999999; let a = a + 1 end" is a neat
thing to try at this stage of the game
 2017-09-19 22:10:53 -07:00
greg
ddb09b453d Implement booleans 2017-09-19 22:10:53 -07:00
greg
9a4760d44f Implement while-reduction 2017-09-19 22:10:53 -07:00
greg
eb5ce2ef9e Don't print newline for empty REPL result 2017-09-19 22:10:53 -07:00
greg
b080ea7c81 Add if-statement evaluation 2017-09-19 22:10:53 -07:00
greg
dffab8ae94 Fixed arg list parsing 2017-09-19 22:10:53 -07:00
greg
48ee6c9a75 Add function name to parse 2017-09-19 22:10:53 -07:00
greg
58d399dace More function parsing 2017-09-19 22:10:53 -07:00
greg
57ea1bae30 Preliminary addition of function blocks 2017-09-19 22:10:53 -07:00
greg
e870d8172a Make = a keyword 2017-09-19 22:10:53 -07:00
greg
72b26755a7 Make ParseResult just a normal Result type 
No reason for it to be different
 2017-09-19 22:10:53 -07:00
greg
1416c9d444 Parse null as simple_expression 2017-09-19 22:10:53 -07:00
greg
5c79563e53 Reduce binop args before reducing full expr 2017-09-19 22:10:53 -07:00
greg
c27c900e7f Add operators % and ** 
semantics for these are hard with floats handle later
 2017-09-19 22:10:53 -07:00
greg
50c5dbe96d Evaluate simple binops 
Woo this is kind of a neat milestone! I can now compute with this
language!
 2017-09-19 22:10:53 -07:00
greg
be8c8b3343 Add paren test 2017-09-19 22:10:53 -07:00
greg
b856023072 Fixed paren parsing 
also made error reporting a bit nicer
 2017-09-19 22:10:53 -07:00
greg
868373f409 use match syntax in simple_parse 2017-09-19 22:10:53 -07:00
greg
582a7fd6dc Make parse and tokens optional 2017-09-19 22:10:53 -07:00
greg
a947ec3cb2 Add simple parsing test 2017-09-19 22:10:53 -07:00
greg
79619025ea Add directive to print precedence chart 2017-09-19 22:10:53 -07:00
greg
56b338a6a8 Move to global precedence table 2017-09-19 22:10:53 -07:00
greg
7d6f946e22 Fill out a few more precedences 2017-09-19 22:10:53 -07:00
greg
0da7f7e3a1 Fully fixed binop parsing 2017-09-19 22:10:53 -07:00
greg
19a344fa77 Fixed precedent-less binop parsing 2017-09-19 22:10:53 -07:00
greg
626a7f3861 Working on binop parsing 2017-09-19 22:10:53 -07:00
greg
8e3a571d67 .env dirctive to display environment 2017-09-19 22:10:53 -07:00
greg
f88f115567 Environment persistent across repl loop 2017-09-19 22:10:53 -07:00
greg
08f1092b69 Variable lookup works 
Note this introduces a panic - if the AST node inserted into the
environment is not reduced, it throws an error when trying to look it up
 2017-09-19 22:10:53 -07:00
greg
6ddea790c0 Beginning of variable lookup 
everything is null
 2017-09-19 22:10:53 -07:00
greg
6897eb1283 Implemented variable binding 2017-09-19 22:10:53 -07:00
greg
34fdf2be00 Add machinery for evaluation environments 2017-09-19 22:10:53 -07:00
greg
4ef93fafb5 Cause tokenize error for unclosed strings 2017-09-19 22:10:53 -07:00
greg
d2108f0f97 First pass at evaluation 
Very incomplete
 2017-09-19 22:10:52 -07:00
greg
2989ac338c Implemented binop parsing 
Uses Operator-precedence parsing algorithm, which I don't fully
understand.
 2017-09-19 22:10:52 -07:00
greg
8b6d54aec2 Fix let clause parsing 
let a = x, x should be expression not just simple expression
 2017-09-19 22:10:52 -07:00
greg
cdb47bb3b9 Add paren parsing 2017-09-19 22:10:52 -07:00
greg
9d6dc5a5f2 Tokenize periods separately 2017-09-19 22:10:52 -07:00
greg
8f9bfbc5bd Rename rhs to simple_expression 2017-09-19 22:10:52 -07:00
greg
c9fdd5e83c Simplified statements-parsing 
Still a little wonky wrt extraneous Separators, need to adjust grammar
to fix I think
 2017-09-19 22:10:52 -07:00
greg
30eddf7737 while statements 2017-09-19 22:10:52 -07:00
greg
42719dc2f2 Change 'input' to 'tokens' 
just to be consistent
 2017-09-19 22:10:52 -07:00
greg
8fcc850d77 Added else clause to if parsing 2017-09-19 22:10:52 -07:00
greg
f421918945 basic if expression 2017-09-19 22:10:52 -07:00
greg
0e4469fa58 Filled out keyword tokenizing 2017-09-19 22:10:52 -07:00
greg
edf100b583 Starting to do if statement parsing 2017-09-19 22:10:52 -07:00
greg
169e662049 Collapse Separator tokens 
only ever gonna be one in a row
 2017-09-19 22:10:52 -07:00
greg
46999beabf Added skeleton of expression() parser 2017-09-19 22:10:52 -07:00
greg
1342a76786 Added support for interpreter directives 2017-09-19 22:10:52 -07:00
greg
05238bced3 rhs production 2017-09-19 22:10:52 -07:00
greg
a97cce184c Empty program is valid too 2017-09-19 22:10:52 -07:00
greg
329c521964 Parsing statement blocks works 2017-09-19 22:10:52 -07:00
greg
bfa16fd6fb Added Keyword lexical class 2017-09-19 22:10:52 -07:00
greg
25f5188d8c Move definition around 2017-09-19 22:10:52 -07:00
greg
5213dd327f Change type to peekable 2017-09-19 22:10:52 -07:00
greg
cea29094cd type alias for Tokens 2017-09-19 22:10:52 -07:00
greg
67eafba97a Put expect into early return macro 2017-09-19 22:10:52 -07:00
greg
1059a88ee6 Separate parsing into module 2017-09-19 22:10:52 -07:00
greg
429ace73bd Move tokenizing into separate module 2017-09-19 22:10:52 -07:00
greg
044e7a6a26 Rename ASTNode -> AST 
saves typing
 2017-09-19 22:10:52 -07:00
greg
dbdae42c1b Add string to AST 2017-09-19 22:10:52 -07:00
greg
fc3dcf792d Start writing recursive descent parser 
I think I get the idea now
 2017-09-19 22:10:52 -07:00
greg
02b34ca105 Wrote expect() 
Hopefully correctly?
 2017-09-19 22:10:52 -07:00
greg
9f4330889a Starting parsing work 2017-09-19 22:10:52 -07:00
greg
3058af4f05 Break on ctrl-D 2017-09-19 22:10:52 -07:00
greg
4f17d5a0dc Add number tokenizing 2017-09-19 22:10:52 -07:00
greg
8e3774ffca Comma as separate token 2017-09-19 22:10:52 -07:00
greg
c6059ada7d Separators and parens 
Separator = ; or \n, they are equivalent
 2017-09-19 22:10:52 -07:00
greg
2aaa600d53 More tokenizer stuff 2017-09-19 22:10:51 -07:00
greg
c6a92728ee Scaffolding for evaluation function 2017-09-19 22:10:51 -07:00
greg
b2e23bed86 Print tokens and parse 2017-09-19 22:10:51 -07:00
greg
3fdacf018e Basic repl 2017-09-19 22:10:51 -07:00
greg
6098bf84d6 Initial commit 2017-09-19 22:10:51 -07:00