greg
/
schala
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
463 Commits 32 Branches 0 Tags 4.4 MiB
Commit Graph

442 Commits

Author SHA1 Message Date
greg f53c14535b Made error! macro more programtic 
TODO implement Display on Token so we're not just displaying the debug
name of the token enum variants
 2017-09-19 22:10:55 -07:00
greg 4f96abd7d9 Changes to make the code more concise 2017-09-19 22:10:55 -07:00
greg fdaf4c302c Fix all compiler warnings 2017-09-19 22:10:55 -07:00
greg 8ce53d7c72 Fix bind error 2017-09-19 22:10:55 -07:00
greg 428d560e2a Add tests for call expr parsing 2017-09-19 22:10:55 -07:00
greg 80bc7ec089 Proper call expression parsing 2017-09-19 22:10:55 -07:00
greg e6591b80d9 Add paren test 2017-09-19 22:10:55 -07:00
greg e099f713ad Add binop parsing test 2017-09-19 22:10:55 -07:00
greg 9b54256521 Import types for breivty 
and rename function to be explicit
 2017-09-19 22:10:55 -07:00
greg 087402ece6 Add more tests 
Need to use box patterns
 2017-09-19 22:10:55 -07:00
greg 252b6e8bd9 Okay, this strategy makes the test work 2017-09-19 22:10:55 -07:00
greg b1163e2ae4 Operator-precedence parsing + tests 
The tests are crippled now, because it's hard to write a test macro that
can also match on Strings
 2017-09-19 22:10:55 -07:00
greg 032d01c9f5 Fix tokenization bug 2017-09-19 22:10:55 -07:00
greg c4ab1ed105 Fix tokenizer tests 2017-09-19 22:10:55 -07:00
greg 9a257f08d7 Introduce Op type 
For operator parsing
 2017-09-19 22:10:55 -07:00
greg 47d56a7b44 fix operator parsing 2017-09-19 22:10:55 -07:00
greg 1f7ae2e30f Paren expression 2017-09-19 22:10:55 -07:00
greg e3c8753a4d Expression parsing 2017-09-19 22:10:55 -07:00
greg e1aa7ecb17 Finish tokenizing Op separately 2017-09-19 22:10:55 -07:00
greg f09a6e14ba Tokenizer work to support operators 
work in progress but committing to transfer
 2017-09-19 22:10:55 -07:00
greg 31da25a66e Expression parsing work 2017-09-19 22:10:55 -07:00
greg fff9cb7d25 Fix functin parsing 2017-09-19 22:10:55 -07:00
greg db1e188fdb Move grammar to top of file 2017-09-19 22:10:55 -07:00
greg 935185ed92 more parsing 2017-09-19 22:10:55 -07:00
greg 0999cbe28e More parsing work 2017-09-19 22:10:55 -07:00
greg 674f70a428 Convert parsing to method-based 2017-09-19 22:10:54 -07:00
greg b1b6672399 Implement function parsing 
With a lot of dummy code, especially around expression parsing
 2017-09-19 22:10:54 -07:00
greg bd1c455dc8 Basic infrastructure parses 
Also got rid of EOF, don't need it
 2017-09-19 22:10:54 -07:00
greg b62ef43f07 Add basic BNF grammar 2017-09-19 22:10:54 -07:00
greg 09b67dc3f7 Change error message 2017-09-19 22:10:54 -07:00
greg a613fa73e5 Basic parsing framework 2017-09-19 22:10:54 -07:00
greg 8c473c554e Fix bug 2017-09-19 22:10:54 -07:00
greg 3e04cbfa29 Add comma tokenization 2017-09-19 22:10:54 -07:00
greg 570650cbfa Finish keyword tokenization 2017-09-19 22:10:54 -07:00
greg 49be163181 Add test to ignore 
For better handing of user-defined operators, which I will do in the
future
 2017-09-19 22:10:54 -07:00
greg b4f93acbd8 Couple more tests 2017-09-19 22:10:54 -07:00
greg 8c65ae3214 Macro-ize token tests 2017-09-19 22:10:54 -07:00
greg e436533638 Passing test 2017-09-19 22:10:54 -07:00
greg 4601a56867 Start working on tokenization tests 2017-09-19 22:10:54 -07:00
greg 2f7a1850db Finish tokenizing 2017-09-19 22:10:54 -07:00
greg 71aef379d3 Tokenize number literals 
TODO: expand this bit of code to handle 0x12, etc. syntax
 2017-09-19 22:10:54 -07:00
greg 8662a3ba0e Make tokenize error-able 2017-09-19 22:10:54 -07:00
greg 5ca98c7d77 Print tokeniziation 2017-09-19 22:10:54 -07:00
greg 13cde3106c Start making tokenizer changes 
Hopefully this time iron out all the bugs from the last implementation
 2017-09-19 22:10:54 -07:00
greg 09d524c74a Changing how parsing works again 
Someone wrote a port of the LLVM kaleidoscope tutorial to rust, namely
https://github.com/jauhien/iron-kaleidoscope

I'm just gonna follow this along
 2017-09-19 22:10:54 -07:00
greg 61c36c4def Fix assign parsing 
= is a keyword not an identifier
 2017-09-19 22:10:54 -07:00
greg bc4fbe4276 Start implementing definition 
WIP, doesn't work
 2017-09-19 22:10:54 -07:00
greg fc11ee753d Add block parsing 
Right now evaluating a block reduces it to just the last AST in it, will
fix later with environments
 2017-09-19 22:10:54 -07:00
greg dd2b4893a4 Get rid of Separator token 
Have separate newline and semicolon tokens
 2017-09-19 22:10:54 -07:00
greg 51745fd800 Make convenience macro for parse errors 2017-09-19 22:10:54 -07:00
greg 8c0ac19fa8 Add full test 
Test evaluate, tokenize, parser all at once
 2017-09-19 22:10:54 -07:00
greg 971ab9ba21 Implement paren expressions 2017-09-19 22:10:54 -07:00
greg 819fb3f58f Basic evaluator functionality 
Interpreter now works for simple arithmetic expressions
 2017-09-19 22:10:54 -07:00
greg 1c23329656 Add boilerplate for evaluation 
Just wires everything up, doesn't actually evaluate yet
 2017-09-19 22:10:54 -07:00
greg 6da20cbfaf Address compiler warnings 2017-09-19 22:10:54 -07:00
greg f48451125e Parsing arithmetic expressions works 
At the expense of an unnecessary move in lookahead()
 2017-09-19 22:10:54 -07:00
greg 06c3999430 Add expect_identifier function 
For utility
 2017-09-19 22:10:54 -07:00
greg 1af1589550 Make Parser class internally use IntoIterator 
And wrap the next() and peek() methods
 2017-09-19 22:10:54 -07:00
greg 32a90b8103 write expect 
and also make the ParseResult type more general so it can be used for
more things.
 2017-09-19 22:10:54 -07:00
greg 186c900920 Implement expect macro 
Seems like there's no way around passing in self manually
 2017-09-19 22:10:54 -07:00
greg 509ab80b9c I can now parse one thing 2017-09-19 22:10:54 -07:00
greg 247638c4db Get parsing REPL working 2017-09-19 22:10:54 -07:00
greg be98f8387e Add another test 2017-09-19 22:10:54 -07:00
greg 841b38d5b1 Add tokenization test 2017-09-19 22:10:53 -07:00
greg 16dfbb27d5 Use state features from simplerepl 2017-09-19 22:10:53 -07:00
greg 3af7e6a409 Refactoring Schala 
Gonna work on Schala in earnest now! Using the simplerepl crate instead
of a build-in REPL, temporarily dropping parsing and evaluation code.
 2017-09-19 22:10:53 -07:00
greg 123f388711 Rename language to "Schala" 2017-09-19 22:10:53 -07:00
greg bb349cda5f Fix newline tokenizing bug 
Still need to fix <statements> parsing because of the final newline
 2017-09-19 22:10:53 -07:00
greg caa331ecdc Read from file as well as repl 2017-09-19 22:10:53 -07:00
greg ae3a030ad8 Use iterative semantics for reduction 
Becuase if you try to make reduce() recursive you blow out the stack.

Incidentally, "let a = 0; while a < 999999; let a = a + 1 end" is a neat
thing to try at this stage of the game
 2017-09-19 22:10:53 -07:00
greg ddb09b453d Implement booleans 2017-09-19 22:10:53 -07:00
greg 9a4760d44f Implement while-reduction 2017-09-19 22:10:53 -07:00
greg eb5ce2ef9e Don't print newline for empty REPL result 2017-09-19 22:10:53 -07:00
greg b080ea7c81 Add if-statement evaluation 2017-09-19 22:10:53 -07:00
greg dffab8ae94 Fixed arg list parsing 2017-09-19 22:10:53 -07:00
greg 48ee6c9a75 Add function name to parse 2017-09-19 22:10:53 -07:00
greg 58d399dace More function parsing 2017-09-19 22:10:53 -07:00
greg 57ea1bae30 Preliminary addition of function blocks 2017-09-19 22:10:53 -07:00
greg e870d8172a Make = a keyword 2017-09-19 22:10:53 -07:00
greg 72b26755a7 Make ParseResult just a normal Result type 
No reason for it to be different
 2017-09-19 22:10:53 -07:00
greg 1416c9d444 Parse null as simple_expression 2017-09-19 22:10:53 -07:00
greg 5c79563e53 Reduce binop args before reducing full expr 2017-09-19 22:10:53 -07:00
greg c27c900e7f Add operators % and ** 
semantics for these are hard with floats handle later
 2017-09-19 22:10:53 -07:00
greg 50c5dbe96d Evaluate simple binops 
Woo this is kind of a neat milestone! I can now compute with this
language!
 2017-09-19 22:10:53 -07:00
greg be8c8b3343 Add paren test 2017-09-19 22:10:53 -07:00
greg b856023072 Fixed paren parsing 
also made error reporting a bit nicer
 2017-09-19 22:10:53 -07:00
greg 868373f409 use match syntax in simple_parse 2017-09-19 22:10:53 -07:00
greg 582a7fd6dc Make parse and tokens optional 2017-09-19 22:10:53 -07:00
greg a947ec3cb2 Add simple parsing test 2017-09-19 22:10:53 -07:00
greg 79619025ea Add directive to print precedence chart 2017-09-19 22:10:53 -07:00
greg 56b338a6a8 Move to global precedence table 2017-09-19 22:10:53 -07:00
greg 7d6f946e22 Fill out a few more precedences 2017-09-19 22:10:53 -07:00
greg 0da7f7e3a1 Fully fixed binop parsing 2017-09-19 22:10:53 -07:00
greg 19a344fa77 Fixed precedent-less binop parsing 2017-09-19 22:10:53 -07:00
greg 626a7f3861 Working on binop parsing 2017-09-19 22:10:53 -07:00
greg 8e3a571d67 .env dirctive to display environment 2017-09-19 22:10:53 -07:00
greg f88f115567 Environment persistent across repl loop 2017-09-19 22:10:53 -07:00
greg 08f1092b69 Variable lookup works 
Note this introduces a panic - if the AST node inserted into the
environment is not reduced, it throws an error when trying to look it up
 2017-09-19 22:10:53 -07:00
greg 6ddea790c0 Beginning of variable lookup 
everything is null
 2017-09-19 22:10:53 -07:00
greg 6897eb1283 Implemented variable binding 2017-09-19 22:10:53 -07:00
greg 34fdf2be00 Add machinery for evaluation environments 2017-09-19 22:10:53 -07:00
greg 4ef93fafb5 Cause tokenize error for unclosed strings 2017-09-19 22:10:53 -07:00
greg d2108f0f97 First pass at evaluation 
Very incomplete
 2017-09-19 22:10:52 -07:00
greg 2989ac338c Implemented binop parsing 
Uses Operator-precedence parsing algorithm, which I don't fully
understand.
 2017-09-19 22:10:52 -07:00
greg 8b6d54aec2 Fix let clause parsing 
let a = x, x should be expression not just simple expression
 2017-09-19 22:10:52 -07:00
greg cdb47bb3b9 Add paren parsing 2017-09-19 22:10:52 -07:00
greg 9d6dc5a5f2 Tokenize periods separately 2017-09-19 22:10:52 -07:00
greg 8f9bfbc5bd Rename rhs to simple_expression 2017-09-19 22:10:52 -07:00
greg c9fdd5e83c Simplified statements-parsing 
Still a little wonky wrt extraneous Separators, need to adjust grammar
to fix I think
 2017-09-19 22:10:52 -07:00
greg 30eddf7737 while statements 2017-09-19 22:10:52 -07:00
greg 42719dc2f2 Change 'input' to 'tokens' 
just to be consistent
 2017-09-19 22:10:52 -07:00
greg 8fcc850d77 Added else clause to if parsing 2017-09-19 22:10:52 -07:00
greg f421918945 basic if expression 2017-09-19 22:10:52 -07:00
greg 0e4469fa58 Filled out keyword tokenizing 2017-09-19 22:10:52 -07:00
greg edf100b583 Starting to do if statement parsing 2017-09-19 22:10:52 -07:00
greg 169e662049 Collapse Separator tokens 
only ever gonna be one in a row
 2017-09-19 22:10:52 -07:00
greg 46999beabf Added skeleton of expression() parser 2017-09-19 22:10:52 -07:00
greg 1342a76786 Added support for interpreter directives 2017-09-19 22:10:52 -07:00
greg 05238bced3 rhs production 2017-09-19 22:10:52 -07:00
greg a97cce184c Empty program is valid too 2017-09-19 22:10:52 -07:00
greg 329c521964 Parsing statement blocks works 2017-09-19 22:10:52 -07:00
greg bfa16fd6fb Added Keyword lexical class 2017-09-19 22:10:52 -07:00
greg 25f5188d8c Move definition around 2017-09-19 22:10:52 -07:00
greg 5213dd327f Change type to peekable 2017-09-19 22:10:52 -07:00
greg cea29094cd type alias for Tokens 2017-09-19 22:10:52 -07:00
greg 67eafba97a Put expect into early return macro 2017-09-19 22:10:52 -07:00
greg 1059a88ee6 Separate parsing into module 2017-09-19 22:10:52 -07:00
greg 429ace73bd Move tokenizing into separate module 2017-09-19 22:10:52 -07:00
greg 044e7a6a26 Rename ASTNode -> AST 
saves typing
 2017-09-19 22:10:52 -07:00
greg dbdae42c1b Add string to AST 2017-09-19 22:10:52 -07:00
greg fc3dcf792d Start writing recursive descent parser 
I think I get the idea now
 2017-09-19 22:10:52 -07:00
greg 02b34ca105 Wrote expect() 
Hopefully correctly?
 2017-09-19 22:10:52 -07:00
greg 9f4330889a Starting parsing work 2017-09-19 22:10:52 -07:00
greg 3058af4f05 Break on ctrl-D 2017-09-19 22:10:52 -07:00
greg 4f17d5a0dc Add number tokenizing 2017-09-19 22:10:52 -07:00
greg 8e3774ffca Comma as separate token 2017-09-19 22:10:52 -07:00
greg c6059ada7d Separators and parens 
Separator = ; or \n, they are equivalent
 2017-09-19 22:10:52 -07:00
greg 2aaa600d53 More tokenizer stuff 2017-09-19 22:10:51 -07:00
greg c6a92728ee Scaffolding for evaluation function 2017-09-19 22:10:51 -07:00
greg b2e23bed86 Print tokens and parse 2017-09-19 22:10:51 -07:00
greg 3fdacf018e Basic repl 2017-09-19 22:10:51 -07:00
greg 6098bf84d6 Initial commit 2017-09-19 22:10:51 -07:00