Commit Graph

27 Commits

Author SHA1 Message Date
greg
3e04cbfa29 Add comma tokenization 2017-09-19 22:10:54 -07:00
greg
570650cbfa Finish keyword tokenization 2017-09-19 22:10:54 -07:00
greg
49be163181 Add test to ignore
For better handing of user-defined operators, which I will do in the
future
2017-09-19 22:10:54 -07:00
greg
b4f93acbd8 Couple more tests 2017-09-19 22:10:54 -07:00
greg
8c65ae3214 Macro-ize token tests 2017-09-19 22:10:54 -07:00
greg
e436533638 Passing test 2017-09-19 22:10:54 -07:00
greg
4601a56867 Start working on tokenization tests 2017-09-19 22:10:54 -07:00
greg
2f7a1850db Finish tokenizing 2017-09-19 22:10:54 -07:00
greg
71aef379d3 Tokenize number literals
TODO: expand this bit of code to handle 0x12, etc. syntax
2017-09-19 22:10:54 -07:00
greg
8662a3ba0e Make tokenize error-able 2017-09-19 22:10:54 -07:00
greg
13cde3106c Start making tokenizer changes
Hopefully this time iron out all the bugs from the last implementation
2017-09-19 22:10:54 -07:00
greg
09d524c74a Changing how parsing works again
Someone wrote a port of the LLVM kaleidoscope tutorial to rust, namely
https://github.com/jauhien/iron-kaleidoscope

I'm just gonna follow this along
2017-09-19 22:10:54 -07:00
greg
dd2b4893a4 Get rid of Separator token
Have separate newline and semicolon tokens
2017-09-19 22:10:54 -07:00
greg
509ab80b9c I can now parse one thing 2017-09-19 22:10:54 -07:00
greg
be98f8387e Add another test 2017-09-19 22:10:54 -07:00
greg
841b38d5b1 Add tokenization test 2017-09-19 22:10:53 -07:00
greg
3af7e6a409 Refactoring Schala
Gonna work on Schala in earnest now! Using the simplerepl crate instead
of a build-in REPL, temporarily dropping parsing and evaluation code.
2017-09-19 22:10:53 -07:00
greg
bb349cda5f Fix newline tokenizing bug
Still need to fix <statements> parsing because of the final newline
2017-09-19 22:10:53 -07:00
greg
e870d8172a Make = a keyword 2017-09-19 22:10:53 -07:00
greg
4ef93fafb5 Cause tokenize error for unclosed strings 2017-09-19 22:10:53 -07:00
greg
9d6dc5a5f2 Tokenize periods separately 2017-09-19 22:10:52 -07:00
greg
0e4469fa58 Filled out keyword tokenizing 2017-09-19 22:10:52 -07:00
greg
edf100b583 Starting to do if statement parsing 2017-09-19 22:10:52 -07:00
greg
169e662049 Collapse Separator tokens
only ever gonna be one in a row
2017-09-19 22:10:52 -07:00
greg
bfa16fd6fb Added Keyword lexical class 2017-09-19 22:10:52 -07:00
greg
1059a88ee6 Separate parsing into module 2017-09-19 22:10:52 -07:00
greg
429ace73bd Move tokenizing into separate module 2017-09-19 22:10:52 -07:00