Building a Lox Interpreter in Julia(lukemerrick.com) |
Building a Lox Interpreter in Julia(lukemerrick.com) |
Once you've done that, you can see how it's done "for real" in a production setting by reading the source code to Wren (by the same author). [0]
Wren's implementation maps very closely to the C implementation of Lox.
At that point, you're ready to do your own language. As far as compilation techniques go, you'll still be missing the "middle-end" of the compiler, which uses an SSA IR. I don't recommend implementing this yourself, I'd look into MLIR (from the LLVM project) if you want to actually work on the middle-end. You can create one or more dialects that are unique to your language, and implement your own compiler transformations. There are lots of existing papers and projects on GitHub demonstrating doing so.
[0] https://wren.io/
One thing you can do with the finished Lox (or Monkey, if you prefer WACIG) before going into the world of intermediate representations is implementing a peephole optimizer. You look for reducible patterns in the bytecode, and replace them with optimized bytecode. You can also look for certain patterns and replace naive implementations with native builtins/intrinsics. You can work with the raw bytes of the bytecode, so you don't need to introduce an IR just yet.
The Apex compiler at Salesforce does a vast majority of its optimizations as peeps.
EDIT: I _just_ wrote another comment to someone asking similar questions a few days ago. Here's a link to the parent question, check out my thoughts as well as others in the thread. https://news.ycombinator.com/item?id=36119915
If this sounds at all appealing to you, I would check it out. I found Monkey so far somewhat more compelling than Lox (might be personal preference, though), and I thought the exposition (and code quality) was overall better. Go is also very easy to pick up.
Another option is to try and contribute to an existing language (Julia ;) ).
I'm not sure exactly how Rust and rust-analyzer keep track of the info necessary to their excellent error messages and diagnostics, but I wouldn't be surprised if pinpoint messages were not the primary motivation for rust-analyzer to do lossless parsing.
I'm not sure how far down the compiler I actually will enjoy going vs. exploring ideas around type systems, linters, etc. up near the AST level, but if I do venture down this advice will certainly come in handy!
Reconsidering now, it seems that there might be benefits beyond type dispatch to having a typed syntax tree, so maybe I'll give that a shot as a next step!