Porting Smarty to Twig with PEGs(developers.freshbooks.com) |
Porting Smarty to Twig with PEGs(developers.freshbooks.com) |
In the future I'd like to use a PEG with an additional tokenization step, as the parser generated from a pure top-down approach will give fairly useless error messages when you have a typo deep in a nested expression, and I think that could be mitigated if I could point those errors out during tokenization.
Errors in general are a hard part of PEG to solve. What you really want to do is save the "most correct" failed descent as you iterate across a rule. Usually they define it as the "deepest" failed subexpression. If your implementation doesn't implement error preservation like this, it's time to add it. :)
I don't think tokenization would really help here, and the preference by most PEGs to not have a separate class of token expressions is often cited as one of the major strengths of PEGs.