I'm trying out a code generation problem using open-source transformer models. With almost all of the models, the context window size seems to be a problem.

I had a thought that, instead of generating the code for the target language directly, what if we can generate a concise code and transpile it? In that approach, I feel generation could be in some LISP dialect and from there transpilation could be to any language. Of course APL can be an intermediate language here but that has other challanges like learning, custom keyboard, etc.

Does this approach seems viable?