Why Target Common Lisp for Code Generation?(funcall.blogspot.com) |
Why Target Common Lisp for Code Generation?(funcall.blogspot.com) |
My understanding is that LLM are deflationary, resulting in code being priced as a commodity (cost plus). If this bears out, then lower token costs and higher performance (esp in my case, where good performance results in lower infra costs to host the service) will start to matter more and more. Common Lisp code bases are famously dense (in the good sense), resulting in perhaps lower token costs when an LLM needs to review and make a change (smaller code bases also help humans / small teams). Common Lisp can achieve performance that’s close to that of an equivalent implementation in C or C++.
Taken together, they make for one case for using CL and other similar languages, although one that’s grounded largely in economics.
Are there any programming/computer science greats that self aggrandize like this? Is it a cultural thing somewhere that I’m not aware of?
Further this 'Democritisation' can be seen as trying to deskill something that taking away many positives and having contempt for skill. I believe This is often simply to devalue software developers and I assume you are one so I believe we should both be cautious of this devaluation.
I think a more interesting question would be why should we not have elite's and I don't know if I am unusual but false humility always rubs me the wrong way?
Small children should learn some kind of Lisp, and easily can.
Richard Stallman famously shared an anecdote in the article "My Lisp Experiences and the Development of GNU Emacs" (https://www.gnu.org/gnu/rms-lisp.en.html) about the secretaries:
> The editor itself was written entirely in Lisp. Multics Emacs proved to be a great success—programming new editing commands was so convenient that even the secretaries in his office started learning how to use it. They used a manual someone had written which showed how to extend Emacs, but didn't say it was a programming. So the secretaries, who believed they couldn't do programming, weren't scared off. They read the manual, discovered they could do useful things and they learned to program.
People, whether inside or outside of Lisp, who spread myths that that it is some scary, esoteric elite family of languages, have been pretty harmful.
Once you've toiled on the learning curve of Lisp long enough to really grok it you might begin to think you've discovered the foundational bedrock of the universe. From that perspective it's kind of hard not to be a little condescending toward fans of other languages.
It doesn't help that younger programmers today who complain about "too many parentheses" are largely unaware that Lisp was the original language of AI because the guy who invented AI also invented Lisp to help him do AI. I for one am just a tiny bit chuffed about that because I'm certain that more and better progress on LLMs (and beyond) would have happened had Lisp been used instead of Python as the primary vehicle for AI exploration.
So yeah, I damn well consider myself elite and I won't apologize for it.
Less so with Clojure.
I think that covers my list of grievances though, and it’s a pretty short list. Other than those, CL has been good to me.
I'd be very interested in reading more about this.
I do agree with a lot of the article though, Lisp plays nicely with LLMs. Definitely had better luck with Lisp and LLMs than C++ or Rust.
Also, kind of random, but here's an interesting tool to use CL with LLMs: https://www.lambda-symbolics.com/autolith
1. The author's first argument is that CL is familiar to them—fine. However, this does not translate into any benefits for the model and is not generalizable.
2. The second argument is that HLLs are an unqualified good, and again that they reflect the desires of the author. It's certainly true that having less boilerplate to manage reduces the surface area for mistakes, and makes code easier to review, but this is only a very weak argument in Lisp's favor. Given the right library, most HLLs can be as flexible as Lisp, especially newer ones where questions about standard library design and collection behavior are still being explored. Conversely CL's standard library is indebted to decades of history. No racehorse wins races forever.
3. The third argument is that Lisp was somehow intended for exclusive use by qualified individuals—this is absolute brainrot. It was designed to manage recursive structures on an IBM 704, and that's about it. It then repeats an unrelated statement about CL being 'built for maximum expressiveness', calling back to the second point; it has been surpassed.
4. The fourth argument trots out the infamous dead horse of homoiconicity. Parentheses have low information density. The LLM wastes a huge amount of context on indentation already, and to have only one style of parenthesis for the vast majority of code is contrary to its expertise. If you want to claim homoiconicity as an advantage, then you need to make your model generate an actual tree data structure, not indented, linear, parenthetical strings. LLMs cannot do this; the harness required defeats the benefit. Therefore point 4 is totally wrong. Homoiconic languages are only of value when you need something that resembles the intermediate state of a compiler—i.e., for macro transformations. The Markov-model-like architecture of an LLM does not benefit from this property whatsoever.
5. Next the author brings up using macros to form a DSL. If the macro is shallow (i.e. it doesn't perform deep syntax transformation) then it is just syntactic sugar and could be a function. If the macro is deep (i.e. it restructures all kinds of stuff) then you are putting a cognitive burden on the LLM to keep track of which syntactic rules apply, and the risk of failure goes up.
6. The author mentions that REPLs exist. This is, again, an example of wearing blinders. REPLs are so common and so good now that even web browsers come with better JS REPLs than a command-line.
7. The author thinks it is amazing when a development environment triggers a breakpoint instead of crashing on a failure. This feature is so standard that even gdb and MSVC do it. Visual Basic, famously, did it quite well.
8. Hot-reloading functions: This is, again, possible in a wide variety of languages, though not always as a default feature. For example, IPython adds this behavior to Python. Hot-reloading is not widespread because it often results in inconsistent program state.
Virtually all of CL's strengths are outmoded or defeated by giving the job of programming to a machine. It's like putting velvet inside your toilet tank: sure, it's fancy, but nobody benefits.
Even more importantly, though, the pool of available training data for CL is vanishingly small compared to other languages, which will increase the error rate dramatically. You should expect a lower quality of code for CL, and a high frequency of it being confused for similar, more popular languages like Scheme and Clojure. LLMs actually excel at Clojure, in part because its syntax is crunchier. All that sugar—applied consistently across the language, not sporadically in a rare DSL macro—helps the LLM keep track of what it's doing and where it is.
There's no such thing as "CL's standard library." I presume you mean the functions built into Common Lisp which provide basic functionality, but nobody uses "bare" Common Lisp any more than anybody uses bare C. The Quicklisp libraries for CL (among others) provide the modern utilities you seem to think CL lacks.