Modern, functional Common Lisp: myths and best practices(ambrevar.xyz) |
Modern, functional Common Lisp: myths and best practices(ambrevar.xyz) |
then M-x slime-connect localhost 4005
BTW, here's a k8s-hosted app I wrote using this approach: https://github.com/atgreen/red-light-green-light
EDIT: off topic, sorry, but I have been actively evaluating CL (using an embedded web server that starts a browser) vs. Swift (using mostly SwiftUI) for a new product I want to write. I find myself using Swift like I would CL: using Playgrounds to prototype low level code and utilities, then XCode for developing the UI. To be honest, Swift and SwiftUI is a better fit technology-wise for what I want to do, but I am so much happier when working in CL.
This is not true IMO, when compared with the languages of today. Even if you include the numerous little details of MOP, the CL spec is nowhere near as confusing as say something like C++ 17.
Have you evaluated CAPI? Have you looked into building Cocoa applications with Clozure CL?
The advantage of Swift and SwiftUI is that my application can run on both a MacBook and iPad, sync data with iCloud.
The advantage of LispWorks is that I could fairly easily support both Mac and Windows (and Linux).
I spent two evenings last week playing with Clozure CL and its Cocoa support. It has low level APIs and I had a peculiar issue with sometimes not being enter text into input components.
EDIT: to be clear, in general I love Clozure CL. In this instance I was probably shooting myself in the foot, somehow, or maybe the issue was that I blew the install in some way getting it running on Catalina which took over an hour. The Catalina problems are hopefully just short term issues.
In particular, is object identity with EQ consistent with functional programming? Constructors do not act like functions if EQ is the equality. Or should that be more "immutable programming"?
Common Lisp, because it has EQ and object identity, cannot perform some optimizations that a truly functional language's implementation could. In particular, it cannot combine equivalent function calls, and cannot merge equivalent data (hash consing or the equivalent.)
A year or so back I picked up a copy of Land of Lisp and burned through it for pleasure reading. And I was struck by how gross Lisp looks in that beginner-oriented treatment. Just this huge slog of car and cdr and let/letrec/let* and the 37 flavors of equals and the function namespace. . . and, all the while, you're being told that persevering in mastering this confusing minefield of subtleties will somehow enable you to write bug-free software. I doubt it's actually fun for most people, and the grandiose claims should beggar belief for everyone. I suppose I should count myself fortunate that I got to learn Lisp in college, where there was little attempt to make it fun, and plenty of graded assignments to keep me motivated.
Racket and Clojure are right to clean up some of the language's evolutionary history. That's a start. But even then, the treatment in beginner's guides isn't all that enticing. I've also skimmed through Realm of Racket and Clojure for the Brave and True, and, while both of them work hard at being entertaining (and were fun to just sit and read), they don't really succeed at dispelling the sensation that what you're mostly doing is wrangling with the language itself.
Compare with some of the more popular Python guides. They tend to be much more dryly written, but the actual flow of the guide tends to get you pretty quickly to
from pypi import have_fun
have_fun()Based on the blog posts I've read, it seems like CL occupies the kind of space I want to be in: sort of the halfway point between theoretical and engineering. Is that a fair conclusion to draw?
We can also use CL in Jupyter notebooks, see those good examples: https://gist.github.com/WetHat/a49e6f2140b401a190d45d31e052a...
And Vim, Lem, cl-repl, Eclipse (not so interactive)… https://lispcookbook.github.io/cl-cookbook/editor-support.ht...
As far as I can tell, it's more like you have to pick one or the other. Distribution is easy if you don't mind distributing a 50 MB image (which is, of course, trivial for some applications and catastrophic for others). Most of the suggestions on reducing application size are essentially accounting sleight-of-hand that doesn't actually reduce the overall footprint (I'm reminded of the compressor that can "compress" any file by one byte... by moving the byte to the file name).
EDIT: Based on a comment and downvote I suppose I didn't make my point clear above. I am saying that myths are not the only thing holding Lisp back. (I may be wrong. I hope I'm wrong. Big thank-you to people who are linking ways to get libraries in Lisp, especially Python libraries.)
I care 10x as much about libraries as about all these red herrings (EDIT: or myths, as you like) combined: " * Common Lisp does not have compile-time type checking. * Common Lisp is for imperative, object-oriented programming. * Common Lisp is too specialized, it’s not for general-purpose development. * Common Lisp applications are hard to deploy. "
> Common Lisp does not have compile-time type checking.
Nothing in the standard mentions compile time checking requirements and there is no useful de-facto standard that you (or tooling!) could seriously build upon either. I'd be suprised if python did not have better "compile-time type checking" for all practical purposes. Yes, SBCL gives much better type warnings at compile time than python, but for python you have mypy and it's A Thing, and still a joke compared to a proper type system like Ocaml's.
> Common Lisp is too specialized, it’s not for general-purpose development.
Common Lisp has no eco-system to speak of for machine learning, web development, command-line utilities, games programming, mobile development, GUI programming, embedded development or pretty much anything real general purpose languages do. If you have something for which Common Lisp is a good fit, you can still be commercially successful using it because not everything requires a super rich eco-system and for some problems what Common Lisp has is in fact highly competitive. But if you want to use common lisp effectively you definitely need to pre-select the problems you want to work on accordingly to an extent that's not true of python, C, C++, Rust, javascript, Go, Java, and half a dozen other languages.
> Common Lisp applications are hard to deploy.
No server or desktop or web-browser comes with common lisp pre-installed. Building common lisp is a pain, because packaging/ASDF is terrible and you don't easily get a nice and small statically linked executable out either. Compared to exactly what language is common lisp not hard to deploy?
That said, I support free software and I believe a proprietary compiler is not a good idea :(
A lot also provide you the source code once you pay. AllegroCL is one of them I think.
https://github.com/linpengcheng/PurefunctionPipelineDataflow
The article recommends SBCL, which does support TCO.
An old (2011) survey of TCO support is at https://0branch.com/notes/tco-cl.html
I know that the functional universe has solutions to these problems, like the State monad and reusable collections protocols based on common minimal primitives, but I really appreciate being able to just freely string together tokens like DOLIST WHEN EXT:COLLECT without any elaborate frameworks.
So I suppose that programming in Common Lisp makes me appreciate non-functional programming.
Having said that, I do feel like a caveman every time I spend brain cycles on picking between EQ/EQL/EQUAL/EQUALP/STRING= or worrying about whether the object I'm updating might come from a quoted constant and invite undefined behaviour. Hard to have it all...
I wrote a wrapper for it and SERIES, called folio, followed by a revised version called folio2. If your requirements resemble mine when I wrote it, you might findit useful.
You can find it here:
https://github.com/mikelevins/folio2/A package I've been involved with lately is fset, which is available through quicklisp, or at
https://github.com/slburson/fset
It has some interesting features, including "functional setf expansion". This would turn something like
(setf (fcar x) y)
into something equivalent to
(setf x (cons y (fcdr x)))
(where "fcar" and "fcdr" are the same as "car" and "cdr", except when they are in a setf-able place form.)
(fcar and fcdr are not in fset; I used those names just for exposition here.)
It could work with nested accessors, but only if it bottoms out in a variable.
By the way, I found a lot of your best practices questionable to say the least. Since it's obvious that you are a newcomer to CL, I would refrain from producing "best practices" type blog posts until I had a few years of experience under my belt.
For that reason, I also found your post confusing and I'm inclined to categorize it as "mostly rehashing stuff that is already there" rather than strong signal. There is ample, good, introductory material for CL on the net, we should strive to think before we dilute it with derivative posts.
The effort to get an editor going and to open a file and load a package (“system” in CL) is a lot. Of course, I’ll die on a hill claiming that those things are a constant overhead and aren’t even detectable when you wield lisp’s power.
Nothing inherently stops lisp from becoming simpler to start with and use, but most everybody who knows lisp has become not only competent with the current tooling, they’re happy with it. I’m happy too: I love using Emacs and SLIME. I love it better than VSCode, PyCharm, etc. I’m faster and more productive.
I wonder why other professions aren’t like programming. Adobe Premier isn’t exactly easy to use, yet professionals aren’t apparently clamoring for an iMovie equivalent.
Interesting. Nobody is born knowing any computer language. Languages therefore are popular (among other things) in proportion to their ability to convert "those who don't know the language" into "those who know", and "those who know" into actual users. A gentle introduction to those who don't know anything is therefore a good start. I suspect that both Lisp and Haskell suffer from this.
There are languages for people who want to get things done, and languages for people who want the perfect language for getting things done. The problem is that definitions of perfection differ between people, so people who are searching for perfect languages wind up in a small niche of people who agree, separated from people next door whose idea of perfection is just slightly different.
In the meantime people whose focus is on getting things done go with a popular language which builds a critical mass of ways to get things done. And the availability of useful libraries makes them actually better to get stuff done with than languages that are theoretically nicer in some way.
This is, of course, a worse is better kind of argument. See https://www.dreamsongs.com/RiseOfWorseIsBetter.html for context on that.
It’s true. Importing flask in Python and starting a server takes about 1/2 as much code as in Lisp. Some extraordinarily routine stuff Python has down to a one-liner. A lot of people have also written more Python libraries, so chances are you’ll be able to cobble any random doodad together with a huge dependency tree. But after a certain very short prototyping period, I end up fighting Python’s terrible deployment, terrible efficiency, and slapdash language implementation when trying to build something robust.
That said, I disagree with some of the points you make:
> availability of useful libraries
Clojure is embedded in both JS runtimes and the JVM. Both feature massive ecosystems.
In terms of getting things done:
Languages like Python, JS and so on are fantastic at plumbing. But when it comes to modelling data-structures, manipulation and algorithms then they are far less expressive and productive than Clojure from my experience.
I personally think the main issue really is the learning curve. There are quite a number of things that 'suck' about Clojure initially:
1. In a Lisp you are essentially manipulating an AST rather than writing statement-based text. Initially this is cumbersome and taxing both while reading and writing programs.
2. In a functional Lisp like Clojure (and other FP languages) there is a wide variety of commonly used functions which are used to manipulate and compose data-structures and other functions. Especially when reading code this can be daunting.
3. Setting up a environment correctly for REPL driven development as a beginner and tuning it as an intermediate user is quite an undertaking in comparison to many other languages.
4. Clojure specifically being a hosted language forces you to understand the hosted ecosystem as well plus the wiring between the host and Clojure.
The payoff for all those four points is worth it:
1. Search for paredit visualizations like this one: http://danmidwood.com/content/2014/11/21/animated-paredit.ht.... Manipulating code this way requires mechanical prowess and exercise but scales up really nicely, especially alongside better understanding of Lisp code in general.
2. The incredible variety of functional building blocks scales really well with your experience and understanding. You are programming with a series of descriptive expressions, rather than lower level statements. Code becomes more declarative, dense and expressive. Abstraction is much more fluent.
3. A nicely set-up environment enables very fast development cycles and allows for understanding pieces of code of any scope in isolation, because you can evaluate, change and test any expression instantly.
4. This is in my experience a necessary evil to get higher adoption and a the massive library ecosystems of JS and Java.
I assume there are a lot of capable developers who simply cannot get over 1 possibly while being pushed back by 2-4. And I have to admit: If I wasn't already infected with Lisp at a younger age, I probably wouldn't have bothered with Clojure or any Lisp. But now it is my favorite for manipulating and modelling data AKA doing 'business-logic' and transformations between APIs.
Also: "Lisp is the most fun you can have programming, yeah from the start," while subjective, is by most accounts, wrong. Especially when referring to Common Lisp; there could hardly be a more convoluted Lisp than Common Lisp.
(defun f (x) (* x x))
(setq g (compose f f))
(g 5)
These might go against the sensibilities one might have had in learning a Lisp in the first place. But in practice, these don’t stop you from writing solid, readable code.
It won’t feel as “clean” or “academic” as Scheme, but you’ll feel it easier to build large and efficient programs without pulling your hair out.
(setf (fdefinition g) (compose f f))
Basic Lisp programming with lists is a lot cleaner in a Lisp in which the empty list is false, and in which accessing nonexistent parts of a list (including the empty list) is a safe no-op that yields nil.
Ashwin Ram's (cdr (assq key a-list)) almost works in Common Lisp in the form of (cdr (assoc key a-list)). See: https://ashwinram.org/1986/01/28/a-short-ballad-dedicated-to...
Imperative programming is better supported in Common Lisp because the evaluation of the arguments of most forms, including function calls, is ordered, mainly left to right, so side effect embedded in expressions will show stable, portable behavior. Scheme function calls have unspecified evaluation order, much like C. (In Common Lisp, the only unspecified aspect is whether the function cell is sampled before the arguments are evaluated, or just before the function is called. It's extremely rare for the arguments of a function call to be redefining the function cell, needless to say.)
Common Lisp forms return a predictable value. If it doesn't make sense for a form to return a value, its value is not "undefined", but either just nil or else "no values". When a value returns no values, and an attempt is made to use its value anyway, then nil is produced as the value. You will not see some annoying #<undefined> in a Lisp REPL coming from a procedural construct.
To answer your question, the JVM has wide acceptance in the industry and a big ecosystem of libraries, from a business and practical point of view, it seems the most adequate choice for a Lisp?
There also might be an issue of speed, since CL is compiled to the metal. Modern JIT JVMs probably have mostly eliminated CL's speed advantage, but I'd still be surprised if some things in SBCL (arguably the best-optimized CL) didn't run faster than in Clojure.
Anyway I don't think the library ecosystem is so dire... Lots of good libraries are distributed through quicklisp. It's pretty straightforward to wrap a C library. (You can even do C++ easily if you switch CL implementations to Clasp...) If you need Java libraries, you can switch CL implementations again and use ABCL. (But of course all your CL code and CL libraries still work.) Lastly, for Python there's https://github.com/pinterface/burgled-batteries (and https://github.com/snmsts/burgled-batteries3 for py3) that even in an incomplete state might suit your particular library needs.
Perhaps this capability isn't very compelling since Python can also access C without trouble (and Java via Jython, .NET via IronPython)? Well I guess all I have left to ask is whether you've considered there might be libraries (or features) in Lisp that would be needed that don't have equivalents in Python? What do you do then? One possible library for admittedly niche applications that came to mind was a hierarchical task planner (https://github.com/shop-planner/shop3) but I forgot someone did indeed make a Python library (https://github.com/oubiwann/pyhop) based on an older version (SHOP1) of the background work, so depending on if you need the v3 features it might suffice.
No-doubt-unnecessary boilerplate: I understand that people have language wars, and it's possible you may have identified me as belonging to one side or another, but I'll pass. That whole state of affairs is just unfortunate.
(And a Numpy clone: https://numcl.github.io/numcl/)
https://run.nextjournalusercontent.com/kommen/parens-for-pol...
¿Por qué no los dos?
https://github.com/metawilm/cl-python
https://github.com/snmsts/burgled-batteries3
My point was that it is not just myths (or red herrings either) that are holding Lisp back. There are some issues based in reality as well.
>No server or desktop or web-browser comes with common lisp pre-installed. Building common lisp is a pain, because packaging/ASDF is terrible and you don't easily get a nice and small statically linked executable out either. Compared to exactly what language is common lisp not hard to deploy?
I'm not here to defend all of the flaws in CL, but I really enjoy the SBCL feature that allows you to compile to a binary. Yes it may not be as small as a pure C/C++/Rust executable, but it's very useful and I've used in various projects with great success.
My heart still goes out to Python but I was I could just as easily create a Python executable as I can with SBCL.
In my experience, deploying a CL app is muuuch less a pain than a Python one: you build a self-contained binary, you run it and voilà, you can access your web app from the internetz. The binary is ±20MB in size (compiler, debugger and all included). With Python, there are so many ways to fail (and even because a dependency of a library didn't pin its dependencies well enough. Gosh.).
Building a self-contained app is 1 line in the .asd project declaration.
https://lispcookbook.github.io/cl-cookbook/scripting.html#bu...
CLI utils: it's simple to use cl-readline. There are a couple libraries for ncurses. The Lem editor is a good example.
mobile: nope. For the adventurous, see https://gitlab.com/eql/EQL5-Android and its REPL.
GUI: https://github.com/CodyReichert/awesome-cl#gui Qt4, Gtk3, IUP, Tk, Nuklear have good bindings. Qt5 is possible with gobject-introspection (for the adventurous). Proprietary: CAPI. Electron: Ceramic. Java GUI interop with ABCL?
ML: https://github.com/CodyReichert/awesome-cl#machine-learning MGL's author won the Higgs Boson Machine Learning Challenge, but yeah.
for the rest: IDK https://lisp-lang.org/success/
GUI: would you develop a commercial app (to pay your rent) in any of these apart from CAPI?
ML: You are joking, right?
https://www.cliki.net/Machine%20Learning
> web development,
https://en.wikipedia.org/wiki/Viaweb
https://en.wikipedia.org/wiki/Reddit#Technology_and_design (though it admittedly got rewritten into Python)
> command-line utilities,
https://github.com/TeMPOraL/hju, among, like, hundreds of other examples
> games programming,
https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp
https://github.com/shirakumo/trial
https://borodust.org/projects/trivial-gamekit/getting-starte...
> mobile development,
http://kriyative.github.io/2011/03/26/ecl-for-ios-updated/
https://common-lisp.net/project/ecl/posts/Lisp-ECL-and-QML-Q...
> GUI programming,
https://common-lisp.net/project/mcclim/
> embedded development
http://www.ulisp.com/ (not technically Common Lisp per se AFAICT, but seems to have very similar semantics)
> Building common lisp is a pain, because packaging/ASDF is terrible and you don't easily get a nice and small statically linked executable out either.
https://www.xach.com/lisp/buildapp/
http://www.sbcl.org/manual/#index-save_002dlisp_002dand_002d...
[] For the record CLIM had interesting ideas, but last I looked, calling McCLIM's implementation of them a toy would be charitable.
SBCL supports TCO: see http://www.sbcl.org/manual/#index-Tail-recursion
One can drive whether SBCL optimizes tail calls by setting the proper optimization settings: see http://www.sbcl.org/manual/#Debugger-Policy-Control
Haskell uses a different (data-centric, non-strict) evaluation model where recursive definitions don't result in recursive calls, so traditional TCO isn't as relevant. Recursion is used very heavily in Haskell—which has no first-class looping constructs—but the resulting programs generally do not require large stacks. It's not unusual to be able to run even large Haskell programs with a 1 KiB maximum stack size (+RTS -K1k). Space leaks are possible, of course, but they take place in the heap.
The point of the article is that while the Common Lisp standard is not really focused on functional programming, nothing prevents the implementions (and libraries) of today to be so.
Myths are widely held to be fundamental about a thing (but not necessarily true). Red herrings, on the other hand, are a distraction that draws your attention away from the actual matter at hand.
So that said, I think what you are trying to say is that contrary to the poster, you believe focusing on these myths is a red herring and there are other fundamental reasons that lisp popularity remains what it is. I suspect because you did not articulate the difference, your discussion got sidetracked into semantics because it sounded like you were saying they were "the same thing"
"Machine learning" is a perfect example of how out-of-touch that claim is considering that Lisp was the language for AI research until very very recently.
> mcclim – really?
I guess you didn't see the link right below it with the dozens of other bindings to other popular GUI toolkits.
https://stackoverflow.com/questions/50645059/is-kubectl-port...
And socat is mentioned in:
https://github.com/kubernetes/kubectl/blob/master/pkg/cmd/po...
I didn't actually manage to find out what or how socat is called - but I'm also on mobile, so it's a little convoluted to browse source code.
Anyone know where the call to socat happens, after kubectl port-forward?
Someone else mentioned that kubectl port-forward also adds these for you, so if you are not in a private cluster, it may be a better option.
CAPI supports at least Mac, Windows, and Linux.
BTW, it just took me about 40 minutes using CAPI to reproduce what took a couple of hours this morning doing the same UI in Swift and SwiftUI.
I understand your comment as a bit judgmental, if not rude. Please keep the tone friendly, it helps if we want to have constructive discussions. I'd be happy (as well as everyone around here I'm sure) to discuss what you find questionable.
Apart from that though I do think it's useful to have the occasional new posts showcasing rebuttals to a few "myths" that won't die, so I appreciated your post. Some gifs of editor interactivity (like in https://malisper.me/debugging-lisp-part-1-recompilation/) would make it better. Though ironically (one myth has been "it has no libraries") perhaps some people might be turned off by the mention of so many libraries ("ok such-and-such is there after all but I have to type a quickload, why can't it come without any work?" -- programmers are often lazy in the worst ways). Before using every utility library under the sun and then some, I'd encourage people to get to know the base language... It's typically quite sufficient for many tasks. But hey, libraries are cool too.
Two other persistent data structure libs that came up in a thread the other day (though fset seems to be the most popular): https://github.com/danshapero/cl-hamt/ and https://github.com/ndantam/sycamore
Regarding the visual aspect: Very nice link, thanks for sharing! I was also thinking of showing off the macro-stepper.
Regarding the lack of libraries: well, Quicklisp is strong of some 1500 libs, which is rather poor compared to most popular languages out there. So yes, it goes both ways!
I'll check out the other libs, thanks!
Languages like Python, JS and so on are fantastic at plumbing. But when it comes to modelling data-structures, manipulation and algorithms then they are far less expressive and productive than Clojure from my experience.
My experience differs. It is hard to find a data structure that cannot be modeled with Python's native data structures with only a constant performance penalty. In fact I believe that I have only seen one, and I don't remember what it is.
I agree with the points that suck about Clojure. To them I must add, "It is hard to hire people who already know the ecosystem." For real businesses this is a non-trivial issue.
As for the purported payoff, I'm glad that you enjoy your editor. For all practical purposes the mechanics of editing code are not a pain point. Besides, I can do the same with vi in any language with braces. (Selecting a block in Python is slightly harder but again has not proven to be a significant challenge.) Designing code as a series of descriptive expressions has more to do with learning to program well than any particular programming language. The third point is not a significant pain point. And as for the fourth point, simply using the hosting language gives you the same library ecosystems without having to understand the wiring and translate all of the examples you find online.
The result is that if you want JS (I only occasionally do), you can just write in JS (or TypeScript). If you want Java (I don't), you can just write in Java.
However Clojure gives you the perfect tool that is customized to exactly how you have to work. Bully for you. My experience with evaluating such environments and tools is that for me to master it is a large investment of time and energy, at the end of which I might or might not (probably not) have much payoff.
(This doesn't stop me from learning said tools. There are good ideas there that I can adopt. But I have not found them particularly useful for me in practice.)
My claim isn't that they don't exist or that one cannot implement them in for example Python (which is very false). I was imprecise there. What I was trying to say is that there is a higher degree of uniformity and generality to do the same in Clojure, which leads to more cleanliness and easier, more fluent abstractions. Clojure code (and I assume similar to true with other Lisps) feels more compatible with itself so to speak. Syntax plays a role here but also the fact that Clojure is designed to be primarily an FP language.
> I agree with the points that suck about Clojure. To them I must add, "It is hard to hire people who already know the ecosystem." For real businesses this is a non-trivial issue. (...)
I fully agree with this and with the rest of your points. I think there might also be factor of personal taste or way of thinking.
I do know from personal experience that programming in a familiar environment is significantly more productive than programming in an unfamiliar one. And that many, many programmers have mistaken their personal productivity for the productiveness of the environment.
This is one of the causes of a lot of "holy wars". Because everything from indentation style, to operating system, to text editor, to language feels critical. And indeed is...for the programmer who hasn't learned how to switch between these things.
If someone had an oracle that could give us the answer, I would happily take an even money bet that something of this at least contributes to how much you prefer Clojure.
I also agree with you that there is a lot of personal taste and way of thinking involved. And furthermore that our opinions usually match whatever environment(s) we imprinted on. (In the interests of full disclosure, my background is math, Perl and relational databases.)
For example, Clojure's built-in concurrency primitives like Atoms and Refs are (in my opinion) substantially easier to work with than dealing with "synchronized" or anything like that in Java, and if you're willing to import a library for it, core.async gives you very pleasant Go-like concurrency with channels as well. This allows for a very reactive design that also happens to be safe and relatively easy to debug.
There's also the ability to use macros, which, depending on the use-case, can often replace the need for reflection that you occasionally might have with Java.
Last, it's relatively easy to hook into a running Clojure session with nrepl, meaning that you can use some very-nice runtime debugging and updating without downtime if you're clever.
---------
These features are in most functional languages in some capacity, but most functional languages don't have good interop with Java; the sales pitch for Clojure is comparatively easy: "We should use this language with all these neat features, but where we can use our current stuff just fine".
"But after a certain very short prototyping period, I end up fighting Python’s terrible deployment, terrible efficiency, and slapdash language implementation when trying to build something robust."
How many programmers would see Python's language implementation as "slapdash" or care? The fact that you both see it that way and care speaks volumes towards you fitting my characterization of the kind of person who uses a minority language.
The other points likewise. Python's inefficiency is well-known and acceptable for most use cases. Where you need better, it lets you escape down to more efficient languages in libraries. Some good examples of this include pandas and various machine learning libraries.
And as for deployment, it is more of an organizational challenge than technical one. The strategy of taking half your servers offline, deploying, then swapping and repeating worked fine 20 years ago. It works fine today. (You might have to do some ceremony with kubernetes. But the strategy remains conceptually similar.) This is only a significant pain point when there are other problems that interfere.
The fact that these things bother you, and bother you enough to rewrite wheels that already exist, speaks volumes.
The fact that people who agree with you on this then can't agree on whether strong typing is worthwhile (like Haskell), or continuations are good (like Scheme) or whether performance is critical (like Common Lisp) or whether access to the JVM toolset matters (like Clojure) causes you to not create a critical mass around a better toolset. And so the situation remains. You all agree that the popular languages are so terrible that you avoid them. But then can't agree on which unpopular language everyone should use instead, or why.
All lispers agree everyone should use lisp. Any lisper that doesnt like lisp is welcome modify that lisp to make it something the lisper does like.
Lispers wind up with a myriad of incompatible Lisps, and a solution written for one may or may not be portable to the next. Sure, you can call them all Lisp. But Common Lisp, Emacs Lisp, Clojure, and Racket are, syntactic similarities notwithstanding, actually different languages. A library written for one will generally not work for the others.
* various platforms don't support TCO. It was designed such that it can be implemented by a simple non-TCO interpreter, transpiled to a non-TCO C compiler, compiled to a non-TCO Lisp Machine CPU, or to a non-TCO virtual machine (like the JVM). Many languages don't support TCO on the JVM and may only implement explicit tail recursion or have a compiler detecting tail recursion - which is far from supporting TCO. Thus a portable conforming Common Lisp program will run in ABCL (a full Common Lisp implementation on top of the JVM) - because it will not depend on TCO to not blow up the stack, or similar.
* another reason was that Lisp has a bunch of features with don't work that well with TCO. For example Lisp always supported various dynamic scoping constructs and made extensive use of those - something which Scheme in its core does not, but provides via libraries or language extensions. Using dynamic scoping constructs makes TCO more difficult, may require a different language design, etc.
As for language features like dynamic scoping, I would say that a function call which needs to be followed by some cleanup activity is not in tail position, so TCO would not apply. The cleanup code could be in tail position, however, if implemented as a function call. In Common Lisp most forms of dynamic scoping or unwinding are explicit anyway, so this shouldn't come as a surprise as it might in languages like C++ and Rust where destructors are called implicitly when objects go out of scope.
If one compiles code or loads compiled code to something like SBCL and dumps an image, you can both run it directly from that image AND have still have the development tools included.
Some Common Lisp implementations also have delivery tools which create applications without development tools. Some applications though include the development tools, because they are thought to be extended by the user. For example a CL-based CAD system might exposed Common Lisp as the scripting language for CAD extensions.
But you didn't say anything about the original question. Have you seen Common Lisp code deployed as an executable or as source code?
ECL and CLISP also use compilers. ECL usually compiles to C, which then is compiled to native code via a C compiler. CLISP has its own virtual machine, which is the target of its compiler. Don't know what ABCL does, would have to look it up - but in the end it usually would execute JVM code, which gets compiled by its JIT compiler.
Question of code deployment. I have seen deployment as an executable, deployment as an executable with also loads compiled code (for patches) and deployment as source code. Especially commercial applications and/or end-user applications tend to deploy as executable. For example ScoreCloud is a LispWorks application as an executable and can be bought on Apple's Mac app store.
My Lisp Machine at home boots into a Lisp image which contains a whole operating system as compiled Lisp code. That's exotic, but many other Lisp development environments already come as compiled Lisp executable. For example on the Mac and on ARM64/Linux I use LispWorks. On the Mac it is the usual Mac application bundle, where the executable already contains much of the IDE and then loads some code on demand from compile code files.
And ECL does ultimately compile down to machine code if you include "compile the resulting C output" in the compilation process, though this is optional.
I'd use Electron with Ceramic, not the GUI bindings. Or maybe Nuklear. But I would build personal projects with them.
ML: yeah, yeah. Just to show there have been work on it, that (of course) the platform is capable.
Anyway, I completely agree that Common Lisp is capable, my point also isn't that it's impossible to do things like GUIs, or write something cool and successful with it – just that the eco-system is not general-purpose language quality. If you want to make something that's successful commercially or as a well-adopted open-source project you have to be much more careful what problems you tackle than with a genuinely general purpose language.
Blog posts that claim otherwise really rub me the wrong way; I think it's possible to present CL in a favorable light without distorting reality and no one is gonna thank you later for joining the rank of people who missed out career-wise for making non-viable technology choices.
= not top-notch in all areas? Yeah, there's room for improvement. Buuut, given CL's features and stability, I will very much consider it for a commercial project in the future instead of, say, Python. I'd reject Python, for sure, for GUIs or web dev (except a small website maybe where admins would use the Django admin). Now I must find another language, and it's difficult. Many things drive me back to CL, including its state of libraries, actually pretty good compared to other languages, newer or older. No kidding.
> If you want to make something that's successful commercially or as a well-adopted open-source project
maybe. Reading you, it seems one should avoid using CL altogether. But there's a lot of room for a successful use of CL before falling into these categories: one-off scripts, quick GUIs at work, personal tools? Commercial websites (with adequate requirements), like I did? yes, it's possible to use CL for that. Yes, it is possible to do web development for a job with CL (what kind of sites? We have to ask the few redditors that do it). My point is that CL is general-purpose enough so that it could be more used in the wild (because it is used in the wild).
Anyways. Just trying to be positive :)
Yeah, I'm not arguing against that at all. I just hate articles which encourage people towards doomed efforts rather than building out existing areas of strengths or address things that are real obstacles towards using them. I'd like CL to be around for at least long enough till all it's important ideas have made it into other programming languages ;)
For example, one core strength of CL is that it's the about the only really interactive/malleable/live language that can generate pretty performant code, and for sbcl you can hook deeply into the machine code generation as well. Luajit is nicer in some ways (e.g. much smaller footprint in every possible way) but CL has a much more sophisticated interactive development experience (restarts, powerful debugging, pretty-printing etc. etc.). So tooling for automation of binary rewriting, refactoring and hardening like GrammarTech seems to be working on (see https://github.com/GrammaTech?utf8=&q=&type=&language=common...) seems like a much more promising area to work on with common lisp to me than e.g. generic web development.
I found it hugely fun--so much so that it took over my brain. I've been primarily a Lisp programmer ever since.
Coral Common Lisp offered an extremely easy way in to Lisp programming that was also tons of fun to work with, and an industrial-strength development environment for the full panoply of Mac applications. At AAAI in Detroit in 1989 I mentioned to someone that I was working in Coral Common Lisp and a passer by quipped, "Oh, the good Lisp."
It ran just fine on a 1 MB Mac Plus. You could start it up with a double-click. It launched with a Listener window and a blank Lisp source file (unless you launched it by double-clicking an existing Lisp file with some code in it.) You could save an image, copy the image file to a different machine, launch it, and you would see the same windows in the same state.
Creating a fully-functional Mac window looked like:
(make-instance 'window)
(save-application "Foobar")
The built-in editor, Fred, was a species of Emacs, with the Lisp-oriented features that you would expect, but you'd never have to know that if you didn't want to, because it was also a fully-featured Tesler-style modeless editor at the same time.
The stepper, inspector, apropos, and other development niceties all had Mac graphic interfaces that made them easy to discover and tinker with. They were all written in Lisp, so you could use the Lisp reflection tools to crack them open and poke around inside to see how they worked.
Coral Common Lisp made it easy to get started using Lisp to build apps--about as easy at it can possibly be. At the same time, though, it imposed no arbitrary limits on what you could do with it. When I first met Dave Vronay he was working on the GUI for SK8, and gushed about how easy it was to write window-definition resources (WDEFs) in assembly language using CCL's LAP subsystem. He was an experienced arcade-game hacker, well versed in assembler, but Coral Common Lisp was his first fully-interactive assembler, able to define and integrate new assembly-language code while the program under development was running.
The CCL of today, Clozure Common Lisp, is still a great Lisp, but it isn't as easy or as inviting for newbies. It doesn't have the same graphical environment or the tight integration with the underlying system.
In part that's because when the Clozure Common Lisp project was created (under the name "OpenMCL"), its creators had the rights to the compiler and the Lisp runtime, but not to the Macintosh graphical environment. In part it's because modern macOS is quite a bit more complex than Macintosh System 9 and its predecessors, and development systems have a lot of additional hoops to jump through nowadays to do the kinds of things that MCL was able to do.
If a modern MCL existed, I expect a lot more people would find Lisp a lot easier and more fun to get started with. I think it's perfectly possible to create such a Lisp, but it would be a heck of a lot of work.
MCL was a whole new world then. First with Object Lisp and later with CLOS. I used MCL before it was owned by Apple - at one time it was called MACL (from a market agreement with Franz, IIRC). But later LispWorks was another step up, because it was a big grown up full extended Common Lisp with everything from the commercial UNIX workstation Lisps (like Allegro CL, Lucid CL, LispWorks): it suddenly ran on small and simple to use Apple or Windows laptops, plus it had a Cocoa port.
But, as you describe, the simplicity and integration of MCL into the early MacOS was a lot of fun and there was a lot of tinkering by users. Some friends were still using MCL years after it was obsolete...
Personally, I do think Lisp is fun, and was having fun pretty much from the start. But, with how many people I've watched just bounce off of it, I simply can't bring myself to imagine that my experience is anything but unusual. Nor am I inclined to console myself with self-congratulatory stories about Blub.
Just personal preference and enjoy programming in it. Programmers give too much importance to programming languages.
Clojure can be made to run very fast (it compiles to the same bytecode as Java after all) for the hot paths at the cost at writing less idiomatic Clojure.
A complex application written with CLOS will generally not be portable to Emacs Lisp. An application written in Emacs Lisp that takes advantage of dynamic scope won't be easy to port to Clojure. A library in Clojure that wraps something in the JVM ecosystem won't work in Racket. And something written in Racket using continuations or Pict generally won't be easy to port to Common Lisp.
And even with closer Lisps than that, I have enough experience of "that should be easy to port" then blowing up over minor details to be suspicious. Sometimes it will be 10 minutes. Sometimes it will be several weeks and incomplete even then. The latter happens often enough that, the developer's confidence notwithstanding, the average is waaaay longer than the estimated 10 minutes. (The phenomena of tasks being rarely faster than we thought, but roadblocks can be unlimited in extent is generally true in programming. This is one of the reasons why our time estimates tend to be systemically low.)
Wow, super impressive. You didn't post this on reddit yet :)
> CLISP has its own virtual machine, which is the target of its compiler
Then CPython is also a compiler? and Lua?
Originally interpreters were implementations which execute source code - which in Lisp is widely available. For example SBCL for a long time only had a native code AOT compiler, but now also includes s-expressions.
Famous is also the Lisp interpreter in Lisp from McCarthy, where he defined the core language in itself. See the paper from Paul Graham about that: http://languagelog.ldc.upenn.edu/myl/llog/jmc.pdf That's a very primitive Lisp interpreter - many real ones are implemented in C, Assembler...
Just be aware, that in the Lisp world 'Interpreter' means something very specific: an interpreter of the source language.
I've done this. I've seen other people do this. Sometimes it works. Sometimes it doesn't. Sometimes it works fine for the demo but then has weird failure modes that you trip over in production months later.
It may be the right thing to do. It may be feasible. But a general "10 minute job" estimate is laughable.
Later on, John Ulrich, who owned Lightship Software, the publisher of MacScheme, came to work at Apple and became a friend and colleague. I continued to use MacScheme off and on for years.
I’ve happily paid for Lispworks licenses many times now, but even Lispworks isn’t a good solution for everything I want to use Lisp for. A modern version of CCL’s graphical environment would still be a nicer place for newbies to start. You could even use Lispworks to build a product like that, if the license didn’t prevent it. (Lispworks doesn’t allow their product to be used to make Lisp development systems, which is understandable. They’re a small company who don’t want to put themselves out of business by enabling a customer to give their Crown Jewels away.)
In former times there was a market for add-one GUI or IDE tools. Lucid CL made money from licensing their Lisp to other companies, which then resold those with tools added on (like the SUN Common Lisp IDE). The Action! GUI designer was available for MCL. CLIM was an add-on product. The big Expert System development environments were available as add-on products to Lisp systems.
For a commercial customer something like LispWorks or Allegro CL might still be worth it - they are technically very stable platforms.
I wrote the original Alpaca in CCL, using its Cocoa interface. I used it to write a book, and a Japanese company approached me about licensing it for a product, but I fell ill before I could do anything more with it. By the time I recovered enough to look at it again, it had bitrotted. The macOS interfaces had changed out from under it.
I imagine I could fix it up and get it going again, but I'd really like to make a version of Alpaca that works on Linux and Windows as well as macOS. The obstacle is that I need a rich-text engine that can do a good job with page layout, and I don't know whether there's a suitable one, or which Lisp (that isn't Lispworks) would be best to use with it. Qt maybe? Perhaps with ECL? I guess I'd rather use CCL or SBCL. Anybody know of a text-and-page-layout engine or collection of libraries that will work well on macOS, Linux, and Windows with CCL or SBCL?