We deploy distributed, multi-language, centrally Lisp/SBCL servers as well. A few specifics that I'd point out:

Many of SBCL's optimizations are fine grained selectable, using internal SB-* declarations. I know I was at least able to turn off all optimizations for debug/disasm clarity, while specifically enabling tail recursion so that our main loop wouldn't blow up the stack in that build configuration. These aren't in the main documentation; I asked in the #sbcl IRC channel on FreeNode.

You can directly set the size of the nursery with sb-ext:bytes-consed-between-gcs, as opposed to overprovisioning the heap to influence the nursery size. While we've run in the 8-24GB heap ranges depending on deployment, a minimum nursery size of 1GB seems to give us the best performance as well. We're looking at much larger heap sizes now, so who knows what will work best.

While we haven't hit heap exhaustion conditions during compilation, we did hit multi-minute compilation lags for large macros (18,000 LoC from a first-level expansion). That was a reported performance bug in SBCL and has been fixed a while back. Since the Debian upstream for SBCL lags the official releases quite a bit, it's always a manual job to fetch the latest versions, but quite worth it.

Great read, and really familiar. :-)

Common Lisp's macros and grammar go together like bread and butter. A grammar module in the app I built [1] uses macros to generate huge amounts of repetitive code.

[1] https://github.com/tshatrov/ichiran/blob/master/dict-grammar...

I wonder if they're still hiring Lispers. I once passed on the opportunity to work in their Kiev office, but I might give it a shot again.

Great article, and good reminder on using trace. Every time I rediscover trace, I can't remember how I ever forgot to use it in the first place for most of my problems.

I used CL in a production environment a while back for a threaded queue worker and nowadays as the app server for my turtl project, and I still have yet to run into problems. It seems like you guys managed to push the boundaries and find workable solutions, which is really great.

Thanks for the writeup!

Slightly off-topic, but does anybody know of a kind of "LISP Challange" set? I recently started the Matasano challenges[0] and I found them really well-suited to my style of learning (learning by doing and expanding by reading relevant material, enabled by my own internal motivation). Is there anything like that that has a relatively small set of condensed yet rich challenges that demonstrate key elements from LISPy functional programming? I read some of SICP but reading long form really puts a damper on my motivation/excitement. Also there were a lot of exercises (with a lot of overlap in concepts) so I didn't know what to do and what not to do, since I wasn't about to do every single one. Any pointers would be much appreciated!

[0] http://cryptopals.com

Awesome stuff. Articles like this are what we Lispers/Schemers need to show that our languages can be used for "real work"(tm).

One of the things I would have liked to see on the article is how do they handle the deployment itself. Do they build an executable with build app? To they used sb-daemon? An home-grown solution using sb-posix:fork?

Is it worthwhile to explore Clojure for web-dev seriously or more as a toy?

Apparently they use "JVM languages", JavaScript, Python, Go, Lisp and Erlang in production.

I may be in the minority, but that would drive me mad. I assume they're not routinely jumping between those stacks multiple times a day, but even so is there really that much benefit that it's worth keeping track of how to do things in that many different environments?

Very informative. Thank you.

Anyone here has any experience with the GCs of Allegro or LispWorks or any other commercial Lisp implementations?

Aw, now they have disclosed their secret weapon! [1]

[http://www.paulgraham.com/avg.html]

> but we value choice and freedom over rules and processes.

Which I exactly why I feel Lisp doesn't see much use elsewhere :(

Good lord, I would go insane if I ran into a bug like this:

"We've built an esoteric application (even by Lisp standards), and in the process have hit some limits of our platform. One unexpected thing was heap exhaustion during compilation. We rely heavily on macros, and some of the largest ones expand into thousands of lines of low-level code. It turned out that SBCL compiler implements a lot of optimizations that allow us to enjoy quite fast generated code, but some of which require exponential time and memory resources. "

I love it :0) I tried grammarly, and typed in a remembered poem. It informed me that it had detected significant plagiarism.

Edit: It's still not advice I would pay for, though.

Thanks, great post and lot of useful references.

This was fantastic. Thank you.

Wow. I can't ever remember reading about a consumer-facing app using Common Lisp. Ever.

Once you go deep enough you are fucked no matter what language you choose. Might as well pick one that doesn't beat you up too much.

Why not racket?

From the top of my head..:

The Praetorian Bootcamp challenges are similar to the Matasano ones: https://www.praetorian.com/challenges/

Lisp Koans: https://github.com/google/lisp-koans

Project Euler: https://projecteuler.net/

Exercism: http://exercism.io/

99 Lisp Problems: http://www.ic.unicamp.br/~meidanis/courses/mc336/2006s2/func...

Hi, I don't know if it fits what you want. But there is something like that for clojure[0] . I liked the language a lot. I didn't have the chance to use it in production yet. Another option is hacker rank with challenges [1] [0]https://www.4clojure.com/ [1]https://www.hackerrank.com/

It's extremely serious.

The deployment is the very well understood JVM and standard web servers. There's a common HTTP middleware framework (Ring), lots of choices for HTML generation and ClojureScript allows some code-sharing with your client side (compiles to JS).

On the back-end, you could use any Java library. On the front-end, you could use any JS framework (e.g. see https://github.com/omcljs/om)

I think it is one of the best stacks out there for web.

Check out this:

https://github.com/bhauman/flappy-bird-demo

It uses figwheel for the dynamic changes of state when you change to code, and it renders it without reload ala Bret Victor style. The first time I saw it I was amazed. It speeds up prototyping so much.

The out of the box performance is decent as well, Ring and Hiccup is pretty lean but you can go for more heavy frameworks (I don't have any experience with those).

I personally use Reagent + Ring, found it easy to use and get productive in a day.

If you haven't used LISP or any homoiconic language before it might look little weird at first but I found it easy to explain to people.

About the toy part, what is your definition of toy?

Seriously. I have my misgivings with Clojure in comparison to other Lisps, but it's a serious programming language.

I've been using Clojure in production systems for 5 years now. Average of 15req/s over 5 years - I'd say that's pretty serious.

I vote that it's worthwhile as well. We've been using Clojure full-time at ReadyForZero for over 3 years now (for web-dev as well as other tasks), and are very happy with our choice.

Is there a Clojure/Clojurescript tutorial similar to the Hartl Rails tutorial? I have amassed quite a few Clojure books, but find the easiest way to learn is to build.

I think SICP states it best:

"In our study of program design, we have seen that expert programmers control the complexity of their designs with the same general techniques used by designers of all complex systems. They combine primitive elements to form compound objects, they abstract compound objects to form higher-level building blocks, and they preserve modularity by adopting appropriate large-scale views of system structure. In illustrating these techniques, we have used Lisp as a language for describing processes and for constructing computational data objects and processes to model complex phenomena in the real world. However, as we confront increasingly complex problems, we will find that Lisp, or indeed any fixed programming language, is not sufficient for our needs. We must constantly turn to new languages in order to express our ideas more effectively."

At a certain level of software design tying together different programming languages that are each the right tool for their job becomes just another type of programming. I currently do data science work, but even then in a given week I typically use R, Python, Lua and Java (and often Scheme in the evenings for fun). Trying to make any one of those languages do something the other is much better at is a phenomenal waste of time.

On the system level, once prototyping ends, if there's something that Java does phenomenally better than R, but we need both, that implies that you have two parts of the system different enough that they shouldn't be tightly coupled anyway. If you write a deep learning algorithm in Lua, but want to do some statisitical analysis on the results of that in R, it's good to force these things to be separated because if in 5 years you find a better model for the Lua part (maybe some better algorithm in Julia) you want to be able to swap it out anyway.

While a Single Language to Rule Them All would be cool, I ultimately prefer using the "best" language for the job based on specific requirements.

The "best" might change over time, too.

It can be a headache to manage massively-polyglot environments. At the same time, it's also pretty great for a variety of reasons. I mean, we regularly use different data stores, messaging solutions, frameworks, etc. and I don't see why languages shouldn't also be up for shuffling.

As for me, adaptability is one of the important traits of a senior engineer. Surely, you don't have to be an expert in every platform, but you also shouldn't go mad if you need to do some work outside of your comfort zone occasionally. Besides, every language has its strong and weak points: if you're putting arbitrary limits here, you're just limiting what you can do and the people you're going to get in a team. At Grammarly, we always erred on the side of more freedom and it worked not so bad for us so far. Although, there are different companies, each with a unique story...

To me: Javascript, Python, and Go aren't all that different.

You must know Javascript in this day and age given it's de facto presence on the web.

Python is my go to language, especially for mathematical analysis. I can do everything in Python that I used to need Matlab for. From my point of view, pick your favorite modern scripting language and run, the differences really are the libraries, not the languages.

I have used Go, but Go just doesn't do it for me. It doesn't offer me anything I can't get, better, in another language especially if I can choose among Python (smaller headspace), Erlang (way better concurrency) or Lisp (way better abtraction power).

Erlang is my go to language for concurrency. Once I architect it in Erlang, I probably understand the problem.

Lisp is useful when my problem requires powerful abstraction. Otherwise, it gets in the way because people can't resist using that power. Clojure has changed my opinion on this quite a bit, but I don't yet have a big project that fits in it's space quite yet.

Since "proper service encapsulation" is mentioned, it may be that each team uses whatever they like, and as long as your component speaks http you don't have to look at what other components are doing.

Actually these errors are relatively easy to spot and and often there are solutions for that. Sometimes a compiler might need to be hacked on. The good thing: the compiler is written in Lisp and debugging is possible.

The more nasty errors are lurking for example in the GC... there we move into C and assembler land...

Most platforms have nasty errors. With popular platforms one can hope that many of these have be found and somebody has fixed them already. With language/runtimes which are no so widespread in production one is more likely to find these problems oneself. Especially in more complex runtimes.

Yeah, but that's something you need to be prepared for. Such bugs happen in literally every platform (for instance, I had similar trouble with the JVM). So the question is not how to avoid them, but how to cope. Usually, there are 2 ways:

- workaround

- investigation (and in this case, if you're on a closed platform you're busted)

It'd be a Hell of a lot more interesting than the millionth time looking into a bug caused by invalid input into a non-validated form in an ancient CRUD intranet app.

You can write C++ template metaprogramming code that will send your compiler into coma, if you're not careful. Code that pushes the boundaries is bound to push through them occasionally.

You know, starting a paragraph by "We've built an esoteric application", would you expect what followed to be a simple bug?

Write an "esoteric" app and you'll start hitting the limits of your platform.

I dont really understand the downvotes... I agree with you would go nuts.

My two cents, if your macro is expanding to thousands of lines of code, your doing something wrong I think. I would expect Macros to expand out to a few lines of code which might have function calls that themselves may contain however many lines of code... but expanding to thousands of lines INLINE via macros seems wrong.

Yeah it's fairly easy to fix this (and you can even do it without unwinding the stack!). There are tons of sbcl global variables which you can tweak on-the-fly.