Back in the 90s. I was part of a team working R&D project on flexible manufacturing. One of the central concepts was the use of buffers (also called decouplers) between manufacturing stations. These buffers were the space needed to place not finished products while they waited for the next station. Typically a conveyor belt. The main service is that both stations could run at slightly different rhythm, and they would not be causing problems on the other one. Either starving it or overwhelming it. Queues are pretty similar. You can have a consumer and a producer working and the queue in between prevents them from being blocked by each other. Thus I now see the main role of queues as decouplers.

Once you've read this, pick up Harchol–Balter's book Performance Modeling and Design of Computer Systems: Queueing Theory in Action. It's a really good introduction to the depth of this topic, and you'll come out with superpowers you didn't have before.

Similar things apply to buffered channels in Go: buffering frequently hides deadlocks, etc, but you don't observe them until the buffer is full. So buffers generally should be kept either as zero or very small to be able to catch synchronisation issues early

Author clearly has a wealth of real experience, but I have trouble reconciling some of it to the “real world.”

Supposing that you have “too many” messages in your queue, commanding your frontend client to retry its transaction that would’ve added one more, instead of accepting and enqueuing one additional job, doesn’t seem to me to change much. Instead of creating a mess for whoever is in charge of those servers, the mess is created directly in view of the end user, who sees whatever you show them when their transaction is being retried.

Their point about the bottleneck being the real problem that must be addressed if loads are going to be sustained at such a high level is indisputable, though.

I think I would define the necessary rule as: the queue’s maximum size just needs to be greater than the spikes you expect, but that’s of course no insight, just a definition.

I have found queues to be incredibly valuable at solving situations where load has occasional spikes, but urgency of the jobs being done is low. For instance, every time a user views a piece of content you want to make sure that you increment a counter of how many times the content has been viewed, and you also want to touch the timestamp of when that user last did a thing. If that happens even two hours late, it’s probably gonna be fine. The thing that the queue pattern excels at in the realm of Web applications, especially, is allowing you to have an HTTP GET which can be served entirely by a Web worker that is only allowed to talk to a read replica, which allows extensive horizontal scale. Analytics and other incidentals can be handled async in background jobs (and indeed, in emergencies, load-shedding those ancillary things has barely any impact).

I recognize that all of this probably sounds “obvious” - but I have seen enough codebases that do synchronous writes during GET transactions that I would stop short of calling this “common knowledge.”

Or you could just play Factorio.

Two past submissions with notable discussions:

https://news.ycombinator.com/item?id=39041477 - 18 Jan 2024, 153 comments

https://news.ycombinator.com/item?id=8632043 - 19 Nov 2014, 60 comments

Reminds me of this other queuing theory blogpost classic https://medium.com/swlh/fifo-considered-harmful-793b76f98374

TLDR LIFO (stack, not queue) is often a better choice for many workloads, despite violating our sense of fairness.

"People misuse queues all the time. The most egregious case being to fix issues with slow apps, and consequently, with overload. But to say why, I'll need to take bits of talks and texts I have around the place, and content that I have written in more details about in Erlang in Anger."

This feels like an unfinished gripe—summarize the key point now rather than promising a future deep dive. Backing claims with a concise example would make the argument useful instead of vague.

> the queue’s maximum size just needs to be greater than the spikes you expect

There is one truth I have come to know, said by someone far wiser tha me: A queue is either empty or full. Which is to say a queue can either handle all the data coming in, or it can’t. When it can’t it will fill to capacity. This is a probabilistic thing, and you can only decide how many nines to plan for. And it’s worse than it looks at first, because queuing theory is very non intuitive with non linearities that make it very hard to reason about wothout having your nose rubbed in it.

So that means, that yes, you can keep doubling the size of your queue. And no, you can’t ever make it big enough to deal with a poisson distribution. And while you’re at it you will likely need to add workers. And you’re still back to capacity planning and deciding how much money to throw at the problem.

What you may be getting at, and what the article sort of failed at, is that queues are still super valuable for smoothing small spikes, or even large predictable ones. But a queue alone, without backpressure, or overflow will likely cause systems to fall over. Sometimes in ways that are hard to recover from, especially if you have some kind of microservices inspired architecture, where one thing going offline causes another queue elsewhere to fill. Or worse, bringing a failed service back online stresses another system causing it to fall offline. (not meant to be a dig on microservices by the way)

I agree with this - queues may not be the end all solution but it is a valuable tool in our kit.

And in the right situations, it can be enough.