The Problem with Threads (2006) [pdf](www2.eecs.berkeley.edu) |
The Problem with Threads (2006) [pdf](www2.eecs.berkeley.edu) |
What are the alternative paradigms that have actually become common use? Coroutines, async/await, that's what I hear about online but what are others? I've seen people who touted zmq-communicating-processes with standard patterns as the solution to all problems, and I'm happy not to have to maintain the results.
Have we effectively “solved” the concurrency problem, and if so what's left as an exercise for the future?
Coroutines don't solve the parallelism part, because they're concurrent but exclusive.
Async/await as implemented in JavaScript doesn't solve the parallelism part either for the same reason, and async/await as implemented in C# has exactly the same problem as threads.
There are many ideas for how to solve the problem - but I think anyone who is honest will tell you none are a perfect solution to all situations where you want parallelism or concurrency.
For example to use zmq-communicating-processes effectively you need a problem where you can divide the data or tasks cleanly a-priori. We simply don't have the mathematical understanding of how to do that to some important algorithms that people really need to run in parallel today, such as triangulation or mesh refinement.
We probably need some radical new idea, or maybe it's looking increasingly like only a mix of ideas will work.
The code for each actor is usually pretty small and easy to reason about. However, emergent behavior of the system, and ordering between messages from multiple actors can become tricky. Also, exposure to this idea long term will warp your mind :)
Most parallel (not concurrent) problems map well to the data parallel model. Even Make is basically a data parallel API with read-only constant data, just with a more complex dependency graph.
Which boils down to problems created by the POSIX implementation with condvars, mutex and semaphores. No lockless and waitfree data structures.
With threads there are also minor hidden contants: limited stack size, high cost of context switches. And random order of evaluation.
Lockless threading semantics needs to know ownership, copy or ref and relationship to be able to fix these problems. I only know a few not well-known languages who actually did a solve these problems.
"They make programs absurdly nondeterministic, and rely on programming style to constrain that nondeterminism to achieve deterministic aims."
You can't write an infinite number of test cases for all those interleavings, and it requires hard thought to suss out where any problems might lie.
"Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
https://www.youtube.com/watch?v=4fFDFbi3toc
They wrote an interesting Actor DSL that compiles to C++ and is completely deterministic, and they torture this deterministic engine with generated test cases on a cluster every night.
I guess you could say that the whole cluster is necessarily non-deterministic, but an individual node is deterministic, given an ordering of the messages it receives.
Sometimes it's hard to tell when a resource is shared, but that has more to do with not knowing how the code works than it does with multi-threading.
With respect, this sort of thing works a lot better for small codebases where you're the only one working on it. Multithreading when you can't contain the entire relevant codebase in your brain is where the real challenge is.
Then you have deadlocks.
My favorite one is where adding debug traces causes the heisenbug to disappear because the printf() inserted a memory fence somewhere deep in the logging library.
Nothing like debugging via atomics.
> To offer a third analogy, a folk definition of insanity is to do the same thing over and over again and to expect the results to be different. By this definition, we in fact require that programmers of multithreaded systems be insane. Were they sane, they could not understand their programs.
I actually had this exact notion when implementing pthreads for a course. I noted to myself "Gee, I keep doing the same thing and every time I get a different result... I must be insane according to the definition"
You are right about the actor model being a low-level choice, but its a choice that has to be made since the whole system revolves around allowing/disallowing shared state.
And concurrency is even harder, especially with the ever-popular "tweak until it parses/compiles, then ship" approach, or when many people are working on the same section of code.
The Hewitt Actor approach reduces independencies dramatically, at some cost for certain algorithms, and with added clarity for many others. And it scales somewhat automatically beyond one machine which is a big win these days.
My $0.02.
So try not to do that. On the other hand, everything that happens with state within an actor is inherently non-racy, because an actor is sequential code and no other actor can mess with its state.
Something like fork-join is deterministic because results come in a fixed order.
And for generating SIMD from actors? Or handling irregularity efficiently? I feel like you’re making the ‘sufficiently clever compiler’ argument.
We cannot currently efficiently solve all parallelism problems in practice using actors, and we don’t know how we would be able to.
And I'm not arguing for a sufficiently clever compiler, just that you can express any concurrency with actors. You can definitely create a convention backed by actors that compiles into SIMD if you need it.
(Christoph von Praun)
If two actors do some work concurrently and when finished send a message to another actor, the order those messages arrive at the other actor, event a and event b, is not predetermined by the program. So it's a race condition.
actor a {
do some work;
send 'a' to x;
}
actor b {
do some work;
send 'b' to x;
}
actor x {
receive; <- has this received from 'a' or 'b'? Nobody knows. They've raced.
}
You can express any concurrency with actors, but we do not know how to do so as efficiently as with other concurrency models for parallelism. Someone might be able to implement it efficiently, but nobody has managed it yet, so we're still reliant on shared memory and other approaches concurrency.In both of them there is tons of parallelism but you can’t work out what work is separate (divide it) until you have started the work.
Imagine that you have a magical black box system that can triangulate A and B.
Would this not help you to triangulate X? I can hardly believe it wouldn't. (Again, perhaps in pathological cases yes, or if a near-optimal solution is not good enough).
That's why I think other models of concurrency, such as the fork-join model, where the equivalent of 'messages' have to arrive in a deterministic order and so there are no race conditions, are safer.
actor A {
receive half an array
sort it
send it to C
}
actor B {
receive half an array
sort it
send it to C
}
actor C {
(a, b) = divide input array into halves
send a to A
send b to B
receive a'
receive b'
merge a', b'
}
They send to A, send to B, and then think they're going to receive from A first, because they sent first, but B could finish first instead. Sometimes their program works, sometimes it doesn't.Yeah it's their fault, but the model hasn't helped them not make the bug and worse they may never see the bug until they deploy into production.
If we used a fork-join model, they could not have made this mistake, and even if they did make some kind of mistake, at least they'd see it every time they ran their program.
(a, b) = divide input array into halves
fork {
sort a
}
b' = sort b
a' = join
return a' + b'If you instead did
fork {
sort a
}
fork {
sort b
}
a' = join
b' = join
you would have the same problem as in Erlang. or you could have actor C sort B inside the actor between send a to A and receive a' and you would also have an implicit ordering.In this case, merge sort could work with either order if a stable sort isn't required, or if the sort key is the whole element.
If it matters, this is easy to defend against, you just send a tag (a ref in Erlang would be perfect for this case, if the merge happened in a fourth actor, a numeric indication of ordering would be more useful) in the message to actors A and B, and use that to enforce an ordering when receiving the replies.
Ah but that's not how fork-join works - you fork multiple jobs, and then you must join them all at the same time - you can't join just one.
You have to do something like
(a, b) = joinThe model is that you can start a sequence of jobs to run in parallel, and then you have to wait for them all to finish. You get the results in the same order as the jobs you created. The order can't vary.
Think about a diamond shape - one job create two more jobs, and then they both send their results to the original job which cannot continue until all child jobs are finished.
> because that seems like the most useful/basic interface
Yes useful and basic, but the problem is it makes it easy to cause race conditions, which is where this thread started! You think you'll get some some thread being ready first so you write code assuming that without even thinking about it, and then once in a trillion you actually get the other result first. Yes, it's a programmer bug, but the point is because it's non-deterministic they may not notice until the one time it actually matters and someone dies.
> The model is that you can start a sequence of jobs to run in parallel, and then you have to wait for them all to finish.
Yes, that's the model.
> You get the results in the same order as the jobs you created.
No, you don't get the results in the same order. Jobs still finish in random order and store results before synchronization happens. Synchronization happens on join after that. And instead of relying on order you specify exactly from where you are getting the result of each individual job. So, if you have to specify that, why do you need an order then? Oh, you don't need it and you don't get to have it. It's not Erlang, where you can actually have a deterministic order and can wait for messages in any order you want, while it will reorder them for you.
You and I just don't see to be on the same page about what fork-join is, so we probably aren't going to agree on this.
> It's not Erlang, where you can actually have a deterministic order
Can, but my point is you can also not have a deterministic order, which is how Erlang programs can end up being racy, which is the problem with them if you are trying to solve the original problems of threads.
These kind of techniques are key to using parallelism to reduce latency. Always having to wait for everyone to finish at each step makes for a lot of waiting.
And lack of order of messages is still not a race condition.