Actors, Green Threads and CSP on the JVM(boundary.com) |
Actors, Green Threads and CSP on the JVM(boundary.com) |
That's not really the issue as far as I'm concerned. Message-passing concurrency actually allows people be productive writing concurrent code that has a hope of working -- that is the main advantage. Occasionally things go wrong. Occasionally you need more performance and have to drop down to a lower level. But I can easily write a web service running on the JVM that gets 1000 requests/s per CPU with a bit of message passing concurrency. That's the point.
Heck, I don't even like the actor model -- I much prefer CSP -- but I'd attack it for other reasons (like lack of type safety).
A while back I wrote a type-safe variant of actors (I called them Agents). It's done on top of scalaz-streaming which also gives better performance than akka (e.g., x.map(f).map(g) doesn't involve 2 trips back to the threadpool).
http://www.chrisstucchio.com/blog/2014/agents.html
Another neat benefit is separating state changes from effects. E.g., if you have a IncrementCounterAndUpdateRedis agent, you can test it's counter incrementing functionality without ever touching redis.
The issue isn't actor model on a virtual machine-- erlang runs on a VM-- the issue is that the actor model is insufficient. You need pretty much most of OTP to build reliable, stable, concurrent systems.
This is why goroutines, everything on the JVM, etc are not going to work.
The sad thing is, I think the primary reason people are not using Erlang is they are afraid of the syntax, but even that reason is obliterated by the existence of Elixir (an even better language on the erlang VM)
"This is why goroutines, everything on the JVM, etc are not going to work."
I've used Akka very successfully on the JVM for non-trivial clustered applications where performance is important, and I'm pretty sure the existence of most of google's infrastructure says that goroutines work at least a little bit.
Do you mean not work well? They won't gain traction?
1) Reliability. This is mostly going to be of the subscribing to other actors to make decisions when they die type. This is a complete replacement for try/catch, and probably isn't possible or desirable on the JVM due to mutability concerns. If you need and want this, Erlang is your bet.
2) Performance. This is probably the most common reason, and a true fiber system can hit this just fine on the JVM. So what if touching old code might cause degraded performance due to blocks, it's not unreasonable at all to say that high performance code will require some care. And if the fiber system warns you, the more the better. Quasar is your bet here.
3) Architectural niceness. Callbacks suck, and we all know it. CSP channels can be seen as a nicer way to structure the flow of a large asynchronous system. In this context I think core.async tends to be the best, because of it's support for transducers and Javascript support. Although Quasar/Pulsar would not be a bad second choice for this because they work outside of a go macro, assuming you only need them on the backend.
That is how most "frameworks" or those that try to copy this aspect from Erlang see it. But the trick is in Erlang this actor pattern is used for fault-tolerance just as much. That was the goal right alongside concurrency initially. (The third one was priority of low latency responses, I believe).
That fault tolerance is harder to copy and that is why most libraries and frameworks give up and copy the "class object+thread+queue" and call it "We have a fast Erlang now".
The closest to get to something like Erlang fault tolerance wise is to use OS processes and IPC (via ZMQ), with some serialization. But it would be hard to run 2M of those on a reasonable machine.
Plus Erlang is not just the language (which is rather small and simple) but the whole set of helper libraries and tools. Including a distributed database, distributed application controllers. Support for rpc and so on. Supervisor patterns you can use etc.
The article mentions bytecode weaving, but dismisses it with very wavery justifications. Bytecode manipulation tools are a successful part of the jvm ecosystem. Frankly, they're part of why I consider the jvm ecosystem so successful: bytecode manipulation has allowed things like:
- third party tree-shakers/minifiers and obfuscators (i.e. proguard)
- cross compilers (i.e. robovm)
- concurrency libraries that DO have real green threads and continuations (i.e. Kilim, Quasar, and others)
- code coverage and complexity analysis tooling (i.e. jacoco)
- scala
- clojure
- groovy
- kotlin
- [... more languages ...]
There are two critical points about the above:
- All of these tools were built without direct cooperation with the compiler and core tool chain. That means experimentation and growth were possible from the community.
- Everyone's tools play nice with each other! You can use Quasar as a library in Clojure and then feed that bytecode into Proguard for minification, and then add code coverage instrumentation, and then feed it into Robovm!
Given the wild success of bytecode and bytecode manipulators, I have no idea how the article can so whimsically poo-poo the entire field.
(Yes, I'm well aware Erlang has a VM that allows alternate languages as well. And yes, Elixer is pretty. OT, no, I won't be making investments of my time into Elixer, because I like strong compile-time type systems, and Elixer doesn't have that.)
It is true that even in the presence of a full greenthreading tool like Quasar, code can call legacy APIs that still block a full thread, but this is not sufficient cause to dismiss the possibilities. To quote back part of the article, blocking will always be an issue in any cooperatively multitasked environment: "There’s no real way to limit what that code can do, unless it is explicitly disallowed from [...] looping." And yet I wouldn't claim Erlang fails to give me concurrency just because it still allows loops! Part of the compromise of cooperative multitasking is the very premise that in exchange for the higher performance possible from cooperative code, yes, poorly written code can suck up arbitrary amounts of CPU before yielding. If this were a practical concern, it would also be entirely possible for a bytecode instrumenting library to inject cooperative rescheduling points even into loops; and yet I have no real desire to see this feature.
Furthermore, I strongly object to the claim ForkJoin is "notorious for its overhead". All thread synchronization is notorious for its overhead. That's completely known to any programmer with experience in this area, and in no way unique to ForkJoin.
For an excellent, in-depth coverage of what exactly ForkJoin is and the problems it solves for you, see https://www.youtube.com/watch?v=sq0MX3fHkro . I highly recommend watching the entire thing despite its length, and even if you are not a JVM programmer -- even if you've been doing concurrent programmer for years, you will almost certainly walk away knowing significantly more about concurrent scheduling from the (relatively) high levels of memory fencing all the way down to CPU architecture choices and their impacts.
I'm not going to claim there are no issues with something like Quasar. In particular, I find that it is harder to operate in an ecosystem where very few existing libraries understand what your application is trying to do with green threads. Mostly, this doesn't phase me if my application is calling out to other libraries, because I control the scheduling one step above them (just like I would in a plainer actor framework without green threads like Akka). The problem is more with "hollywood" style frameworks -- the "don't call me, I'll call you" type -- so far it feels like these are very hard to use when your application is using green threading, but the calling framework has no clue about it. Some sort of interfacing code is required and usually has thread handoffs of its own, which can be moderately unpleasant, and limits your scalability at that juncture. But this is a present-tense bummer, and can be solved by patching (or outright replacing) these hollywood frameworks, or simply avoiding frameworks of that kind altogether.
But in short, I still think it's a bit unreasonable to dismiss the existence of ponies.
- you have to balance load across true OS threads to take advantage of multiple CPUs. (You often pin an OS thread per CPU.)
- if you make a call to a blocking OS function you have no way to pre-empt your lightweight thread.
HTH.
Update: "you have to balance load" --> I mean the green thread library implementer. Users of a lightweight threading library typically don't concern themselves with this, though they might if performance becomes an issue.
http://en.wikipedia.org/wiki/Green_threads#Green_threads_in_...
http://www.cs.bell-labs.com/who/rsc/thread/newsquimpl.pdf
A good overview (of CSP) document is written by Russ Cox:
Both the above articles are more focused on implementation as opposed to theory.
PS: at least not the V8/Node.
1. ForkJoin is not "notorious for its overhead". In fact, it is among the best implemented, best performing work stealing schedulers out there. Scheduling a task with ForkJoin takes a few nanos, and is almost as cheap as a plain method call. Don't take my word for it: go ahead and benchmark it.
2. Like Go, Quasar doesn't constrain the running code from mutating shared state -- if you're using Quasar from Java, that is. But it's still just as useful as Go, and when used from Clojure, it's even more flexible than Erlang, and actually quite safe.
3. My macbook isn't cruddy.
4. The stuff possible with Quasar, like running a plain Java RESTful service on fibers to gain a 4x increase in server capacity -- without changing the code and without even starting to parallelize the business logic with actor/CSP -- speaks for itself.
5. I'm not spreading FUD on threads -- you can watch my talk at JVMLS (linked in the article) to see my precise point: kernel threads cannot be used to model, one-to-one, domain concurrency, because the concurrency requirements of modern application (and the capabilities of modern hardware) exceed by several orders of magnitude the number of threads supported by the kernels. Fibers keep the (excellent) abstraction provided by threads as the unit of software concurrency, while making the implementation more suitable for modern soft-realtime workloads. When your average programmer can spawn up a (lightweight) thread without thinking about it -- say one for each request, and even many more, concurrency becomes a lot easier.
6. The linked Paul Tyma slide are completely irrelevant. I've got nothing against doing kernel-thread-blocking IO. The problem becomes writing simple, yet scalable code to process incoming requests. Modern hardware can support over a million open TCP sockets, but not nearly as many active kernel threads. Asynchronous libraries give you the scalability but fail on the simplicity requirement; fiber-blocking IO gives you both the performance and the simplicity of blocking code.
7. As to the "strawman benchmark" with "too many threads", the author is welcome to repeat the experiment using a thread pool with as few or as many treads as he'd like -- the result would be the same: switching kernel threads costs about 10-20us, while task-switching fibers costs 0.5us (and can be improved).
> few existing libraries understand what your application is trying to do with green threads
That's exactly the purpose of the Comsat project, which integrates existing third-party libraries with Quasar fibers. You're right, integrating "inverted" frameworks does require more work, but so far Comsat integrates, servlets, JAX-RS services and Dropwizard.
[1]: http://blog.paralleluniverse.co/2014/05/29/cascading-failure...
Indeed! :) And Comsat is a hugely important part of Quasar's growing usability in real-world applications. (I'm using the servlet & JAX-RS code right now. So yeah, it's safe to say I'm thrilled about those integrations.)
Quasar also has great abstractions available if one needs to generate new bindings to any code which can currently produce callbacks: FiberAsync [1] is every bit as simple to use as the docs indicate.
But it is still slightly-more-than-none work required when dealing with hollywood frameworks that haven't already been adapted. It's totally manageable; at the same time, it's my personal hope in the long run we see more frameworks growing up that deal with green threading naturally.
[1] http://docs.paralleluniverse.co/quasar/javadoc/co/parallelun...
Does any of this benefit a desktop app? I realize that most of the green-thread interest lies in asyc i/o and i/o bound workloads. But can a desktop app with a couple of dozen threads (i/o + cpu mix loads) gain something from Quasar?
I think it's fair to say that what we are being offered is not ponies. It's a useful tool, but also a leaky abstraction and not what I want to be working with in the long run.
Also the statements about Erlang and loops is odd. My understanding is loops in Erlang are preemptible and that the VM bends over backwards to provide consistent scheduling even if it means losing some performance.
Erlang is still essentially cooperative and not preemptive, if I've understood my reading. That means the BEAM VM is doing something very similar to the style of instrumentation Quasar is doing: it injects yields into your code at points it thinks are reasonable. This is not quite the same thing as true preemptive scheduling as OS-native threads do. Quasar could do this kind of safepoint injection as well, though afaik that's not currently a feature.
Your definition of ponies and mine seems to diverge here, and that's fine :) I agree that true preemptibility is an even higher bar we can hold scheduling frameworks to. But it's also a very complicated area to get into, it's not completely without it's tradeoffs (full preemptibility pretty much gets us back to OS native threads, right? and there's very real performance reasons there's so much momentum away from that right now), and I also feel that I can get a lot done with green threads without these features. Maybe we'll see a growing swing towards safepoint injection for psuedopreemptibility -- I'm just making words up at this point, as far as I know; if there's a better existing terminology for these shades of grey I'd love a link -- in the coming years. I don't know where I place my bets on that, yet.
EDIT: this also appears to have been discussed before at https://news.ycombinator.com/item?id=7962838
This will generally cause scheduler collapse and result in all sorts of weird problems, so in the most recent version of the BEAM, there's 'dirty scheduler' support so that you can work around that problem if you have native code that runs for a long time (> 1 ms).
A good primer on all of this is http://jlouisramblings.blogspot.com/2013/01/how-erlang-does-....
Yes, given that the post links to Quasar code :-) Take a look at what the "bytecode weaving" link points to...
> hopefully without altering its meaning
...What?
I'm using Quasar right now. It works as advertised. Bytecode manipulation tools are not some scary unexplored arena of JVM tooling.
Erlang spreads processes across OS threads, like most other green-threading impls, and its not always great at it. (I don't know what you mean by "across OS processes." Processes on different Erlang VMs can communicate but the scheduler isn't going to move processes between them)
Calling into native code isn't an easy problem in Erlang either. NIF calls will block a scheduler thread, but the scheduler knows nothing about how long a NIF call is expected to take and will happily queue up processes to be run on a thread that is blocked inside a NIF call.
"Regular" erlang I/O is done by queueing up requests to be fulfilled by .. you guessed it, a pool of threads that spend most of their time sleeping in blocking i/o calls.
A) Frankly, channels result in prettier, more maintainable code. I've seen enough questionable uses of LinkedBlockingQueue to last me a lifetime. Inability to so much as "close" a BlockingQueue in the face of multiple concurrent consumers is an unbelievable cramp -- it won't bother you until it does, but when it does, it's just a bellyflop-onto-concrete sort of sensation.
B) I'm even more pessimistic than Pron's sibling response about scalability of threads. A minecraft server with a even a few dozen concurrent players is starting to feel the limitations of naively scheduled threads, as an anecdote. Part of this comes down to the choices of concurrent data structures, how interaction with shared data strucutures is batch and the resolution of locks, the devil is in the details etc etc etc, but I'd venture that the abstractions with green threads and channels make good code a heck of a lot easier.
Truly trivial apps with one "compute" thread and one "UI" thread are unlikely to see serious performance gains. Similarly applications that have workloads that are highly parallel (say, somewhere around $num_cpus threads which exchange information only once every few hundred millions of cycles -- spitballing a bit, but for context I think the Doug Lea talk I linked in earlier (https://www.youtube.com/watch?v=sq0MX3fHkro) mentions thread unpark can take up to a million cycles in a worst-case scenario) are unlikely to see serious performance gains. So there are situations where green threading can't help you from a purely performance perspective, yes. But in practice, it's my observation that it's startling how quickly "simple" apps end up doing enough concurrent UI or network operations that naive threading starts getting unpleasant.
Out of curiousity how is Quasar detecting blocking?
To my knowledge ForkJoin doesn't detect blocking?
From the JDK 7 ForkJoin javadoc > However, no such adjustments are guaranteed in the face of blocked IO or other unmanaged synchronization
Sure the framework has extra threads and will work around it via work stealing, but you can get a lot of blocked threads at the worst possible time when you hit a correlated source of blocking.
You also lose thread affinity once work stealing kicks off.
Notifying a framework of potential blocking is certainly less nasty than what I do to work around blocking without one, but for an existing project of sufficient scope it's tough to transition.
Just to clarify: it's perfectly OK to call blocking code on Quasar fibers. In fact, it's encouraged. But the blocking call must be "fiber aware", and there's a project called Comsat, that takes many popular Java libraries and makes them fiber-blocking without changing their APIs.
This leads me to another point, which is time-slice based preemption of fibers. That's a feature Quasar had early in it's evolution, but has since been taken out (Quasar is preemptive, but doesn't offer time-slice scheduling). The reason is that time-slice scheduling is great when you have hundreds of threads running, but quite terrible when you have a million, because it means that the threads (lightweight or not) constantly compete for CPU cycles that the CPU just can't keep up with. In Java, plain threads are still available (with the same API as fibers, i.e. new Thread vs. new Fiber etc.), so for long-running computations, you're better off using a kernel thread; work-stealing scheduler aren't great at scheduling such tasks anyway. In Erlang, you don't have access to kernel threads, so time-slice scheduling is necessary to support the occasional heavy-computation process.