The architecture resembles the Actors model, providing semantics of encapsulation and location transparency, thus allowing applications to be massively concurrent while exposing a simple programming interface to application developers.
and
S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data.
go stream processing. also see Flume for related recent works.
Anyone care to illuminate to me how this works? "Real-time MapReduce" sounds almost like an oxymoron.
The project's site ( http://s4.io/ ) isn't very helpful either. Full source:
the S4 open source project. coming soon.I understand they have to perform better than the average webapp, but still...funny.
By virtue of Java's structure and culture (as codified in its libraries), you have to spend approximately twice as much physical memory as you would in C. E.g., I had a 100,000,000 record array that I needed to sort. That's great -- Java has an Array.Utils.sort() method; But I needed to know the _sorting permutation_ as well. That requires an array of Integers, each taking >=16 bytes (so, 1.6GB of ram). Which slows everything to a crawl. So I wrote my own heapsort routine which needed only 400MB of ram. But it's 10 times slower than the equivalent C -- I suspect because of array bounds that the compiler can't throw away.
Those infrastructure projects are "cool", but horribly done. I don't know about Cassandra, but Hadoop makes jobs take between twice and ten times the resources that they need. Sure, it scales to 4000 machines, but you'd only need 1000 if it were properly written. And if you only have a 10-node cluster, you could probably run it on one machine with comparable throughput.
Color me unimpressed with Java.
Against C there is always the "Java makes it more difficult for you to mess things up" argument...
Between C, Python and Jython, I don't think there's a single reason left to use Java.
I don't think there's a measure of "absolute productivity", but if there is, Python will probably take world cup in a variety of settings.
Java is an excellent language when you need to use Java. If that's not a constraint, then it isn't such a good choice.