How we’ve made Raptor fast

How we’ve made Raptor fast(rubyraptor.org)

178 points by triskweline 11 years ago | 71 comments

gnufied 11 years ago |

Just some glaring inconsistencies that I found:

> It’s less work for the user. You don’t have to setup Nginx. If you’re not familiar with Nginx, then using Raptor means you’ll have one tool less to worry about.

>For example, our builtin HTTP server doesn’t handle static file serving at all, nor gzip compression.

Sounds like I would need nginx(or another frontend server) anyways?

2. > By default, Raptor uses the multi-process blocking I/O model, just like Unicorn.

> When we said that Raptor’s builtin HTTP server is evented, we were not telling the entire truth. It is actually hybrid multithreaded and evented.

So, which it is? I assume by default is multi-process + events, but a paid version offers multithreaded + events? If so, isn't unicorn's model of multi-process+blocking IO is pretty good as well because OS becomes load balancer in that case.

Overall it seems they wrote a very fast web server. Kudos to that! But I don't think the web server was ever the problem for Rack/Ruby apps? Still on fence with this one until more details emerge. :-)

shandogs 11 years ago | |

The first point makes sense for me given that all my static files are served gzipped out of cloudflare/cloudfront on another domain.

wereHamster 11 years ago | | |

And the rails asset pipeline can pre-compress the files, so you don't waste compressing them for each request individually.

bratsche 11 years ago |

Does it seem weird to anyone else that they're doing all this marketing for a project that's supposedly going to be open source? Why all the suspense, why not just release it? Or, if it's not ready yet, why not just finish it and then start promoting it?

theossuary 11 years ago | |

Half way through the document they mention their plan to add paid for features: "Raptor also optionally allows multithreading in a future paid version."

teddythetwig 11 years ago | | |

Yep, that's when I stopped reading and closed the article. Why hide the fact that you are building paid software? Perhaps it was naive to assume that it was open source, but that piece really surprised me.

lbotos 11 years ago | |

They have a full drip with email capture around this product as they are looking to sell a portion of it. (I think it was a multithreaded variant.) I do agree that I wish the product was ready but I can't fault them for "following the playbook".

bithive123 11 years ago |

As someone working in an enterprise environment, I've sort of lost interest in this breed of Rack server now that I've gotten used to having SSO and LDAP authorization available via Apache modules, to name a few features. Apache allows me to accommodate all sorts of requirements like setting up vhosts that require authentication except on the LAN, or vhosts which allow members of certain groups to access an internal app via reverse proxying.

I don't mean to be negative; other posters have that angle covered. But I would comment that this ongoing proliferation in prefork backends is hardly disruptive to organizations who have already made significant commitments to Ruby web apps. Our Apache/Passenger servers aren't going away anytime soon.

rarepostinlurkr 11 years ago |

This is Passenger Phusion +1. As was pointed out in a thread several months ago, the DNS resolves to the same place. The writing style is similar and the feature set far too mature for a 1.0 product.

randall 11 years ago |

How would one use this on Heroku? It doesn't support static file transfers allegedly... and per Heroku they require your app server to serve them by default.

https://github.com/heroku/rails_12factor#rails-4-serve-stati...

Any ideas?

snikch 11 years ago | |

You use a rack middleware to handle static files, which is how Rails handles it by default. So, unless I'm completely mistaken (which I may well be), this should work just fine?

randall 11 years ago | | |

Wouldn't that sit behind the app server? (Equally mistaken potential. :)

_raul 11 years ago | |

Using a CDN (Cloudfront, Fastly add-on, ...) is a common choice, as it allows delivering assets from a closer datacenter to the client and removes load from the app server.

randall 11 years ago | | |

Yeah but getting assets to said cdn is annoyingish without having app server origin, I'd argue.

fiatmoney 11 years ago |

"You will need 5000 processes (1 client per process). A reasonably large Rails app can consume 250 MB per process, so you’ll need 1.2 TB of RAM."

Quibble: most multi-process web servers use fork() for child processes, which means they can share identical memory pages.

rustyconover 11 years ago | |

Not deeply understanding copy on write[1] semantics for virtual memory paging and designing application servers seems just foolish.

I'll chalk this one up to the PR/marketing person probably not taking an OS course.

Still it would be nice if they really did go back and read a little W. Richard Stevens[2].

[1] - http://en.wikipedia.org/wiki/Copy-on-write [2] - http://en.wikipedia.org/wiki/W._Richard_Stevens

rubiquity 11 years ago | | |

> I'll chalk this one up to the PR/marketing person

This entire web server is a marketing hype since day one. I imagine they are trying to build a pro product and support company out of this.

It's a web server with event loops and some fancy memory allocation. Shouldn't Node.js have taught us all by now the perils of event loops and insanely tweaked HTTP parsers? Sure, it looks great for "Hello World" benchmarks but falls right on its face as soon as you have an app of significant size spending real time on CPU.

FooBarWidget 11 years ago | |

In my experience -- as the Phusion Passenger author, and as the one of the original developers behind the copy-on-write feature in Ruby -- a moderately large Rails app can use 250 MB per process even with copy-on-write.

Rapzid 11 years ago | |

Now that they have fixed the garbage collector. However, how much of that 250MB is built up after the fork, and how much is static? I actually don't have that answer.

rubiquity 11 years ago | | |

In the case of Rails, all of your Gems, Rails and application code are loaded pre-fork.

hurrycane 11 years ago |

I strongly believe that this is the next version of Phusion Passenger.

grandalf 11 years ago | |

The writing style is very similar.

hurrycane 11 years ago | | |

Their architecture looks to me really mature - a lof of features are the same as Passengers'

triskweline 11 years ago |

Spoiler: Insane amounts of low-level optimization.

film42 11 years ago | |

Actually, everything was pretty simple: Use libev since it's faster than libevent, use Node's http parser because it's fast. Then allow each thread in the pool to run its own event loop. This pretty much sums up my university's Internet Programming course. There were few hairy bits about tcmalloc, but they did a great job about explaining how they took advantage of object pooling and region-based memory management. Great post guys, I can't wait to give your source a read :)

yazaddaruvala 11 years ago | | |

Any idea why they didn't use libuv instead?

nteon 11 years ago | |

I would have loved to see some data that showed their bump pointer regions and thread local stuff performing significantly better than tcmalloc or jemalloc, both of which do thread-local caching that avoids locks for the vast majority of allocations. Additionally, what they came up with sounds like talloc[1], which has been in production for years with samba.

1 - http://talloc.samba.org/talloc/doc/html/index.html

jonaphin 11 years ago |

Congratulations on Raptor, I'll definitely give it a whirl. Regarding static asset serving, I'm fairly certain serving them through the application server is often not the way to go anyway.

resca79 11 years ago |

Raptor seems pretty interesting, but personally I don't like its marketing approach, and I'm not the only one.

On twitter some ruby heroes say : " Raptor is 4x faster than existing ruby web servers for hello world applications" :)

The strong proclamations in favour of an open source project is a little bit strange if the open code is not yet released.

However I hope that all graphs on the home page are real for the ruby programmers happiness

alvare 11 years ago |

http://www.yesodweb.com/blog/2012/11/warp-posa

covi 11 years ago |

The section "Hybrid evented/multithreaded: one event loop per thread" suggests that the whole model is basically SEDA [1]. I'm surprised the article does not directly reference the project/paper.

[1] http://www.eecs.harvard.edu/~mdw/papers/seda-sosp01.pdf

jrk 11 years ago |

The "hybrid" IO architecture is historically known as AMPED (asynchronous multi-process event driven): https://www.usenix.org/legacy/event/usenix99/full_papers/pai...

simonmales 11 years ago |

Puma says it runs best when using Rubinius or JRuby, but from my limited understanding, not everything can run on those implementations.

Are there any giveaways in the blog that wouldn't allow Raptor to run on Rubinius or JRuby?

steveklabnik 11 years ago | |

That's basically because Puma uses threads, and threads are way better on Rubinius and JRuby, due to lack of a GIL. It's not due to some kind of incompatibility.

riffraff 11 years ago | |

well if part of it is written in C++ it most likely won't run easily on either?

jcampbell1 11 years ago |

Does anyone know if the 60.000 in the chart means 60 or 60,000?

djur 11 years ago | |

It's from Phusion[1], which is headquartered in the Netherlands. Non-English Europe conventionally uses . as a digit separator, so 60.000 would be sixty thousand.

[1]: % ping rubyraptor.org PING rubyraptor.org (97.107.130.55) 56(84) bytes of data. 64 bytes from shell.phusion.nl (97.107.130.55): icmp_seq=1 ttl=50 time=98.3 ms

davedx 11 years ago | |

If it's the Passenger guys then 60.000 is how you write 60,000 in the Netherlands ;)

mrinterweb 11 years ago | |

I was thinking the same thing. 60 reqs/sec is not very fast. I suppose it depends on what is being tested. "Hello World" is pretty useless to base a benchmark on as DB and other IO is not involved. Who knows what that number means without some source code it was benchmarking.

Ono-Sendai 11 years ago |

Not bad work. Seems somewhat futile though, since the speed will probably be massively slowed down by the actual ruby application code and database accesses etc..

derwiki 11 years ago | |

If it means you can squeeze more requests/sec out of the free tier of Heroku, I'm all for it!

mrinterweb 11 years ago |

I am all for a faster ruby application server. If Raptop can stand behind its claims on November 25th, that will be the best birthday present I could get.

corford 11 years ago |

Slightly OT but does uwsgi feature much in the ruby world? /from a curious python guy

bradleyland 11 years ago | |

Ruby uses a specification called Rack.

http://rack.github.io

kevinastone 11 years ago | | |

uWsgi is an implementation of the python WSGi spec that also supports Rack: http://uwsgi-docs.readthedocs.org/en/latest/Ruby.html

rurounijones 11 years ago | |

I try to keep up with most things ruby and I have never heard it discussed in a ruby context

swrobel 11 years ago |

Does it support SPDY?

coned88 11 years ago |

what are these instead of just using a webserver like apache or nginx?

SpikeGronim 11 years ago | |

You generally use Unicorn etc. behind something like nginx. The last time I did that I used nginx to handle thousands of concurrent connections that were forwarded to nCPUs Unicorn instances. Nginx is very good at handling lots of connections, Unicorn is good at handling a Rack app.

tryp 11 years ago | | |

To be a bit more explicit, using a separate httpd and application server allows a division of labor between the resource-bound task of handling the request + building the response from the network-bound task of dibbling bytes back to the original requestor.

Nginx (and the general class of highly concurrent servers) is good at handling lots of connections largely because it tries to minimize the resources (memory, process scheduler time, etc) required to manage each connection as it slowly feeds the result down the wire.

The application server generally wants an instance per CPU so that it can hurry up and crank through a memory-, cpu-, or database-hungry calculation in as few microseconds as possible, hand the resulting data back to the webserver and proceed to put the memory, DB, and CPU to the task of processing the next request.

This is in contrast to the (simplified here) old-school CGI way that say ancient Apache would receive a request, then fork off a copy of PHP or Perl for each one, letting the app get blocked by writing to the stdio pipe to Apache then Apache to the requesting socket. All the while maintaining a full OS process for each request in play.

lbotos 11 years ago | |

This is an "application server". When you are doing anything that isn't PHP you basically need a process behind your web server (Apache/Nginx) that runs your actual ruby/python/java application code and speaks HTTP.

girvo 11 years ago | | |

Even in PHP you want that for decent performance: php-fpm is similar in concept to a Ruby/Python application server, albeit slightly different due to PHP being a web language first and foremost, and it's interesting execution method. Still, you run a process and connect nginx to it :)

jrochkind1 11 years ago | | |

Even for PHP you do, it's just delivered as an apache module. (Passenger can be run as an apache module too, fwiw).

Although to be fair, the PHP model doesn't require a _persistent_ process between requests (I think?). But most other platforms do.

kondro 11 years ago |

Does this mean that the majority of the performance improvements over Puma actually come from the fact that they are using the considerably less battle-tested HTTP parser of PicoHTTPParser over the Mongrel one?

Of course, this may be mitigated by the fact that any reasonable production environment will have a web server layer over the app server/s anyway for load balancing, fail-over and exploit detection/prevention anyway.

adamnemecek 11 years ago | |

They actually aren't using PicoHTTPParser.

kondro 11 years ago | | |

Whoops! My reading skills need some work.