Erlang/OTP architectures: Cowboy(medium.com) |
Erlang/OTP architectures: Cowboy(medium.com) |
nginx came in at 14612 requests per second, and the dynamic code under Cowboy option came in at 13336 per second.
The same server was pulling 147 requests per second for a "hello world" rails application (Unicorn, production mode). Cowboy really isn't joking when they talk about performance.
Edit: Eh, I forgot a "not" in the last sentence, which is now added.
I have no dog in this fight, but I've seen many people say that -despite the newness of the library "ecosystem"- they get an awful lot done with Elixir/Phoenix, and have a lot of fun doing it.
(Remember that Elixir is -like Erlang- a BEAM language. This means that Elixir gets the ease-of-understanding and interesting features of its syntax and has access to all Erlang software ever written. Similarly, any and all Elixir code can be called from Erlang with little trouble.)
Edit: (Also remember that the BEAM equivalent of Ruby is Elixir, and the (most popular?) BEAM equivalent of Rails is (AIUI) Phoenix. Phoenix uses Cowboy as its HTTP(S)/SPDY server.)
In fact I recommend it for a lot of cases. Problems that were solved years ago in Rails, like sane and default CSRF protection, are showing up in newly written PHP7 and Node applications that I'm seeing, because the things that Rails does are generally there for your own good.
I'm also aware it's not a direct comparison because Rails actually had an edge as far as its codebase.
The point I believe we can still take away from the original comment is - Cowboy is fast.
I'm very much looking forward to v2, the way it handles parameters is a much more natural fit imo.
Elixir and Phoenix give you the great productivity and the performance of being run on the Erlang VM.
RoR is productive for website/web apps.
I mean looking at just request per sec that metric would be much better for something like API/data serving app.
Performance might not be on the top list, but when it comes usually a full re-write might be the only way out, as it was on our case.
Depending on the overall business situation such re-rewrite can have a huge impact on the company, due to shift on the development resources.
Since then, I only advice languages that have either JIT or AOT compilers available to them.
But in reality we need a better benchmark than "hello world". Something like TodoMVC, but for server-side development.
eg.
GET /todos
* authenticated
* 100 todos
* + threaded comments
(edited for formatting)
So if you read this Loïc, thank you for all your work and for sharing it with the community.
Our company has been sponsoring Cowboy development for about 18 months (https://medium.com/@abs/what-we-learned-from-sponsoring-an-o...)
Join us, if you can.
It seems a lot of people are having success with Erlang but I can't figure out how these applications actually run; it almost seems like there isn't a single step-by-step tutorial on the Internet for setting up an Erlang system using "best practices."
Am I just missing something? Because I feel like I'm hugely missing out.
$ mix phoenix.new my_app
$ mix phoenix.server
It may be worth asking josevalim and chrismccord in #elixir-lang for tips on setting it up, since they obviously must know how to set up cowboy pretty well to build a system on top of it.
I think Erlang and things like cowboy and OTP are absolutely amazing for the web, but Elixir has made things 100x easier to get up and running with them.
You can also clone the repo and checkout the examples:
Make sure you have erlang installed
$ cd examples/hello_world
$ make
$ ./_rel/hello_world_example/bin/hello_world_example start
OR
$ ./_rel/hello_world_example/bin/hello_world_example console
As someone who's been down this road in the past, unless you're already rather familiar with Erlang and OTP's conventions, the gap between the User's Guide and the API documentation is far too large to make connecting information you learn from one with information you learn from the other easy.
I've summarized my personal struggle with the Cowboy user's guide and documentation in my other comment in the thread. Please read that comment before you downvote this one. :)
It walks through a simple "Hello world" application starting with creation from template (erlang.mk), and discusses static files, SSL, websocket.. A very good first guide in my opinion. Though it doesn't really cover other techniques like how to architect your application or interact with a database.
Several months later, with a lot of additional Erlang experience under my belt, I come back to the documentation, and it's -somehow- 60->70% comprehensible. I figured out what was wrong with my simple HTTP server. (I was calling my request handler in init, rather than returning {ok, Req, []}, and letting Cowboy call my request handler later.) I still can't get cowboy_rest to work, but (OTOH) I haven't spent more than five minutes trying to get it to work since my first attempt way back when.
I don't know how much experience you have with Erlang, so I'll give you some basic advice that might be entirely unhelpful. Feel free to ask additional questions. Also feel free to ask questions on the erlang-questions mailing list. [0] Everyone who regularly posts there is very helpful. Many (most?) of them use Erlang professionally.
Anyway. Advice:
Projects/frameworks like OTP and Cowboy provide a callback API that application code is expected to conform to. Cowboy's documentation for this API is kind of all over the place, [1] but OTP's is well organized. [2]
Because Erlang tries to eschew global state as much as it can, OTP has a convention that one of the values in the tuple returned from an callback is the state for that particular instance of your application code. This State variable seems to always be the last element in the returned tuple, and the last argument to your callback function. State can be any Erlang term; it only has meaning to your application code. If your callbacks don't care about state, then you can pass anything as state. Many other frameworks have adopted this convention; it's a pretty good one.
Cowboy appears to be a pretty good HTTP/SPDY server. The beginner's documentation needs a LOT of love. (Even now, I had difficulty trying to piece together that walk through the docs in the footnote.) Given that it's a one-man shop, it's surprising that the docs are in as good a shape as they are; documentation isn't easy. :) But, still...
Anyway, questions? Comments? Ask away either here, or on the erlang-questions mailing list. :)
[0] http://erlang.org/mailman/listinfo/erlang-questions
[1] First you have to read the examples shipped with the source code https://github.com/ninenines/cowboy/blob/1.0.x/examples/hell... https://github.com/ninenines/cowboy/blob/1.0.x/examples/hell... to kind of get the impression that you need to use the cowboy_http_handler API (kinda introduced here http://ninenines.eu/docs/en/cowboy/1.0/guide/http_handlers/ and documented here http://ninenines.eu/docs/en/cowboy/1.0/manual/cowboy_http_ha... ) in order to handle plain HTTP requests. If there's a better way to arrive at this conclusion, I overlooked it. :(
[2] For example, the docs for OTP's application API: http://www.erlang.org/doc/man/application.html or the gen_server API: http://www.erlang.org/doc/man/gen_server.html The parts before the "CALLBACK FUNCTIONS" section are functions you use to manipulate an application or gen_server. The bits after are functions you implement in your module that the application or gen_server machinery will call.
I also got started from the cowboy user guide (with prior Erlang experience), but found it relatively easier, as I used the template, then cross-referenced the idiomatic template with the docs as I worked through them (tweaking 'knobs' as I went along), which worked well in helping me understand the design.
Along with the Erlang mailing list you've mentioned, I would also like to mention #ninenines (and #erlang for more general questions) on freenode. Very helpful communities!
[1] This page describes the setup: http://ninenines.eu/docs/en/cowboy/HEAD/guide/getting_starte...
> RoR is productive for website/web apps.
Phoenix [0] uses Cowboy as its HTTP server.
erlang.mk was around. My two gripes with it:
* It enforces warnings-as-errors. This is a personal preference, but I hate this while developing. I want to iterate, not have to clean up unused variables that I already know about.
* It -until very recently- failed so, so, so hard when run with parallel make. I -at first- idly wondered if -somehow- Cowboy application builds were just non-deterministic. I then remembered that I aliased 'make' to 'make -j3'. Building with 'make -j1' fixed the build and put me squarely in the Rebar camp.
Rebar is just a far better fit for how I work.
> ...did you use the cowboy_http / bootstrapped release template...
Nah.
When learning new things, if I'm not in a rush, I tend -at first- to avoid templates. I've been burned by boilerplate code in the past, and have had boilerplate code cover up the true crawling horror that was more than a couple of APIs.
I'd rather initially do things "from scratch" [0] so that I can better evaluate the thing I'm learning, and the quality of its supporting documentation, or -if the documentation is inadequate- the explanatory power of its test suite and other sample code.
I did -however- read Cowboy's Getting Started guide, along with a huge chunk of the documentation. I actually started with the API documentation, figuring that it was going to be like erlang.org's documentation. Whoops! [1]
[0] And by that, I don't mean "from first principles". I'm quite happy to read an example, partially or fully digest it, then paste it into my text editor.
[1] Again, I know how good erlang.org's documentation is and how very much work it is to make such high-quality documentation. I don't mean to slight Loïc Hoguin. :)
1) The Elixir community is still in its infancy and the community is growing rapidly and flourishing. If the concern is there isn't an equivalent Elixir library for X gem, it's only a matter of time before it is. So please join!
2) I attended the Phoenix training at ElixirConf this week and an easy answer to registration and auth is "hey, write a plug". Once you dive into how straightforward plugs (1) and what (init/1, call/2) do, it's really simple.
You will appreciate the explicit nature of plugs compared to say, Rack middleware...
I'm rewriting a Rails app in Phoenix and overall, I've found it to be a really pleasant experience.
If you'd like to learn about the concepts behind an OTP application, you can read this guide: http://learnyousomeerlang.com/building-otp-applications or the Erlang documentation for OTP apps (which is pretty good I think).
Another viable option in my opinion would be to just look into the Phoenix framework for the Elixir language. Under the hood it uses cowboy. And it feels a lot like Rails.
> As far as I know there's no such thing as a `Cowboy application`.
From one pedant to another: "I know that." >:)
However, there is (to put it loosely in OTP terms) a set of Cowboy behaviours. Your application is expected to implement one or more of these. While the API for these behaviours is reasonably documented, the Cowboy equivalent of the -say- gen_server "User's Guide" is so sparse as to be almost non-existent.
I think one reasonable strategy of learning how some piece of software works or is meant to be used – especially when documentation is poor – is to just look at the source code (tests included).
I'm just getting started with the Phoenix framework, and I'm learning Elixir at the same time and reading through the source code helps me a lot, because:
1. I can check out how things are implemented
2. I get to read some idiomatic Elixir which helps me learn the language better
I was either unaware (or had forgotten that) he had taken a second stab at the benchmark. I do hope he updates the document with the root cause of the enormous variance in response time.
Edit: Good find, BTW. Thanks for digging that up.
I note with interest that the two frameworks which are faster than Phoenix (in both throughput and latency) are Gin, written in Go, and Play, written in Scala. In this run of the benchmark, they also have more consistent latency than Phoenix, although the author notes a suspected problem with his Erlang installation; previous runs had consistency better than Play, but still not as good as Gin.
I note with amusement that:
> Play consistently appeared to generate hundreds of socket read errors (see the detailed output), so I believe it should probably be semi-disqualified from the results for now.
I'd be interested to see another JVM framework tested. Play is really rather complicated, and it's written in Scala, which is also rather complicated. I'd love to see a comparison with a Sinatra-for-Java type framework like Spark, or perhaps Undertow (since it does so well on the TechEmpower benchmark). Oh well, maybe it's pull request time ...
A couple of things:
Both Erlang and Python compile down to bytecode. Python compiles the first time you run a given script, Erlang won't execute Erlang source code; you must compile it to run it.
As go1979 mentions, Erlang has HiPE, which compiles Erlang down to the host platform's native code. If you find that your program is spending a lot of time in the Erlang code in a given module, you can -at any time- recompile that module with HiPE and reload it with either a system restart, or the hot code reloading machinery.
As an additional escape hatch, Erlang makes it rather easy to push performance-critical code out to a faster language -like C or C++ or whatever- and call that directly from Erlang. [0] Such code is called an NIF [1].
If most or all of what you're doing is hard-core number crunching, Erlang is probably not appropriate. If what you're doing requires tricky logic, needs to contain failures, or you're building a concurrent or distributed system, Erlang may very well be the right tool for the job.
[0] See an NIF as it is used https://github.com/davisp/jiffy/blob/master/src/jiffy.erl#L1... the corresponding C boilerplate to interface with Erlang https://github.com/davisp/jiffy/blob/master/c_src/jiffy.c and the implementation of the guts of that NIF: https://github.com/davisp/jiffy/blob/master/c_src/decoder.c#...
It seems my post came out badly.
Ah. Yeah, it did, a little. Next time, maybe mention one or two languages that you recommend and one or two languages that you don't, for context.
Oh, well. :)
AOT = head of time compilation. https://en.wikipedia.org/wiki/Ahead-of-time_compilation
Erlang's HiPE compiler is apparently AOT. The challenge seems to be dynamic typing. At points, it has to guess types. I assume compiled Python would have similar issues.
However. If the language the project is written in is wildly unfamiliar to you, and you've never had experience with a similar language, then this technique is often... less than satisfying.
From your comments, it appears that you have more than a passing familiarity with Erlang/OTP. You have to admit that it would be substantially harder to understand both Elixir and Phoenix without your understanding of Erlang. :)
Until he figures out what was up with his terrible Elixir consistency numbers, I would treat the second round of results with some suspicion. Maybe something changed in Phoenix/Elixir/Erlang, or maybe something's up with his machine.
If you do submit a PR, I do hope that you have the time and gumption to re-run all of his round 2 benchmarks, in addition to any additional ones you care to see. :)
If you stick to writing code only in a BEAM language, you're insulated from all errors except for those of system resource exhaustion (edit: or -obviously- system hardware failure ;) ) or programmer logic errors. [0]
The parts of Elixir and other such projects that are written in another language that compiles down to native code, [1] are vulnerable to the usual array of issues endemic to the language selected.
> Yes, BEAM is proven in enterprise use, but most of us don't have those resources.
I'm not sure what you mean by this, and what concerns are implied by this statement. Would you care to expand this statement?
The WhatsApp folks have found that they got a lot done with Erlang with a small staff and (what is reported to be) very few machines when compared to the amount of work performed.
[0] Erlang can neither save you from your own faulty logic, nor can it add resources to your system. ;)
[1] Erlang provides a mechanism called NIF that lets you write performance-critical (or whatever) code in C or another language, and link it in to Erlang. From there, you can access your other-language code with from your Erlang code what appears to be a regular Erlang function call.
"Would you care to expand this statement?"
It seem proven enough if you have dedicated highly skilled developers. Not necessarily if you're a smaller startup that have to do a lot of other things and are relying on the ecosystem. With one of the more well know stacks you have a long history of not only security fixes, but knowledge about how to do things.
It kind of boils down to if I would implement say a payment system with the stack could I feel confident we wouldn't get compromised? Currently I have a lot higher confidence in e.g. Python + Django + Nginx than Elixir + Phoenix + Cowboy.
That said, I'm still playing around with it.
So, the following disjointed commentary might be entirely superfluous. It also makes no mention of Elixir or Phoenix, as I've not yet used them:
AFAIK, everything shipped in Erlang/OTP has a test suite that you can run and -if you've the time and technical chops- inspect and evaluate for completeness and correctness.
Given that the Erlang community tends to be concerned about safety and reliability, I have substantially more confidence in the correctness of some random Erlang library on Github than of some random JavaScript (or -ye gods- Perl) library. ;)
Given that neither Cowboy nor Ranch appear to make use of any NIFs, I would be fairly confident in their safety in the face of garbage or malicious input.
The erlang-questions mailing list is a good place to ask questions of folks who use Erlang professionally and non-professionally for a wide variety of things. I've heard that the Elixir equivalents are similarly helpful.
EDIT: I also believe Heroku's request routing system is built on Erlang.