Show HN: Rocket – Web Framework for Rust(rocket.rs) |
Show HN: Rocket – Web Framework for Rust(rocket.rs) |
I recommend checking out the Yesod web framework [0], which leverages Haskell's strong type system to provide type-safety and a whole range of nice guarantees, including preventing vulnerabilities like the ones you mentioned.
Spock [1] is another cool web framework also written in Haskell that looks quite promising.
[0]: http://www.yesodweb.com/page/about
[1]: https://www.spock.li
But the whole point of a framework is to encapsulate all the things that you have to implement for every new web project, so you don't have to keep redoing that every time, and can instead focus dev time on new features and unique elements of each individual site. Securing against common vulnerabilities should be part of that.
There are a few frameworks that get this right, Lift (https://liftweb.net/) being the best I know of, designed to eliminate most of the OWASP Top 10 Vulns (https://www.ibm.com/developerworks/library/se-owasptop10/ind...) using Scala's strong type system and a virtual diff architecture similar to now-in-vogue Javascript frameworks like React. Haskell's Yesod framework is, as mentioned above, another good one in this respect.
I am writing "answers", plural, because although I post this as an answer to SkyMarshals text, I also mean the replies by aban, peller, and petilon.
[0] https://www.owasp.org/index.php/Category:Popular
[1] https://www.owasp.org/index.php/SQL_Injection_Prevention_Che...
[2] https://www.owasp.org/index.php/XSS_(Cross_Site_Scripting)_P...
[3] https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(...
[4] A good framework, used correctly, should take care of most (all?) escaping for you. For SQL injection, it's the ORM's job. For XSS, anything using a virtual DOM should escape stuff for you (I know React does). CSRF is more to do with session management, which is where using battle-tested auth code comes into play.
Really, it's about taint checking [1]. Distrust all sources of content by default that might have seen user input (or that you know have seen user input), and require explicit trust declarations from the developers to remove taints.
When you're about to use the data (e.g. appending to the DOM), you simply check for tainted data and escape is for the context you're in. Most times in the browser, that just means escaping content that might be valid HTML, but there are probably other contexts that require escaping as well.
For once, modern implementations are actually really helping security.
While not difficult to implement correctly yourself, I've found that applications written on frameworks that don't include this in the box tend to be applications vulnerable to CSRF.
struct Message {
contents: String,
}
#[put("/<id>", data = "<message>")]
fn update(id: ID, message: JSON<Message>)
Is Rust simple and elegant in the sense that Python is? I've also heard it's for systems programming, which makes me think of C and has kept me away from it. I like how I can bang out scripts quickly in Python. Is the same true of Rust?
edit: found this http://lucumr.pocoo.org/2015/5/27/rust-for-pythonistas/
Errors or mismatches in implementations are caught at compile time, saving minutes (or potentially hours or days) in frustrating debugging.
Nice work! This looks fantastic.
Sadly this functionality currently requires nightly or using a code generating build script via syntex :(
(Rocket currently only works with nightly, but someone could pretty easily add a syntex port since the APIs are the same)
With macros 1.1, some of this functionality is available on stable, but not the functionality needed by this. I don't know if we plan a "macros 1.2" that stabilizes tokentree expansion for all decorators (not just type item decorators). ("Macros 2.0" is the final polished version of Rust procedural and regular macros, for which there are concrete plans but it will take time to get to)
If you want to know more about our glorious macros 1.1 future, David Tolnay just have a great meetup talk all about this a few weeks ago:
error: 'id' is declared as an argument...
--> src/main.rs:8:9
|
8 | #[get("/<id>")]
| ^^^^
error: ...but isn't in the function signature.
--> src/main.rs:9:1
|
9 | fn hello() -> &'static str {
| _^ starting here...
10 | | "Hello, world!"
11 | | }
| |_^ ...ending hereRocket uses Hyper for its HTTP server. So I checked to see if the Hyper HTTP server was really production-ready. Particularly, if it could handle async IO / solve the C10K problem[0]. It looks like Hyper implemented async IO[1], so it should be adequate for production use in this regard. This is great news for users of Rocket because it means you don't need a separate HTTP framework to run your web service, as you would with, say, Django + gunicorn.
Sure, it's a dependency, but in this case, having a production-ready HTTP server out of the box is really nice!
Having said that, is there any literature on Rocket/hyper for security? The production-ready HTTP server is great, but it also means it has to be prepared to deal with certain security issues, like listening on 0.0.0.0 and handling file uploads. You could put HAproxy/nginx/whatever in front of it, but I think Rust has the potential to supply / manage all of this within one unit and simplify the stack / attack surface area.
Documentation is very sparse. Indeed, the linked to page seems to be the primary documentation repository. This is particularly painful for a brand new project, because googling is wholly useless at this point (the only relevant results being the linked-to page). You can try to infer things from the API, but there's a lot that I would refer to as "hidden gotchas" which you'll only learn about through experimentation or by reading the source.
For example, consider the FromForm functionality. Does it decode url-encoded values? Turns out the answer is: maybe. If your struct defines the field as a String then the input will be url-decoded, but if you define the field as a str then it won't be. FromForm only works at all if the submission is of content-type 'application/x-www-form-urlencoded', failing entirely 'multipart/form-data' for example -- I have no idea how you're meant to go about parsing a multipart form besides doing it manually using just the raw post-data string as input. Also, good luck parsing anything with a checkbox (as their code example does) -- if the checkbox is checked, no problem, the field is populated with 'true'. But if the checkbox is left unchecked, then you get a parse failure because you're "missing" a field. There may be a way to work around this but if there is it isn't well documented and I haven't read through that part of the rocket source yet.
FTA: Rocket is a web framework for Rust that makes it simple to write fast web applications without sacrificing flexibility or type safety. All with minimal code.
I'm at a total loss for why I really care about this major part of their message (type safety). If I have already switched to Rust then I care about type safety already and you don't need to sell me on Rust, you should focus on differentiating this web framework from the ones that came before it. But if I haven't switched to Rust then I am essentially unswayed by their messaging. I can't see why it's particularly important or different over other language/frameworks for web development.
Before learning Ruby I watched the Rails blog video sometime around version 0.14.3. Immediately you could the value of the framework and the ease of a language I was unfamiliar with. The win I could see was productivity and it was enough for me and a lot of other people to learn Ruby. I don't see that with this framework or really anything in the Rust world.
The framework seems to nail the API, but what's it doing down at the bottom of the stack? Any benchmarks?
Apologies if they are somewhere obvious, but didn't see it on the site.
Today, it's built on top of hyper, which is still synchronous. It's also only using the lowest parts of hyper, apparently.
There's a branch for tokio integration, but needs the tokio 0.1 release, which is coming soon but not _quite_ out yet.
But I don't mean in comparison to other languages or frameworks, I mean in generally, as in, I'd avoid creating syntax extensions or using them in any project.
UUIDs seem like it would be easier to use for demonstration : https://doc.rust-lang.org/uuid/uuid/index.html
And Cargo.toml declaration might need an extra sentence or two to explain the "features" part but that shouldn't be too bad:
[dependencies]
uuid = { version = "*", features = ["v4"] }Rust 1.14, released a couple of days ago, introduced experimental support for WASM/asm.js backends. There are experiments like https://github.com/tcr/rust-todomvc which show off the possibilities. At the moment it produces a rather massive blob (over 1MB as JavaScript, not sure as WASM), but that will improve, and for some situations that doesn’t matter so much anyway.
(The counter would be missile, I guess, and then icbm... followed by planetary-defense-cannon?)
This pattern is really nice for web frameworks. I'm super excited to see it for Rust! To me, this signals that we're getting pretty high up there on the early-adoption curve.
It's funny to note how much something small like this matters. It's syntax sugar and presentation, but the difference a comment like this has at the top of an HN post vs an ambiguous but detailed discussion about feature x...massive in effect.
It doesn't seem to be the case, but the entire framework under that could be bad yet it's caught mindshare on the basis of routing syntax (one of the smallest worries in a production web application). Wonder how many frameworks have failed just on the basis of not making a similar decisive first impression?
Syntax matters?
A quote from it:
> By far, the most common attribute people said they considered in the survey was whether a crate had good documentation. Frequently mentioned when discussing documentation was the desire to quickly find an example of how to use the crate.
Open source projects, in a sense, are like startups: if you want to acquire users, your users have to be able to understand the product! A deeply impressive technical project is, well, impressive, but if nobody can figure out how to use it, it's not going to see nearly as much uptake as a technology that's got a clear and easy way to use it.
There's secondary effects too: a project with a slick website like this makes people take it more seriously. It's less likely that someone who put this much effort in is just going to drop it immediately, though of course, that's not an absolute rule.
TL;DR: developers are users too. Developer marketing matters!
1) handle route (read data, transform, prepare other resources, check access rights)
2) generate response with reading/writing to db.
First step is not so primitive. Reading request data is often boilerplate-rich code, so syntax which can remove tons of boilerplate is very welcomed.
Just add a callback for your route and off you go.
The Regex crate (which I think is being merged with std) uses the same syntax as Python's regexes.
The Path/PathBuf type in std makes manipulating paths easy (and type safe). No more slicing notation/regexes to find where extensions start. Also it handles unix/windows / vs \ for you.
Python DuckTyping slightly drives me insane. What interfaces are/aren't typed feels like Russian roulete. Nothing like the 4000th processed file crashing your script
The real win is compatibility. If you avoid unsafe you don't need to think to port to Linux/MacOS/Windows.
Lastly it's a binary. So to share with a coworker, just copy it. No module management.
It's on that path, but it's unclear if and when a literal merge into std will happen. Right now, the design of 1.0.0 has been accepted, and once that lands and is released, it will move from "the nursery" to the regualr rust-lang org. From there it could go into std, or it may just stay there forever.
To add to what Steve said: there are currently no plans to export regex from std.
One of its major goals is making systems programming more accessible. So you shouldn't dismiss it because "systems programming = C", it's trying to change that! :)
That's not to say it won't involve learning some systemsy concepts. But it won't be as scary as C.
You can't necessarily bang out scripts quickly. A type system tends to make this task verbose. However, larger applications benefit a lot from that type system.
We've often had feedback from Python/JS shops using Rust that learning Rust had the side effect of teaching them systems programming. I think that's pretty awesome.
In Java, I don't think the type system gets in the way so much as its classpath. It's been a long time since I've done Java, but you'd have to get your jars in order and write an ant or maven script to do the compile/run steps (or at least a bash script).
I like that with Python I can just do pip install whatever and begin using the module immediately in my main method. In Java, It's not standard practice to copy jars into the system classpath IIRC, so you'd have to copy them around every time you want to do something new.
I need to investigate Rust's command line interface. Obviously, the more it can get close to
python main.py
vs.
javac -cp foo/Bar.jar:baz/Quux.jar Main.java; java --classpath foo/Bar.jar:baz/Quux.jar Main
the better it will be for this use case I mention.
Thanks to both of you for responding!
I'm a self-taught developer who only practiced memory-safe languages before and I never dared trying C because of its reputation of being too difficult. Rust taught me a lot of things about system programming I never though I will learn one day.
A big thank you to the whole Rust team for that !
Rust gives you much more control over how your program runs, but this necessarily means paying a price in expressiveness.
That said, Rust is very elegant for the work it is doing. Rust iterators and iterator adapters are a powerful and clean way of manipulating collections, and the `match` syntax is an elegant way of handling errors and Option types.
Rust isn't a scripting language in terms of its strengths, but it isn't far away in terms of how pleasant it is to read and write. I would definitely suggest taking a look at it next time you run into performance constraints in python: it's easy to create python-native modules & interfaces with tools like rust-cpython (https://github.com/dgrunwald/rust-cpython).
And some more recent writing from him as well: https://blog.sentry.io/2016/10/19/fixing-python-performance-...
Unfortunately, Rust doesn't have an interpreter so it's not as great as a general scripting language but realistically, there's only a few extra lines you have to learn (extern crate and fn main() {...} mostly) along with the compilation step. Imports work more or less the same in both languages but I find Cargo to be far superior to pip as a package manager. You won't run into dependency hell very often and deploying a Rust project is very easy.
As far as writing Rust, it has static typing so if you want to carry over your programming style from Python it will take some getting used to. The extra type information in definitions can be annoying at first (especially with generics and iterators) but overall provide an intangible productivity boost with type inference filling in the gap. Rust objects are basically pure data structures without inheritance so you have to get used to thinking in traits. That said, with conversion traits like Into/From and generics with trait bounds, I can write code as if it were a dynamic language with duck typing (once I've defined my types) with just a little extra boiler plate to tell the compiler the similarities between this duck and that duck. For example, you can write a function fn foo<T: ExampleTrait>(T bar) and call it with any argument that implements ExampleTrait or Into<T: ExampleTrait> for quasi duck typing.
Finally, I feel I'm far more productive in Rust than in Python, assuming I don't need any functionality in the Python stdlib not yet implemented as a crate. On average, Rust crates are slightly lower level than Python's libraries but I find that using them is no less ergonomic, albeit more verbose. Most of the time, if I haven't made a logic error, any Rust code I compile just works and i don't spend countless hours debugging silly errors like variable name typos or magic object changes. More importantly, in Rust, you can encode a significant fraction of your logic into the type system which will be enforced by the compiler. For example, in my STM32 motor controller, I encode physical units as types so I can't accidentally feed the motor a 1000 volts when I mean 1000 millivolts because the Into<ControlSignal> trait calculates the signal based on the input type.
I would definitely give Rust a shot but don't expect to be as proficient in Rust as quickly as Python. The community is younger (but no less helpful) and the ecosystem is still catching up. Most importantly, don't let Rust's status as a systems programming language deter you. It is a low level language but is a modern language that was designed from the ground up by very smart people who incorporated the lessons of the past. Rust's zero cost abstractions, compiler plugins, and tooling have started to bridge the ergonomics gap between high level scripting languages and system development.
I think that's important to those both familiar with rust (who want a clean framework) and those who aren't (who want a safe language without sacrificing productivity).
Honestly, you might not care about type safety. It depends a lot on the kinds of programs you write.
In general, a language with a good type system[1] brings several major advantages to the table:
1. If your program compiles, there's typically a 90% chance it will work correctly on the first try, even if you just changed hundreds of lines of code.
2. Big refactorings are much less stressful.
3. Emacs or Visual Studio Code (for example) can provide pop-up completions of names and methods.
4. Problems like mysterious nulls or data races can be prevented almost entirely at compile time.
One big downside is that it gets harder to do certain kinds of (runtime) metaprogramming and you need to use macros to get similar effects.
At this point, having worked professionally with Rust, I would recommend Rust to people who are really enthusiastic about the above benefits. But if you look at that list and say, "Meh, not worth it," then you might not want to adopt a younger language like Rust. Well, unless you need to get reasonably close to the metal without sacrificing safety, which is where Rust really shines. I just wrote a fast CSV sanitizer in Rust for processing gigabytes of input, and the compiler caught a subtle memory error when one CSV line spanned two buffer reads.
[1] For the sake of argument, a "good" type system in this context includes the ability to define custom collections with type parameters, it only allows NULL to appear where explicitly permitted, and it supports tagged unions with a nice "match" construct. This seems to be roughly the minimum feature set to get the full effect described above. Examples include Haskell, Elm, Rust, ML and (to a surprising extent) TypeScript.
but it doesn't prevent all of them
Yesod uses xss-sanitize [0], and their sanitize function does indeed prevent "javascript:" attempts. They even have a test case for it [1].
Playing around with it in the REPL:
Prelude Text.HTML.SanitizeXSS Data.Text> sanitize $ pack "<IMG SRC=javascript:alert('XSS')>"
"<img>"
Prelude Text.HTML.SanitizeXSS Data.Text> sanitize $ pack "<IMG SRC=\"javascript:alert('XSS')\">"
"<img>"
Prelude Text.HTML.SanitizeXSS Data.Text> sanitize $ pack "<IMG SRC=fine>"
"<img src=\"fine\">"
Prelude Text.HTML.SanitizeXSS Data.Text> sanitize $ pack "<IMG SRC=\"this is ok too\">"
"<img src=\"this is ok too\">"
[1]: https://github.com/yesodweb/haskell-xss-sanitize/blob/9a9101...
let always_forward = Route::ranked(1, Get, "/", forward);
aho-corasick, memchr, thread_local, simd, utf8-ranges
That functionality would have to be in std.
Regex in std would sure be nice, though.
It's not a hard-and-fast rule. It's something that I've experienced when trying to write small scripts in all kinds of typed languages. I keep going back to Python.
For larger scripty things I've found typed languages to work just as well.
> but this necessarily means paying a price in expressiveness.
I don't think this is true. You can have higher level abstractions in Rust too.
But lack of garbage collection and the need for lifetime annotations for references and memory management with Box, Rc, Cell, Mutex, etc, combined with the type definitions makes Rust a lot more verbose than any dynamic language or even statically typed languages with GC like Scala / Go.
That can't be avoided, but you have to be prepared. Rust code can be short and elegant, but it can also be tediously verbose.
someone offered a counter-example where a JSON parser had more lines of type definitions in Haskell than total lines of code in Python
If you link me the example maybe I can provide a Haskell version using lens.
[0] - https://github.com/SergioBenitez/Rocket/blob/master/examples...
In my experience, it is pretty straightforward to build your resources (as Jersey calls them) into various classes without any issue, even for applications with many API endpoints.
As for metrics, security and routing, most of these are canonically managed using annotations in Java. In Spring, application context XML configurations are used, but they are very big and not close to their actual implementations, which makes them get out of control quickly (and inconvenient when they aren't out of control). It feels like losing either way.
I'm a big fan of Coda Hale's work on Dropwizard, which uses metrics, security and routing through annotations, and it works very well for large scale production applications. I use these techniques at scale at Bazaarvoice with great success.
To be fair, I won't go so far as to claim the pattern is imperfect. It's just the best pattern I've used for web services to date, even at scale.
I would love for you to expound more on your own experiences in more detail!
edit: clarification on where I've used the pattern
Also, because the framework wants to create your classes you cannot simply send in the objects dependencies into the constructor anymore. You find yourself in need of a DI framework instantiating your classes. And suddenly the data dependencies of your app become very vague. Adapters get registered based on the presence of JARs and/or classes in directories which are scanned in runtime.
You have reinvented the work of the compiler and the language, and what do you gain? A big mess where you no longer have programmatic control. I cannot easily start parts of my application for testing, I have to rely on Spring or Jersey having some sort of metaprogramming magic triggered by some magic annotation supplied by a JUnit<->Spring integration module. Suddenly modules like JUnit and Spring aren't combinable without metaprogramming. You see this trend spreading like wildfire: "Is your lib Spring compatible?". We should instead be asking ourself why the heck unrelated libs have to be compatible with one another. They are supposed to be orthogonal, that's the point! To see an example of how ridicilous this can get, have a look here: http://stackoverflow.com/questions/35957287/cucumber-testng-...
I wouldn't go as far as saying that frameworks like Spring or Jersey are unworkable. But I believe you could acheive the same level of service a LOT easier if you wouldn't go metaprogramming crazy. Just consider the amount of time wasted trying to figure out why something isn't wired correctly, why some plugin is loaded, why your component isn't picked up, why some property isn't resolved etc. With normal programming, you fire up the debugger and step through the code. Solved in 5 min. With metaprogramming, it's a struggle on a whole different level.
This is exactly the reason why I refuse to use Lombok. It literally mutates your bytecode! That's a disaster waiting to happen, and a potential debugging nightmare.
I think that there is a fair balance where metaprogramming has great utility. For example, I'm a huge proponent of Guice. And yes, sometimes it makes debugging issues a little more complicated. OTOH, without it, there's a lot of boilerplate. Admittedly, DI in Java is a workaround for an inherent Java problem (boilerplate). Maybe it wouldn't be so useful in other languages/ecosystems. I have written services both with and without DI, and prefer it, and haven't had significant issues debugging or managing it. But it must remain scoped.
As for library compatibility issues, that is more of a byproduct of dynamically linked libraries than metaprogramming. If you're writing in a JVM language, you're kind of stuck with this (OK, you could shade your library and all its dependencies to get away from it, if you want to deal with the bloat / impact to the JIT'er).
Having said all that, your statements are arguing against metaprogramming, and I'm not really arguing for or against metaprogramming in my original comment. Simply, I think the pattern of defining HTTP resources in the aforementioned way is very intuitive from a software development perspective. If it can be done without metaprogramming, then great. But the presence of metaprogramming doesn't, in my mind, preclude the effectiveness of the pattern.
And then there is the problem of CGLib wrapping e.g. for @Transactional and other annotations. Of course, it only happens for other services calling into your wrapped service. If you call your own methods from within the service, you're hitting just the service instance, not the wrapped instance, which causes all sorts of headaches. Yet people still take this disaster further with Lombok/bytecode manipulation and aspects.
http://sparkjava.com/ looks very appealing; it looks very much like http://koajs.com/ which I really respect.
The latest incarnation of this problem is Spring Boot. It does some really far out stuff to your app, like making m separate metrics for each unique url of your webapp. That goes south pretty quickly. Not to mention how much longer the app takes to start, how much slower it is compared to coding the same functionality manually (x4), how much trouble people have getting the right mental model of how the app works (with instrumented transaction boundaries and stuff that you mention), how hard it is to debug (why did this endpoint return 404?) etc.
We did some POCs with vert.x and spark, and all of a sudden you see what type of overcomplicated mess we have been creating for ourselves.
Bytecode modification is not too bad IMO, as long as it's within reason. I use a notnull annotation weaver to add notnull assertions. Not Lombok, that's too much for me as well. Should be another language really, like Kotlin.
Metaprogramming does have utility. But it has a tendency to go overboard. I actually find Jersey to be a good example of a nice and clean API for defining resources. I recently tried to integrate it into a small app where we do all the wiring manually for clarity. The problem is that it's hard to stay in programmatic control. You define a resource. You realize you need a Service in your resource. You realize that the only way of getting it into your object is to inject it with the JAX-RS annotations. And so suddenly your service has to be a JAX-RS @Singleton service. You have to make your whole app JAX-RS compatible. The framework takes control. This is really bad IMO. A library which requires control over your program.
This is not an intrinsic problem of metaprogramming, but IMO it's a big problem of Java as it stands today. We need to give up on frameworks, make sure that all libraries are combinable and that they don't take control of your app. Doing so would make everything a lot better.
I wish I could just do:
new Jersey(config).addRoutesIn(new HelloWorldResource(new MyBackendService(dbDriver))).addSerializer(new JsonSerializer()).start();
This would then utilize some reflection when routing requests, but not take over my entire app.
Regarding the libraries, my point was not that you get conflicting classes (DLL hell sort of thing), but that we're building multiple products which want to take over the world. JUnit wants to be in control, and so does Spring. So when you try to combine them, you end up with a problem. Can you start Spring programmatically from a JUnit test? Well, not really. You can, but it's very involved. So they want to provide you with a Spring-JUnit integration. This is a really bad idea, since every linear combination of libs has to get an integration, like the link I included in my post. If all these libraries just tried to stay out of the way of the programmer and just provide a programmatic API, there would be no need for such integrations.
That makes me prefer minimal, magic-free tools, because even if it's sometimes a source of extra effort, at least I won't have to spend time fighting the framework. The fear of boilerplate seems strange to me. I'd much rather spend the extra ten seconds of adding a route manually through a line of code than no time at all if it spares me having to spend an hour figuring out why my ContextProvider isn't being registered by the classpath scanner.