Rust fact vs. fiction: 5 Insights from Google's Rust journey in 2022(opensource.googleblog.com) |
Rust fact vs. fiction: 5 Insights from Google's Rust journey in 2022(opensource.googleblog.com) |
Wow, I didn't even know this job existed. IMO Rust as a C++ replacement is fine, Rust as a C replacement has more trade-offs than I still care to make. C is still far simpler (you can still read K&R in one day and keep most of the language in your head), has faster compile times, and the pain points cough macros are still often pain points in Rust.
I think the biggest thing is that systems programming still requires a language that gets out of the way so you can focus on very technical problem domains where what the hardware is actually doing really matters. Rust is a language designed to get in your way and force you to create type abstractions. Adding too many abstraction can be exceedingly dangerous in an environment where not having a full view of how memory and hardware registers are laid out leads to even worse errors than just buffer overflows. IMO Rust makes this type of programmer more difficult just as C++ does.
The abstraction layer that one can build with rust allow the programmer to actually focus of the actual business logic instead of trying to get low level details right.
The innovation of Rust is the borrow checker, which is primarily of interest to systems programmers. If your primary interest is highly abstracted business logic, there are tools that don't require manual memory management or being pedantic about the different types of strings. You could just use Go, Java, Haskell, Python, etc.
Does this include all 193 cases of undefined behavior?
a) UB enables valuable optimizations and is important to keep (or even add) when performance matters
b) UB makes the language unusable/insecure to anyone but genius level experts and should be avoided
Whenever someone (including famous/relevant people like Dennis Ritchie [0], DJ Bernstein [1], or Linus Torvalds [2]) tries to suggest cleaning up, removing, or simply not adding new cases of undefined behavior in C/C++, the optimization experts come running from the other room screaming about how important it is that "signed integer overflow must be undefined" [3] or else things will run a percent more slowly (signed overflow being just one example of UB). Also there are people who suggest adding new UB to Rust [4].
So really, either Rust is significantly slower than C because Rust doesn't have the UB you're criticizing, or C could be a cleaner language without compromising on speed and the compiler writers and standards committees are wrong. You choose, but both options are considered heresy.
[0] https://www.lysator.liu.se/c/dmr-on-noalias.html
[1] https://groups.google.com/g/boring-crypto/c/48qa1kWignU
[2] https://lkml.org/lkml/2018/6/5/769
svd2rust is pretty good for having safe abstractions for hardware registers. That said, as an example, no, the type system doesn't prevent you from deallocating your DMA buffer while the hardware is using it--I don't think it's reasonable to add that to the type system (and the type system right now doesn't know about DMA).
I think re DMA buffer lifetimes, the easy approach is static buffers; they never drop.
const uint8_t zero[] = {
CHAR_GRID(
_,X,X,X,_,
X,_,_,_,X,
X,_,_,X,X,
X,_,X,_,X,
X,X,_,_,X,
X,_,_,_,X,
_,X,X,X,_
)
};
#define CHAR_GRID(c1r1, c2r1, c3r1, c4r1, c5r1, \
c1r2, c2r2, c3r2, c4r2, c5r2, c1r3, c2r3, c3r3, c4r3, c5r3, c1r4, c2r4, c3r4, c4r4, c5r4, c1r5, c2r5, c3r5, c4r5, c5r5, c1r6, c2r6, c3r6, c4r6, c5r6, c1r7, c2r7, c3r7, c4r7, c5r7) \
c1r1 | (c1r2 << 1) | (c1r3 << 2) | (c1r4 << 3) | (c1r5 << 4) | (c1r6 << 5) | (c1r7 << 6), \
c2r1 | (c2r2 << 1) | (c2r3 << 2) | (c2r4 << 3) | (c2r5 << 4) | (c2r6 << 5) | (c2r7 << 6), \
c3r1 | (c3r2 << 1) | (c3r3 << 2) | (c3r4 << 3) | (c3r5 << 4) | (c3r6 << 5) | (c3r7 << 6), \
c4r1 | (c4r2 << 1) | (c4r3 << 2) | (c4r4 << 3) | (c4r5 << 4) | (c4r6 << 5) | (c4r7 << 6), \
c5r1 | (c5r2 << 1) | (c5r3 << 2) | (c5r4 << 3) | (c5r5 << 4) | (c5r6 << 5) | (c5r7 << 6)People who say this somewhat perplex me. Yes you can get the syntax of the language down in a day, but that does little to stop you from running into your first Bus Error or Segmentation Fault within the first 30 minutes of trying to write any software, not to mention all the hidden errors/exploits you've put in your code that are only a platform switch or a compiler version change away from being found explosively. And you can completely forget trying to write a multithreaded C application, which basically confines you to very slow single-threaded code, completely tanking performance versus even the slowest dynamic language that supports multithreading, erasing any advantage for using C.
This is not a personal attack but when I have to try to come up with an assumed background for people who say this it usually involves some assumptions that the person isn't keeping in touch with the "real world" of some sort. I have trouble rationalizing it otherwise. Thus I'll usually ask what their background is when they say this to try to make sense of things.
The only places C is still the optimal choice is where C is already being used or in extreme platforms where there aren't good toolchains (various ASICs/rare 8bit microprocessors). There's zero reason to use it otherwise.
> the pain points cough macros are still often pain points in Rust.
Hygenic syntax checked macros are an entirely different animal than just string insertion/substition macros. I don't think this comparison is fair.
The abstractions can be more used like static interfaces you want to reuse. E.g. a byte stream interface, a regmap interface, etc.
Just the fact that Rust doesn’t do implicit integer conversions is by itself a huge win over C which has promotion rules that can easily trip you up when you are trying to exactly specify bits.
Except that isn't what most compilers expose, including the UB semantics.
One is in for a sea of surprises when trying to write portable C code and using K&R C as language reference.
If the answer is still No, then it can't replace C.
* Macros
* Ownership and borrowing
* Async programmingAsync programming is the area I would like to see the most improvement, especially in the standard library.
So much concurrent and parallel Rust code relies on third-party libraries because the standard library offers primitives that work but lack the "creature comforts" that developers prefer.
It would be really nice if the Rust standard library were to get structured concurrency similar to what Ada has:
https://en.wikibooks.org/wiki/Ada_Style_Guide/Concurrency
https://learn.adacore.com/courses/Ada_For_The_CPP_Java_Devel...
> The respondents said that the quality of the Rust code is high — 77% of developers were satisfied with the quality of Rust code.
Well, that’s exactly what I’d expect Rust developers to say. Nobody loves Rust more than Rust adopters. Would be interesting to see more objective measures of code quality (e.g. defect rate)
Also, the type of person to work on a Rust codebase might also be more likely to write high quality code in any language, as compared to the average developer (or even average Googler).
I wish there was more context to these, especially this one. For example, how much of this is perception compared to what they were used to (go?, Python?, C++?)? Or is it "any waiting is bad"?
From an improvement perspective, I'd also love to know why their builds are slow. Is it proc-macro heavy? Do they have wide and deep dependency graphs? Do they have large individual crates? And so on.
Some of the stuff people say about Rust reminds me of iOS users talking about Android. "Tell me you are operating from a place of near total ignorance, without telling me that you're talking out your butt".
See: the number of people, here, acting like you can't do raw pointers in Rust, or acting like it's militant woke youngins forcing poor big Google to adopt a safer, productive language.
― Upton Sinclair
Amount of dislike on HN for Rust is frankly unexpected, one part might be response to evangelization, but I've seen more hate on evangelization than actual evangelization. Sure, Rust ain't perfect but like C++ is even more imperfect. So that leaves me with job security in C++.
I do wish to know did Rust impact their velocity and by how much.
This tends to lead to people putting in unsafe code to work around a borrow restriction. I don't do that, but I don't have deadlines.
How? Explain please
> Carbon Language is currently an experimental project. There is no working compiler or toolchain. You can see the demo interpreter for Carbon on compiler-explorer.com.
But this is Google, and the people doing self-assessments were likely influenced by the context of operating in cut-throat bureaucracy where self-aggrandisement is a requisite to career progression within the org.
Whether or not this survey was tied to any performance evaluation (and from the article it's not even clear that it wasn't) the relevant thing is whether the employees knew without a doubt that they weren't going to be compared against one another based on their self-assessment
edit: I'm curious if the people downvoting disagree with my assertion that the survey methodology is flawed, or the assertion that it's unlikely to become as competent in rust in 2 months as you would be in languages you have years of experience with.
Upvoted even though I anecdotally disagree with your perspective based on personal experience. I wrote my first line of rust in March this year (just as a hobby), and now am one of the maintainers of a popular TUI framework (Ratatui). I feel just as productive or more than any of the previous languages I've written code in (over the last 30 something years).
I'm at the point now where I'm productive (took me over a month to even get to that point), but I still feel incredibly slow compared to Typescript. The compilation time doesn't help.
Anyway, thanks for the perspective.
I'm still skeptical that the survey reflects honest feedback given Google's culture, but perhaps I'm just biased from how long it's been taking myself and the rest of the team to achieve a higher level of productivity
The path Rust is going means async becomes viral, and is something I dislike a lot about JavaScript[0] and other languages I’ve worked in[1].
I’d love to see Rust avoid this trap.
[0]: I work in TypeScript in actuality not sure which to use here. It’s certainly by far the language I’ve used the most in my career now.
[1]: I remember it infected Python too and it was a pain as well when I did Python development years ago.
For those not aware of the history and looking for background, I laid it out here: https://www.infoq.com/presentations/rust-2019/ and here https://www.infoq.com/presentations/rust-async-await/
Those style systems are useful and have advantages, but they also have disadvantages. Not every tradeoff is a good call for every system, and that goes both ways in this scenario.
So, someone may like exceptions and green threads more than `Result` and `async` (and this is a completely valid PoV, even though I personally like the explicitness better), but thinking `async` is somehow special is just a conceptual mistake.
Edit to give an little more substance to the parallelism:
If you want to call a fallible function inside an infallible one, you MUST handle the result. If you want to use `?` then your function MUST return a Result.
Symmetrically, if you want to call an async function from a non-async one, you MUST `spawn` the future. If you want to use `await` then your function MUST be async.
The only practical difference between async functions and functions returning a `Result` is that `Future` is a trait, not a struct like `Result` (and that means that your future may have a lifetime that's not visible in your function definition, which is an endless source of confusion for beginners).
But Rust is not a language which can dictate its execution environment. It needs to be able to exist in a C-ish world, and that's not something that supports yielding. It's a shame, but at least you can write kernel modules in Rust.
I agree with you on this in case of high-level languages. Rust is not that, and wouldn’t be half as interesting that way — but by going the system/low-level language route it does have to make certain design decisions that are not ideal. They can’t do what java’s loom does as it requires knowing every method implementation, which is fine with a fat runtime, but is not possible in case of Rust, with plenty FFI boundaries, etc.
(Generally, I avoid this problem these days by avoiding threads in favor of other abstractions or multiple processes communicating over an RPC channel).
> So much concurrent and parallel Rust code relies on third-party libraries because the standard library offers primitives that work but lack the "creature comforts" that developers prefer.
This seems to be an repeated antipattern with a lot of languages/ecosystems, resulting in fragmented and half-baked solutions. A fully-featured async standard library involves making a lot of opinion-based decisions, and not everyone will be happy. But it's better for 80% of people who probably don't care that much, and nothing stops the other 20% from implementing libraries for their use-cases.
I think elixir's sigils are probably the closest thing I've seen to "routine, encouraged macro use." Since almost every application will end up with a bit of template lite almost-dsl pseudo language for something or other. They're simpler than defining a grammar & writing a parser and more maintainable than regex.
This creates a self-selection, where Rust lovers work on Rust projects and report utopian happy-go-lucky times. It's normal for most technologies.
Not necessarily, but it seems not unlikely.
> Is anyone who ever uses Rust a "Rust adopter" who is unable to give an unbiased opinion?
It’s still a relatively new and niche language, so, yes.
> Who would be able to give that opinion, in your view?
Nobody, and maybe that’s my real point, which is why I’d like some metrics to supplement the anecdotes. This especially applies to Rust, but I think also applies to any language.
Note that I don’t mean to imply that there isn’t value in anecdotes. There is.
> unbiased opinion
If the engineering manager after putting so much effort into switching to Rust, and trying to convince upper levels, putting their head at risk and after busting balls for months to make everyone learn Rust, comes into the office one day and asks if we love Rust, we the 77% that want to keep our jobs would answer "OF COURSE WE DO!!!!! COULDN'T BE HAPPIER!!!", with a big smile.
An increasing number of Android developers in Google are adopting Rust because of the org wide strategy rather than developer passion, so I guess the numbers in 2023 and 2024 would be more interesting to see.
50% of developers think they are as productive in a language they have four months of practice with as they are in a language they have fourteen years of practice with.
50% of developers think they are as productive in a high-performance bit-bashing-capable language as they are in a high-level glue language.
The people in this statistic are switching from languages they have years or sometimes decades of productivity in, and they're switching from languages like python and go and java. I see a lot of programmers who do similar switches never reaching productivity parity with C or C++. 50% of devs getting there in 4 months is amazing.
Anecdata but I found myself very productive with Haskell when I was learning it for grad school, to the point where I knew that if it compiled, it was most likely right.
I had similar experiences though not to that degree with Rust with very little time spent on it in comparison to Haskell. I feel a lot more comfortable sleeping at night over C or Python.
"Overall, we’ve seen no data to indicate that there is any productivity penalty for Rust relative to any other language these developers previously used at Google."
Weird enough we saw more junior devs pick it up faster. They had less preconceived notions and practices to unlearn and more willing to trust rustc. It's just that when they hit their stride they are still less productive than the senior devs.
> They had less preconceived notions and practices to unlearn and more willing to trust rustc.
this is really what I experiences: rust told me a thing or two about coding I never realized. And it took me pretty long to accept that :-)
In fact it's frankly unbelievable, I'd have to imagine these guys are coming from a C++ background.
You can see similar opinions expressed in this thread, like here, for example: https://news.ycombinator.com/item?id=36496654
I think C++ has its own legacy difficulties (which also make transitioning to memory safety tricky), and Rust's choice of borrow checking is only one (sometimes difficult) technique for getting these aspects. But there are almost a dozen other methods out there for getting memory safety besides RC, GC, or borrow checking.
I rather think that these other approaches aren't mature enough yet to enter the mainstream, and we haven't seen them yet.
I'll see about whether I can push an update. Thanks for the catch!
Given that #2 is talking about people who are all professional programmers and where only a small percentage of respondents previously knew Rust, that's pretty amazing to me.
This sort of surprised me, because rust felt a lot harder for me personally to learn than Go. But data is far more valuable than an anecdote so there you go!
Sounds incredible.
This being Google, it probably means something like "this C++ build takes 24 hours locally, but thanks to magical distributed build infrastructure it completes in 10 minutes, whereas Rust build takes 18 hours locally but even with magic does not complete in under 30 minutes, which is too long". That is important to Google, but it is almost completely irrelevant to anyone outside Google.
It is unclear to me whether improving rustc performance is the right solution to Google's problem. It is probable working on Rust integration to Google's build infrastructure is higher ROI than working on rustc.
Yes, this is the problem. Waiting is always bad for productivity. Even a second is long enough to lose a bit of focus. When that stretches out to 10 seconds, it starts getting tempting to, say, check Hacker News and lose your train of thought. I believe that most of the programs that I might be tempted to write in rust could be written in an alternate language with a compiler that is up to 100x faster. Of course this hypothetical language would have to be simpler than rust and would lack many of its features. As it currently stands though, I believe that it will be impossible to make the rust compiler 10x faster, let alone 100x faster so it would be nice if there was more effort to design alternative languages that build on what we've learned from rust to make something better.
Rust itself might as well be considered a highly constrained macro language at this point.
Based on my experience they're overselling how easy it is to learn and underselling the compiler speed.
Compilation is fairly fast these days. I would say it's faster than C++ feature-for-feature, at least for clean builds.
But on the other hand most people could probably learn all of Go in the time it takes to begin to understand the borrow checker.
Faster than C++ is of course very faint praise. C++ is also very slow!
Which we seldom due on most C++ projects, we rather rely on binary libraries and build only our own code.
Also when comparing with Delphi, Ada, D, or even Haskell or OCaml, it isn't that great.
You might feel like pointing out that Haskell or OCaml can be even slower, which is true, however they package multiple toolchains and a REPL, and as of today Rust still isn't as flexible in having multiple toolchains for different purposes.
The long dependency trees are part of it, but usually not too bad and only really bad the first time, since you don't have to rebuild every crate every time (I could be wrong, but it seems that way). I haven't been using it day in, day out though. I've installed a few apps via cargo, and have done some experiments for service applications, and Tauri as well.
As for the day to day use and how painful it is... I haven't had enough exposure to really comment on... it seems "fast enough" but I'm not running compiles often enough, simply because my knowledge and experience aren't really great in Rust. I've looked at it and played with it a few times, then I set it aside for months at a time and every time it's like I'm starting over.
Where I'm working now, there are some serious issues that may result in areas needing better start time on services, so that may be an opportunity to advocate for Rust. I've never really loved C or C++, so I'm less inclined to want to use them.
We do have plans for adding backends for unconventional targets eventually.
Ada is also used in embedded and low-level environments where the execution environment can be limited. The way that Ada "gets around" such limitations is through language "annexes". Annexes are optional language extensions for specialized use cases:
https://www.cambridge.org/core/books/abs/programming-in-ada-...
http://www.ada-auth.org/standards/22rm/html/RM-1-1-2.html
Rust partially does this already with [no_std] (https://docs.rust-embedded.org/book/intro/no-std.html), so the concept is not too different.
So, while macros are "discouraged" in Elixir, in practice they are very much encouraged by several prominent libraries. Picking on Phoenix is very easy because it's so blatantly bad in this regard (and others) but it's almost impossible to do useful things with Ecto if you go outside the macro bubble, etc., as well.
Example that shows how an eco system that definitely could have done stuff with macros (Clojure) has correctly decided that writing functions that take data is better than using macros:
Elixir and `Plug.Router`:
defmodule MyRouter do
use Plug.Router
plug :match
plug :dispatch
get "/hello" do
send_resp(conn, 200, "world")
end
forward "/users", to: UsersRouter
match _ do
send_resp(conn, 404, "oops")
end
end
(def router
(r/routes
[["/hello" {:get (fn [r] {:status 200 :body "world"})}]
["/users" {:name :users
:router users-router}]
["*" {:get (fn [r] {:status 404 :body "oops"})}]]))
But rustc's error messages helped it click that there might be a race condition on when the application returns from main and the background thread terminates. So it really needs to have the static life time to be safe. It's a small subtle thing but depending on the application it could lead to real bugs. I've definitely written variations of that bug in C before. A newer dev would have just accepted that flat out without arguing.
A lot of the features Rust offers regarding traits and types can be emulated in C++ with templates, but the way C++ does it is far more obfuscated. Seeing the same thing implemented in Rust helped me wrap my head around what some complicated template nestings were doing ("Oh, this is implementing traits!") in our C++ code.
Me and all the other developers I know complain endlessly about the tools we're forced to use. We complain to anyone patient enough to listen, including all the engineering managers. No one I know ever got fired or laid off for this.
For example, Rust std lib has blessed Mutex. Even though these can't be used in the kernel, it is still good to have them in the std lib for >90% of normal crates.
An example of this, is aliasing mutable references. Rust has been designed to assume that two mutable refs will never alias, if you attempt to do this it is instant UB. My understanding is that even creating the aliasing reference is UB, even if you don't use it.
Another example would be uninitialized values. In rust, the compiler assumes that values are always "valid" and initialized. Since you need to allocate memory before writing to it, you need some way to safely have uninit values, and this is what the MaybeUninit wrapper type is for. The wrapper allows you to safely have uninit values, and once you write to them you can tell the compiler that they are initialized, but if you tell the compiler before on accident, it is UB.
References also are guaranteed to point to initialized and valid data, and can never be null, though my understanding is that there is some uncertainty about the exact rules of this with regards to uninitialized values and the exact semantics may change in the future.
(There are also a lot more things that I don't know very much about!)
All of these things are assumed to never happen, and optimizations are performed based on that assumption.
The nice part about rust, is that it makes it impossible to represent invalid states using the type system!
For aliasing, you can only have a single live mutable reference at any time, attempting to create a second one while another is live is a compile time error!
For uninitialized values, you simply can't create uninitialized values at all in safe rust. My understanding is that the only way to create an uninitialized value is with the MaybeUninit type or by using raw pointers.
So rust still has heaps of UB, but it doesn't allow you to do it by default, so you still get some of the optimizations you'd expect.
I think there are some cases where rust is missing out on optimizations though, like with signed int overflow for example, and probably more that I don't know about!
> I think there are some cases where rust is missing out on optimizations though, like with signed int overflow for example
Before you push for UB in safe Rust, I politely suggest you write your own benchmark for this in C or C++. Try a few combinations of { signed, unsigned } X { 32 bit, 64 bit } X { gcc, clang }, compiled at -O2 or higher, and see what you get for results. Maybe throw in -fwrapv for some of the runs. My own conclusion on my own benchmarks is that UB advocates are mostly wrong.
However, Rust, the language has its issues on embedded.
Rust's ownership model is directly at odds with lots of embedded. These bits inside a register are owned by the ADC and these bits inside a register are owned by the DAC is not a happy thing in Rust.
Lack of arbitrary sized integers and how they slice.
Cargo. Quite annoying to deal with cargo and an embedded toolchain. The Rust embedded guys have done really good work if you're on ARM. If you're not, good luck.
That having been said, if you have to go implement something like Reference Counting in something not Rust, you will weep tears of blood debugging every single time your reference counts go wrong.
Embedded is engineering. It has tradeoffs. That's life.
1. A lot of the APIs make use of the typestate pattern, which is nice, but also very verbose, and might turn many people off.
2. The generated API documentation for the lower level crates relies on you knowing the feel for how it generates the various APIs. It can take some time to get used to, especially if you're used to the better documentation of the broader ecosystem.
3. A bunch of the ecosystem crates assume the "I am running one program in ring0" kind of thing, and not "I have an RTOS" sort of case. See the discussion in https://github.com/rust-embedded/cortex-m/issues/233 for example.
For example, I see inefficient patterns that are common in frontend world but have no place in an embedded system being promoted as "proper" way of doing things.
It appears most of the peripheral support libs (eg those that use Embedded-HAL traits) are not designed with practical ends in mind; I've found it easier for all I2C/SPI etc devices I've used to start from scratch with a `setup` fn with datasheet references, than DMA transfers. So, you have these traits designed to abstract over busses etc; they sound nice in principle, but are (so far) not useful for writing firmware.
I get a general sense that the OSS libs are designed with a "let's support this popular MCU/IC, and take advantage of Rust's type system and language features!" mindset. A bare minimum is done, it's tested on a dev board, then no further testing or work. There are flaws that show up immediately when designing a device with the lib in question.
So, at least for the publicly-available things, they're designed in an abstract sense, instead of for a practical case.
Use an actual different language. Ada, Rust, Zig, D, Lisp/Scheme/Racket, Tcl, Forth, etc. ... something other than C.
Don't preprocess C into a slightly broken other language that you wish it were. Use C as C, or use something else.
There’s one approach. I wouldn’t personally recommend using macros outside of include guards and file inclusion. Still, if you need more functionality, the methods exist.
So in practice my statement stands.
I recently told this to someone. The very next day my Elm code compiled just fine but it had three relatively tricky logical bugs which took me two hours to find and fix.
This was a rare enough occurrence that I remember it. Higher cosmic powers took note of my praise of strong typing and decided to teach me a lesson.
Depends on the project. Many commercial projects do vendor dependencies and build them too, because you can't rely on the OS version. Especially on Windows or with more niche dependencies.
Just compiling Boost takes 15 minutes - more than any Rust project I've ever compiled.
Not sure what you mean about multiple toolchains.
And yes exporting C++ from DLLs is compiler specific, which doesn't matter, as there is only one specific compiler version that is usually validated for the whole project delivery pipeline.
Plus we can edit and continue on C++ Builder and Visual C++, with incremental compiler and incremental linker.
There is a reason why so many companies forbid Boost.
Regarding toolchains, JIT, AOT, bytecode interpreter, REPL. Here, 4 variants to compile and execute code, depending on release requirements and developer workflow loop, each with its own sets of plus and minus. It is great when there is a choice.
The thing is, once you grok async in rust, other things make sense, like being able to construct futures by implementing `Future` on a struct. You don't actually need `async fn` to do async rust. It's just syntactic sugar, just like await is.
On a more serious note, "I want other people to write my code, but they're not following my standards" is rarely a sympathetic point of view.
It's a bad fit if you have enough compute work to keep all the CPUs busy. Then you're dealing with thread priorities, infinite overtaking and starvation, fairness, and related issues.
1. Rust doesn't force you to do manual memory management. Rust memory management is automatic by default and only if you really, really want to, you can do it manually.
2. Memory is not the only resource. The GCs in languages you listed only solve the memory management problem, but regarding the other types of resources, their ergonomics are often worse than C - you have to remember to close the resources manually and you get virtually no help from the compiler.
3. None of the listed languages address problems related to concurrency, e.g. data races. Ok, Haskell kinda avoids the problem by imposing other restrictions - by not allowing mutation / side effects ;)
4. Rust offers way better tools for building high level abstractions than Go, Python and Java. It has set a very high bar with algebraic data types, pattern matching, traits/generics and macros.
3. Rust only addresses problems related to data races, not as an example. All the other race conditions are still on the table. It is a good thing to have, but I think they are the least problematic, and easiest to solve part of concurrency issues.
4. All of these have been known for like 3 decades. There are plenty of managed languages with these, ML, OCaml, Haskell, Scala. But I think your claim is subjective at best.
Then you've got a different definition of "manual" than mine. Manual means that developer has to insert calls to allocate / deallocate memory and that the developer is responsible for proving the correctness of those calls. Automated means those calls are done by the runtime or by the compiler automatically, and the compiler makes sure they are correct. In case of Rust, those calls are inserted automatically by the compiler.
> memory/ownership details leak into public APIs
The fact that ownership is a part of public API is a good thing, similarly how it is a good thing to specify an argument is an integer and not a string.
> There are plenty of managed languages with these, ML, OCaml, Haskell, Scala.
I referred to the ones mentioned in the above comment, which mentuoned Java/Go/Python. Haskell/Scala/Ocaml/ML are quite niche even compared to Rust these days.
But even though Haskell / Scala might get close on some type-system features, they don't offer similar experience as Rust in other areas. Haskell is more restrictive in terms of managing state than borrow-checker, and Scala tooling / compile times has been always horrible.
> Rust only addresses problems related to data races, not as an example. All the other race conditions are still on the table.
This is like saying a statically typed language doesn't stop you from putting a string telephone number into a string surname field. Sure it doesn't. But despite that, the value of static types is hard to overestimate.
In practice, the borrow checking + RAII + Send/Sync rules can be used to make the other types of concurrency problems very unlikely by properly modeling the APIs. Sure, no language can protect from all concurrency problems in general, but at least Rust gives you some good tools. For instance it is trivial to forbid concurrent access to something that shouldn't be accessed concurrently and let the compiler enforce that. Now try enforcing that in your "business oriented language of choice". In my experience the majority of concurrency related problems in real large-scale software development happen when some code not designed to handle concurrency accidentally becomes executed concurrently because developers don't realize something is shared and mutated at the same time. Another type of common issue is with communicating concurrent threads of execution, when one sends a message but the receiver is not there on the other end because of premature exit e.g. due to error, leading to a deadlock. Rust protects from those really well.
For embedded software, the underlying code basically reads and writes a bunch of registers. Unsafe memory access with side effects. The benefit of using rust here is that you can easily model these access patterns to make an api that cannot be abused. So the driver reads and writes addresses and the user code operates through the driver with all the benefits of ownership at hand to avoid race conditions and other foot-guns.
So, in this way rust does indeed “solve for” ownership of devices. You can’t have two threads (or interrupt handlers) mutating the same device without satisfying the ownership rules.
The trouble with this is that abstract 'devices' don't necessarily map neatly to the underlying hardware. Configuring peripherals on a typical microcontroller typically requires setting flags in a bunch of random registers which don't necessarily have neatly separated responsibilities.
Take PWM as an example. Is there a PWM 'device'? Is there a PWM setting for each port, according to some abstract representation of ports? What about the timer used to generate the PWM output? Does the PWM device own the timer, or does the timer own the PWM device? Any such abstractions cause more problems than they solve. You really just need to think carefully about how you are manipulating the underlying hardware.
First layer gives you safe access to the hardware registers. For example, ensure atomic/synchronized access, forbid invalid/reserved values. Name the flags/bits to reduce human mistakes (reg |= Prescaler::Div8.
You can still miss-configure the PWM/Timer settings of course.
The second layer gives you a safe driver interface. Giving you all the options to configure a timer for a some PWM settings for example.
To be honest, this is what I've taken away from this conversation so far: it doesn't seem like anything will satisfy your desires here.
> which is why I’d like some metrics to supplement the anecdotes
This conversation started with you not liking that the measure of quality is subjective, which is fine. How would you objectively measure code quality though? What metrics would you have preferred to see, other than the ones in this post?
I don’t have a problem with this Google poll. I think it is valuable.
What would satisfy my desires is exactly what I stated in my original comment: something more objective, like defect rates of Rust code bases compared to non-Rust ones. Metrics like these have their owns problems, but would be a nice supplement to the opinion-based poll.
Neither the objective metrics, nor the opinion poll can provide a complete picture, and neither is a substitute for the other. Both would be awesome.
> It’s still a relatively new and niche language, so, yes.
If nobody is able to give a non-partisan reply, I don't see how you could ever be satisfied.
> like defect rates of Rust code bases compared to non-Rust ones.
Cool, thanks. That does seem like one that is more objective, though there are confounding factors in that too, because sometimes defects lurk without being detected. This stuff is hard!
Google did put out something about this specifically on Rust (and others) use in Android, by the way, you might find that interesting: https://security.googleblog.com/2022/12/memory-safe-language...
That is, a lot of people expect that the compiler is a tool that must do what they say, not a collaborative partner. Back in the IRC days, people used to join, and be like "Rust won't let me do X. X is obviously okay. How do I get the compiler to shut up and do X?" and the reply would be something like "what if there was another thread? that's what Rust is saving you from" or "Rust's pointer rules are different than the language you're used to."
Whereas folks from scripty backgrounds don't have preconceived notions of "I should be able to do this in this way," and so tend to trust the compiler more. Heck, that's why a lot of them are learning Rust instead of C; they know the compiler can help them out, and C's compilers cannot to the same degree.
Now, that doesn't mean that scripty people don't struggle, or that C and C++ developers don't have their own advantages in learning. Just that I don't think it's straightforward to say who has the overall easier time. It's more of a "having learned something like C or C++ does not universally advantage you."
(In fact, if you decide to be responsible about what you're telling your poor compiler to do and use a `size_t i` instead of an `int i` in your for loop, since iteration can never go negative and, if overflow happens, overflow on a signed type is worse since that's UB, I think you just made your for loop slower since all the optimizations the compiler was going to generate from inferring that iteration won't be infinite because that would overflow the `int i` and that would be UB so it can just ignore the possibility that the iteration would be infite went out the window and...man I really need to switch to Rust.)
((I've pretty painlessly learned and love Rust, btw. I'm still using C at home lately though cuz... masochism.))
I'm very top-down, always starting by sketching the API I'd like to see for the feature, and then filling in the implementation. A lot of people I know from C++ are bottom-up, they toy with the implementation and then go up to the API. I found that what they like in systems programming is being close to the machine, while my interest lies in the OS boundaries and interfaces, not really the hardware.
I'm thinking maybe those different ways of approaching programming make it easier or harder to learn rust.
The new experimental languages with effect types might be able to give us the best of both as they actually expose what you are talking about as an abstraction at the type level. We will see.
Currently, none of those trait implementations can be async because that would change the function signature.
So the only option you have in a trait implementation that needs to call a library that happens to use async apis internallt, is to "block_on" the async. Unfortunately iirc blocking on an async like that is executor specific and your impl must pick a concrete async executor which may be different from the one used elsewhere in the program.
The reason why we don't want to be polymorphic between fallible and “infallible” functions is that we put a clear social hierarchy between “properly handle error cases” and “panicking”. Using panics instead of `Result` make error handling much less cumbersome in Rust too, but we've clearly internalized that this isn't something you should do for real. Symmetrically, I'd argue that there's no much point to have both async and sync interfaces, if the user just want the quick and dirty approach, `spawn_blocking` is barely more typing effort than `unwrap`. The broad developer community (most of which having programmed well before Futures/Promises went mainstream) disagrees with me on that point, but that's a cultural thing.
Oh, and Result/Options aren't the sole “stack coloring” thing in Rust either: if you have an “owned” variable down the stack, then you need to either change your entire call stack to take the variable “by ownership” instead of “by reference”, or you can `Clone` it.
And you known what's even worse than this: `&mut`, because then you have no quick-and-dirty fallback (cloning and mutably borrowing the clone variable means you're now dealing with a reference that has a much shorter lifetime than the original one and it only works if your reference doesn't leave the current scope).
As a personal anecdote of someone that's been doing Rust full time for 6 years now, I've encountered the `Option`/`Result `/“owned”/`&mut` function coloring problem many times, and exactly zero times the “async/blocking” function coloring problem. Yet for some reason people are obsessed by an old rant about JavaScript's callback hell ¯\_(ツ)_/¯
This is not true of async/blocking — there can be semantic differences between two, otherwise equivalent implementations, and it is a recursive problem as I mentioned — how should an async-block-async call chain work exactly? Java can decide it at runtime, while rust with its tradeoffs meaningfully can’t - but it is a net negative tradeoff in its case.
Exception that now you need to bubble up the error condition comming from these functions (if you function didn't have other error already).
And in fact, adding those call do change things about how the calling function is run (yields point are inserted and the function isn't being run sequentially anymore), it's just not visible in the code, exactly like exceptions vs explicit error return values.
Except*, damn autocorrect.
Minor nitpick, but it's an enum.
https://github.com/rust-lang/rust/blob/master/library/core/s...
I actually really like the borrow checker as a tradeoff, I think it makes code much easier to understand and it makes all aliasing bugs impossible. The removal of aliasing bugs is I think an undersold benefit of using rust.
* Vale combines generational references and linear types with regions to eliminate overhead.
* Verona lets you divide memory into regions which can be backed by either arena allocation or GC. I think this is promising because for most GC regions you can completely avoid the collections.
* Cone lets you put borrow checking on top of any aliasing memory strategy, so could be something like the best of all these worlds.
* No language is doing this yet, but RC plus regions to eliminate the refcounts, then adding in value types for better cache usage, could be a real winner.
On phone so it's hard to get links, but you get the idea. The nice thing about regions is that they allow composing borrowing and shared mutability, something that the borrow checker struggles a bit with. Regions let us alias freely, and then freeze an entire area of memory all at once. Not that Rust isnt a good approach (it is!) but there are some easier techniques on the horizon IMO.
I'd be delighted to see it, because right now I am not aware of any practical way to have memory safe regions without static tracking of borrowing from these regions. It's either that or runtime checking.
It uses region-based static analysis without borrow checking: it doesn't impose aliasability-xor-mutability per object, or even per-region.
Though, if you'd like to move the goalposts further to no form of borrowing at all, then I recommend looking at languages like Forty2 and Verona, they might be what you're looking for.
I have already spent too much time trying to compile Vale compiler which is a weird mix of Scala and C++ with a small Vale driver. Once it is actually written in Vale without segfaults, I'll revisit the language again.
Thanks for the Verona recommendation.
As a parallel. A team using Scala at Amazon likely uses it because everyone (let's say 90%) was on board. It's just not something you force on your team unless there is existing interest.
Finally (an perhaps more importantly), the parent comment also mentioned older codebases. It does not seem likely anyone of those 1000 is currently doing maintenance so much as completely new projects/features. This tends to be work software developers enjoy more irrespective of language. So if you were pulled from maintenance in C++ to work on something new in Rust I'm pretty sure you'll say Rust is great just because you feel more productive.
Google hired Ferrous Systems to train employees, as well as writing their own training curriculum. That sounds to me like people who would not otherwise use Rust being asked to use it at their job, and their job investing in their skills because they wouldn't or hadn't done it on their own. Is that different than "forced to use it"?
> It does not seem likely anyone of those 1000 is currently doing maintenance so much as completely new projects/features.
Google has been using Rust in android since 2019. That's four years. That is of course not "legacy" in any large sense, but at what point for you is something legacy? Does none of that work over four years count as "maintenance"?
> So if you were pulled from maintenance in C++ to work on something new in Rust I'm pretty sure you'll say Rust is great just because you feel more productive.
The start of this sub-thread, and a lot of the discussion inside of it, implies that people are using Rust simply because they want to, and not because it provides actual advantages. Is your position here that the sole advantage of Rust over C++ is that since projects are newer, they're better to work on? And if so, is that advantage illegitimate?
Furthermore, at least in 2023, only ~25% of respondants are under 25. I'm not sure what counts as "younger" to you, but 37% are over 35, so it would seem that the survey skews older, not younger, to me anyway.
EDIT: since you're now flagged into oblivion (I tried to vouch for you but it didn't work), that statistic is what they changed "most loved" from. It counts people who have used Rust before, and want to continue using it.
Most of people work in other languages.
Did Google only interview these Rust-loving developers, and none of the people they're supposedly pushing Rust upon?
A libraries next version which switches up some internal representations memory handling should ideally not mess up your application, but it also mandates a higher refactor rate when you are only working within your application’s boundaries. These are worthwhile tradeoffs for the niche rust is targeting, but not for every use case.
I’m not saying Rust is a bad language, I really like using it for its intended niche of complex applications where absolute control is needed, like a browser engine. But it is not a panacea and I would definitely not choose it for a CRUD webapp.
It doesn't have to because internal memory representation can and should be abstracted out, and Rust gives a plethora of tools to do that.
Your argument works against against static typing in general. The next version changes the address representation from String to Address (in managed language) and messes up your app. That's the same thing.
The simplest concrete example, I guess, would be a function that returns &'str. There are plenty of Rust APIs out that have functions with this type signature.
I don’t think this argument works. Ad absurdum a very strong type system would even specify the implementation itself, making any change breaking — is that good? No, it isn’t as the useful property of the type system is no longer there. I don’t agree that this usefulness line is behind “ownership annotations” — that’s what you would have to convince me of.
Edit: I missed that the original person I responded to was talking about being paged when a problem arises and not about memory performance. I'm still curious though if Rust memory guarantees give the programmer better tools for dealing with memory paging.
I have an unusual application, a metaverse client for big 3D worlds. It has to deal with a flood of data while maintaining a 60 FPS frame rate. It's essential that the rendering thread(s) not be delayed, even though other background threads are compute bound dealing with a flood of incoming 3D assets. This does not fit well with the async model.
This sort of thing comes up in games, real time control, and robotics, but is not something often seen in web-related software.
I think you're not familiar with web-related software.
> It's essential that the rendering thread(s) not be delayed, even though other background threads are compute bound dealing with a flood of incoming 3D assets.
This is exactly how the browser behaves, and why Javascript needs to be entirely async on the main thread.
For your application it should be very easy to spin up a render thread (pinned to a single core or whatever priority mechanisms you want to use) that loops, and use message passing to get the results from Tokio based futures.
If you create a library that uses async, you're forcing everybody that uses the library into async as well (with the same executor).
If somebody writes a library now that's generally useful but uses async, it forces others to use async or rewrite the library themselves.
On the one hand a lot of people put this down as whining about free code, which is somewhat true, but the infectious nature makes the whole ecosystem less useful if you want to build something non-async.
If you request here or via the email at my web site (in profile), I can provide a more detailed example from a test I have.
(note to self: see fn test_basic_sql_connectivity_with_async_and_tokio() .)
> Is your position here that the sole advantage of Rust over C++ is that since projects are newer, they're better to work on? And if so, is that advantage illegitimate?
No, to rephrase, I think that maintenance work on average feels less productive and more grueling because you can spend 3 days debugging for a single line change. With a new project (or any new feature) you get to write your own thing which feels more productive.
I'm not saying people are less productive with Rust than C++. I am pointing out that there is a natural bias in the type of work that those two languages are being used for at Google and that this bias will impact self-assessed productivity.
> Does none of that work over four years count as "maintenance"?
Respectfully I think you are building a strawman because you (likely) work with Rust and enjoy it. Google is built on a staggering amount of 10-20 years old C++ codebases that have seen several runs of refactoring at this point and stand on the critical path of several of their most important products. Working on those is inherently slower and more meticulous than writing new Android features (even if 4 years old) in Rust.
I'm just saying that Rust fans are currently self-selecting by choosing to work on Rust projects which skews the satisfaction numbers against other languages.
That doesn't have a lot to do with previous experience - with Rust being so new, __MOST__ developers, even fans, don't have much experience with it.
I do not understand how what you're saying here relates to this post, which does not seem to be saying the things that you are saying. I do not understand how to reconcile "we paid to train 1,000 people on the job and here's what they thought" and "only 13% of people had Rust experience before this" with "Rust fans are currently self-selecting by choosing to work on Rust projects."
But this is a discussion about Rust! And my entire point is that async/await is entirely consistent with Rust's overall design.
> But exceptions (especially checked exceptions like in java) don’t have that problem, and are exact analogues to Result types.
Unchecked exceptions don't have this problem (but checked exceptions do), and that's exactly my point. Async/await vs green-thread is exactly the same trade-off than Result vs exceptions: one is “simpler to use”, the other is “simpler to read”. After years of programming, I personally came to the conclusion that we spend more time reading code than writing it (and it's going to be even more true in the near future with LLMs) so I lean on the Result/await side of things, but I don't have fundamental objections against green thread and exceptions.
I do have a fundamental objection against the idea that “async” is somewhat special.
> This is not true of async/blocking — there can be semantic differences between two, otherwise equivalent implementations
Result vs exceptions have also a significant semantic difference: unwinding, and especially the fact that you can trigger unwinding at any point. This is a significant issue when you're dealing with pointers.
> how should an async-block-async call chain work exactly?
I don't really understand this question. An async function is just a regular function that returns a Future, and yes since there's no marker for blocking function you can definitely call one inside an async context, even though it's often a very bad idea from a perf PoV (well it depends, locks are mostly fine, but you need to use them with caution).
In fact, the `async` marker in functions doesn't bring much (again “async function are just regular functions that return a Future”), and it would make much more sense to have a `blocking` stack-contaminating marker on function that call a blocking syscall in order to avoid performance problems due to those, but we can't have nice things because something Path Dependence something…
Of course it doesn't happen too often, but nor does your recursive async/blocking function example (been using async/await for a decade now, and I've never encountered the issue in actual code) and I suspect that for most purpose, using `block_on` in the blocking function is the sensible thing to do, and it has the same role as a `catch`: the upper function has no way to know there was actually some async stuff under the hood.
This is IMHO a good thing. If it returns a reference to T, it means it still owns it and allows only temporary usage of it. This is semantics of ownership and does not have anything to do with memory management. If you wanted to allow sharing for unspecified lifetime, sure, you can. There is Rc/Arc.
I've fixed plenty of bugs in code written in managed languages, where a reference to T was handed out from a library (because there is no other choice - everything is a reference) and then someone stored it for longer than it was valid, leading to a logical equivalent of use-after-free.
E.g. get an entity object managed by Hibernate. Pass it up outside of the context of Hibernate session. It will likely blow up because the object references a session that's now closed. Rust ownership model would prevent exactly that problem.
I find this "flexibility" of managed languages actually a problem in large codebases, similarly how flexibility of goto is universally considered bad. It severely hinders maintainablity. It allows to pass references freely and create implicit, complex, often cyclic, reference graphs which are very hard to reason about.
In my Rust code 99% of objects don't need shared ownership. But managed languages make shared ownership the default, optimizing for the edge-case.
BTW, your statement can be rephrased to: "If version 1 of your library has a function that accepts a reference to T, then that constrains your choices of values of T more than it would be constrained in a dynamically typed language."
You may say that you can use Any / Variant / Object in a statically typed language to overcome that limitation. True, and similarly you can use Rc/Arc/Copy types in Rust.
This is all the same thing. It just takes static typing to the next level. Not only it allows to express constraints on values, but it also allows to express constraints on time they can be used.
This is dogmatic. In can be a good thing sometimes, in some domains. In other instances it can just be a pain.
Let's say I'm using a 'names' library that provides the following utility function:
first_name<'a>(name: &'a str) -> &'a str
first_name<'a>(name: &'a str) -> Cow<'a, str>
func firstName(name string) string
Now let's relate this example back to what you originally said in response to kaba0:
>> [kaba0:] A libraries next version which switches up some internal representations memory handling should ideally not mess up your application
> It doesn't have to because internal memory representation can and should be abstracted out, and Rust gives a plethora of tools to do that.
The above is a simple example of why this is not true. Any time you write a function that returns a reference, you are limiting the changes you can make to internal representations (both in the library itself and the calling code) without making a breaking API change.
Please don't respond by saying "this API was badly designed in the first place!" Most languages don't give you the opportunity to design APIs badly in this particular way. If all APIs were perfectly designed on day one then of course we'd never have to worry about API changes.
Again, none of this is to bash Rust. I just think it is important to be realistic about the downsides as well as the upsides of Rust's ownership system.
first_name<'a>(name: &'a str) -> &'a str
Similarly by writing this in Go:
func firstName(name string) string
Then you want to change the semantics (the contract) of the function, break your promise and you complain you have to change the signature. You can't do that in any statically typed language.
In Go's case, if you suddently wanted to change the memory representation of name to something else than string, e.g. to a name struct:
func firstName(first_name name) name
If you don't want your caller to be affected by lifetimes, just don't specify them in the signature:
fn first_name(name: String) -> String
You may say it might be slow because it forces a copy, and forces a particular string implementation. So as I said, Rust gives you tools to abstract out the implementation details:
fn first_name(name: impl AsRef<str>) -> impl AsRef<str> {
// all are correct:
// return name;
// return "foo";
// return String::from("foo");
}
> I just think it is important to be realistic about the downsides as well as the upsides of Rust's ownership system.
So the downside is that it offers you more choices and allows to express more constraints in the signatures, and you can choose wrong. I guess that's quite ok in a general purpose language that wants to be applicable to many different niches.
>Please don't respond by saying "this API was badly designed in the first place!" Most languages don't give you the opportunity to design APIs badly in this particular way. If all APIs were perfectly designed on day one then of course we'd never have to worry about API changes.
I did edit in that paragraph ~5 minutes after submitting the comment, so apologies if you missed that.
>fn first_name(name: impl AsRef<str>) -> impl AsRef<str> {
As I said previously, "...unless there is to be a ban on functions returning simple references..." If you're willing to eat all that extra generic ceremony, then yes, you can make Rust APIs that are flexible w.r.t. ownership. However, you can't control the code style of the libraries that you're using.
> With this signature I actually can change the name representation from String to any type that can be exposed as a slice, with no additional runtime penalty like virtual calls, which you'd need otherwise in Go/Java.
Off topic, but you could do this in both Java and Go via generics.
Ok, I agree. But this is just as well a problem for any language that specifies API in some way. So it is really a decades old debate about static typing disallowing to change things easily. I mean, there might always be certain case when the library author has to change the signature because they made it too restrictive. It just happens that Rust allows you to constrain things that might not be constrainable in other languages, so I guess the chances for that happening accidentally are somewhat higher. But generally I like going from more concrete / restricted to more generic when needed rather than the other way round.
> Off topic, but you could do this in both Java and Go via generics.
I don't think so you can achieve same level of flexibility and efficiency at the same time.
Go doesn't have function level generics, so you'd need interfaces and runtime polymorphism as usual (and runtime penalty).
And in Java you cannot add a new interface implementation to a builtin type like String, so the idea of switching from String to any other type won't work. There is much more upfront ceremony needed to add such flexibility (defining custom interface + implementations and using them from the beginning).
>Go doesn't have function level generics
Not sure what you mean by this. Go functions can take generic parameters that satisfy a given interface (at compile time). E.g.
func firstName[T AsBytes](name T) T
>in Java you cannot add a new interface implementation to a builtin type
Yes ok, fair point.
func firstName[T AsBytes](name T) T
Yes. However, because strings in Go are immutable and byte arrays are necessarily mutable, you can’t safely convert strings to byte arrays (or vice versa) without copying, so it would not necessarily be a particularly useful interface.
By the way, this was an interesting discussion. I do see your overall point more than I did at the beginning of it, although I don’t think we completely see eye to eye on the costs/benefits of lifetimes.