Oxidizing Source Maps with Rust and WebAssembly

Oxidizing Source Maps with Rust and WebAssembly(hacks.mozilla.org)

207 points by mnemonik 8 years ago | 30 comments

MrBingley 8 years ago |

> Tom Tromey and I have replaced the most performance-sensitive portions of the source-map JavaScript Library’s source map parser with Rust code that is compiled to WebAssembly. The WebAssembly is up to 5.89 times faster than the JavaScript implementation on realistic benchmarks operating on real world source maps.

Wow. Some people have been wondering if Rust has a "killer app", but this could be it. Right now WebAssembly only supports non-GC languages (so C, C++, and Rust), and of those three Rust is the easiest to get started with. It looks really appealing.

steveklabnik 8 years ago | |

I personally am a huge advocate of this vision.

This stuff is a bit hard to get into at the moment, as it's early days, but as things coalesce, we're also working on making everything well documented and super-smooth to use. It's already reasonably easy, but we want it to be trivial.

jjtheblunt 8 years ago | |

Why do you say Rust is easier to start with than C or C++?

[Disclaimer : i'm old enough to dream in K&R C and later ANSI C, and then C++, but am more than enthused about Rust, so a genuine question]

seba_dos1 8 years ago | | |

I would guess that a language in where the compiler holds your hand so you don't shoot yourself in the foot is probably by definition a bit easier than one that hides traps for you behind corner every now and then :)

mden 8 years ago | | |

Maybe it's not easier to start depending on languages you are familiar with, but I think after the starting phase it gets easier by having standard libraries for common things like hashmaps, by having a much easier to configure build system, and by having a very solid package manager. Longer term into a project the actual language's features remove a lot of C or C++ snafus you can run into, though in fairness they also make certain common patterns obsolete which can be very frustrating until you get used to them.

tcoff91 8 years ago | | |

Perhaps he means the tooling for using emscripten with rust is superior, and easier to get set-up? (i'm not actually sure if it is, i've never tried it). I'm really excited about the future of rust and have really enjoyed writing a raytracer in it, but I don't see how it's an easier-to-use language than C or C++. Easier to write safe code in sure, but not easier to pick up for a newbie.

vvanders 8 years ago | |

Yup, allocation and performance are tightly tied for a variety of reasons. Languages like Rust that have a nuanced control over allocation will exceed in spaces like this.

kbenson 8 years ago |

Wow, I had assumed a lot more of the Rust runtime/stdlib would need to come along with anything, but the wasm-unknown-unknown target and aggressive culling of unused code looks to make this really competitive. That's awesome, and I'm really happy to have my predictions and assumptions proven wrong. :)

Edit: Some of the numbers:

Original JS: just under 30,000 bytes.

Closure compiler minified JS: 8,365 bytes.

New combined JS and WASM: 20,996 bytes

Roughly half of the WASM output is JS and half of which is WASM, since not all components in the original JS were replaced, just a specific subset. There is some duplication of functions that both the remaining JS still uses and the new WASM code does as well. Rust diagnostic and error messages appear to still be present inthe data section although unusable, so could be cleared out with better tooling.

steveklabnik 8 years ago | |

The smallest file we've gotten so far is 116 bytes. I went over it here https://www.reddit.com/r/programming/comments/7fn87w/rust_wa...

ridiculous_fish 8 years ago |

VLQ decoding is sort of subtle, and the one shown in the article under "Base 64 Variable Length Quantities" looks broken.

1. `shift` can grow large, and an overlong left shift will panic in Rust. I expect an input like "gggggggC" to crash.

2. `accum >>= 1` is wrong because it rounds down. It needs to be a round-towards-zero: `accum /= 2`.

3. `accum` is a 32 bit int which is too small. It needs to be larger so it can represent -2^31 in sign-magnitude form.

mnemonik 8 years ago | |

Thanks for the report!

We've already fixed some of these issues[0] but I didn't update the code snippets in the article -- woops!

[0] https://github.com/tromey/vlq/commit/3b41a2b6c778ce476eaaa28...

js2 8 years ago |

FYI, the Sentry folks have also written a Rust-based sourcemap decoder:

https://github.com/getsentry/rust-sourcemap

steveklabnik 8 years ago | |

The authors of it and the authors of this were talking on Twitter; the one in the article is pretty specialized, so that's why they didn't use this. They recommend Sentry's library for every non-wasm use case, basically.

couchand 8 years ago |

This is brilliant. Very excited to see more applications like this making great use of the possibilities of Rust+WASM.

One thing that stuck out to me was the exploded-struct callback mechanism for reporting Mappings back to JS. I've also been struggling to handle the low-bandwidth interface between JS and WASM. That wasn't a strategy I'd considered, but it's pretty neat.

It's simple enough and will work in this case, but unfortunately doesn't generalize very well. I've been exploring using well-defined binary encoding for this purpose (specifically Cap'n'Proto, but Protobuf or another binary encoding would work, too).

See an example I put together: https://github.com/couchand/rust-wasm-capnproto-example. I'm definitely going to go back and clean that up with some of the FFI patterns from this article.

eridius 8 years ago |

parse_mappings is incorrect. It has the following line:

  Vec::<usize>::from_raw_parts(capacity_ptr, size, capacity);

But size is wrong. size here is the number of bytes that JavaScript instructed Rust to allocate. It's not the size of the Vec::<usize>.

This should be harmless as the resulting Vec is immediately deallocated, so the only thing that size is used for is deinitializing values, and since the values are usize there's nothing to deinitialize. But it's still technically incorrect.

mnemonik 8 years ago | |

Great catch -- thanks for reporting this!

Fix is over in https://github.com/fitzgen/source-map-mappings/pull/15

eridius 8 years ago |

The get_last_error() snippet shows LAST_ERROR as being a static mut (which IIRC requires unsafe access as it's not thread-safe). But the parse_mappings() snippet shows safe access to LAST_ERROR using an API that clearly indicates it's not just an Option. I assume that at this point it's actually a std::thread::LocalKey.

mnemonik 8 years ago | |

Thanks for the heads up. It used to be a `thread_local!`, but switching to a `static mut` resulted in smaller code size. I just forgot to update the code snippet in the article.

> which IIRC requires unsafe access as it's not thread-safe

Yes, and also has no guarantees against mutably aliasing and re-entry.

Note that wasm currently has no shared memory threading (just message passing via FFI through JS and workers), so thread-safety isn't an issue to be wary of here, just re-entry.