SpacetimeDB: a short technical review(strn.cat) |
SpacetimeDB: a short technical review(strn.cat) |
The model of putting ECS like logic as code and putting all that code as stored procs solves a handful of db transaction issues that none of the major mmorpgs really had issues with 20 years ago when mmorpgs were all the rage and when computers were 100x-1000x slower than today. A lot of engineering is going to solving and cutting out these database round trips that were not even an issue 20 years ago and even less of an issue today.
On the other hand the actual hard parts about building a multiplayer game, such as roll back, physics, collisions, all of that sort of stuff, well this actually doesn't really help with any of that. If anything it makes all of those things even more difficult. I imagine that the developers of this game probably had to come up with some pretty interesting hacks that probably didn't benefit from their DB to get anything resembling physics to work.
Layered on top of that are the known anti-patterns around using stored procs within databases, which spacetimedb does't address at all. You can't really run your tests just on your client because the application logic is split between client and db-server leading to spaghetti code and poor iteration cycles for game design. The database itself also doesn't horizontally scale beyond a single node.
Just take a peek at the steam reviews for BitCraft. The recent ones are actually pretty scathing and it seems like, despite all this engineering, the developers haven't really listened to their playerbase. With all of this effort centered around building a custom database, users still can't, say, build homes for themselves in this game, something that you can certainly do in probably most all other MMORPGs.
I think this will go down as a cautionary example of not listening to your users and spending engineering effort on the wrong types of problems. I think one of the most important skills for an engineer is actually not just having good technical ideas but knowing the right types of problems to solve and, most importantly, listening to your audience, your users, and your stakeholders. I wish the developers the best of luck and hope they can pivot successfully.
Previously they talked about "testnet" and you use "energy credits" to pay for services. They also described aspects as being similar to smart contracts.
Naming the game "Bitcraft" circa 2021 is a big tell as well
I suspect the answer to that is probably nuanced. Spacetime probably gets more VC funding based on the DB rather than being a game. Trying to make a successful indie MMORPG without pre-existing IP is going to be an uphill battle in 2026 with or without fancy backend innovations.
Kind of bummed to see its essentially 2015-era React Flux in Rust around a mutex.
If you're interested more in our benchmarking methodology, you can read our technical blog post on the topic: https://spacetimedb.com/blog/benchmarking
This actually surprises me? I was a very early employee/contractor working on this product but left after 3 months. But this wasn't the shape it had when I was there. It would have been better described as a CoW tree of various sorts, which lets reads and writes advance on separate transactions and reconciles at commit with what I recall was fairly minimal lock-holding time.
When I was there there was no global lock being held while a WASM "reducers" was executing. They operated on their own separate transaction in their own thread.
I haven't looked at their source since -- and I really don't want to because I work on similar things I don't want IP contamination as last I looked they had some sort of weird license -- but there's been .. 3 years? of development on it since and I know Tyler well enough and he hired smart-ish people.
I had a lot of criticisms of what he was making and the way it was being made... but ehh....I would want to confirm in source that this claim is accurate, it makes me a bit suspicious of the accuracy and fairness of the rest of TFA.
Robot brain summarized:
"Your historical description may be completely accurate. Your request that the author substantiate the claim was warranted. But the source substantiates it extremely clearly. The blog’s concurrency conclusion is right. Its “hash table,” fairness, “cannot sync WAL,” and “questionable to call it a database” language is deliberately contemptuous and sometimes technically sloppy."
touche
"Technically sloppy" is an understatement. The article just gets several things plan incorrect about the durability guarantees, etc.
In particular, we are not making trade-offs. The assertions SpacetimeDB provides anything less than full durability guarantees is just wrong.
There seems to have been a fair bit feature churn across their whole stack. I’ve seen a fair few projects and tooling fizzle out cuz of the moving targets. It’s not really clear what is long term stable.
There is a lot of over promising and under delivering (features advertised in their 1.0 still not here). They own your whole stack, so deviating from what they provide is an unhappy path. If you need something beyond what is in their current feature set (not what is promised!) then you will probably have an easier time constructing such a system out of the hundreds of open-source components built for modular construction.
Their hosting has not been stable (no LTS either, so you are forced to live on the edge of their deploy schedule). Limited hosting options (only US iirc). Migrations and long term management don't seem to be in a great state either (external tooling needed). Trying to escape their stewardship and selfhost leaves you with a restrictive license…
It seems they are trying to follow in Michael Stonebrakers footsteps, first with copying VoltDB and now DBOS. It’s not clear what their priorities are, when they haven’t yet delivered a single solid product yet.
I do not really see how their business model is any different than VoltDB which was also open source with gated features. The VoltDB community died off for this reason, not sure why we would expect anything different to happen here. Unless they change their license, I would not expect a thriving community like Postgres or SQLite/LibSQL
There’s also an excellent paper called “Fair Benchmarking Considered Difficult: Common Pitfalls In Database Performance Testing”.
Benchmarks are genuinely hard. Modern systems and hardware are notoriously complex and have different behaviors across runs and with minor tweaks to performance settings.
PostgreSQL itself is remarkably slow in many configurations and leaves a lot of performance on the table due to its architecture and assumptions. But it works and people trust it. So they use it.
You may be interested in our technical blog post on benchmarking methodology for this benchmark: https://spacetimedb.com/blog/benchmarking
>The absence of side effects or stalls cannot be enforced by the type system ...
Side effects is probably correct, but stalls would imply wasm code that calls out to a long-time blocking function - that's generally quite easy to type-system-ify, and wasm stuff often does so with promise-like constructs. So there would be a need for spacetimedb-side markers for "this func might do HTTP", but that kind of marker for WASM-contact-able code is very much a normal expectation.
If they don't have that kind of marker, and do allow blocking calls in their beta API, then yeah - huge problem with that kind of internal structure (shared global lock), completely agreed.
Is there a more technical explanation post somewhere? I'm definitely curious how it works, and I like the core idea quite a bit - I'm fairly convinced that WASM uses like this are going to grow immensely, and "it's just a global mutex/single thread" reduces complexity so much that it's always interesting to see where it's possible to use.
The latency+throughout brings many questions though, like "are you holding responding to writes and any reads that occur after until changes are written to disk" and "if so I kinda assume that change log is constantly streamed to disk with a moving high water mark for responses". I can generally see how that'd hit such benchmarks, especially with non-dirty reads being near instant, but details are always interesting.
Please read our reply to this and other criticism here: https://spacetimedb.com/blog/benchmarking
For as long as I’ve known about SpacetimeDB the premise seemed to have its uses, but I don't really trust how new it is nor all the weird marketing behind it
If only that were true...
Nobody is asking for my advice here, but if I were asked I would tell them they have a blind spot wrt to outside perception. Perhaps they’re too much in their own bubble, or overly locked into an us-versus-them posture.
Perhaps you'd be interested in reading our technical response: https://spacetimedb.com/blog/benchmarking
Also you’re spamming the same comment up and down this thread. Did this post upset you?
Your comment was about whether SpacetimeDB was "just a hashmap behind a RwLock" and the blog post I linked addresses that question. Namely, it does have a RwLock, but that is by design and produces better results than the alternative.
Hm I wonder why people find this benchmark unfair...
I can say that your last paragraph is essentially correct though. It works exactly that way.
https://chatgpt.com/c/6a886e34-c784-83ea-8298-2d8215f5994a
https://claude.ai/share/658d00e4-550a-470e-b4a3-7a71a06222cb
> In particular, these statements should not be relied on:
> readers wait “up to 0.5 ms” because of parking_lot;
> all reducers/views are Wasmtime/WASM;
> reads normally happen through Views;
> external side effects are merely something reducer authors must avoid;
> durable/confirmed reads are an unusual opt-in intended only > for non-ephemeral data;
> 50 ms is the normal/current durability delay;
> SpacetimeDB has no meaningful replication;
> the benchmark numbers themselves are simply “not true.”
yeah my bot also flagged the parking_lot thing :-)
it was a hit piece with an unprofessional tone. but extraordinary claims get extraordinary responses
i do think dropping mvcc is an odd choice. do you at least let pure reads proceed while a single writer is active?
The article says that we don't provide strong durability guarantees. Pure ipse dixit. And IIRC he's already cleaned it up a little.
The shamelessness with which people published bogus claims about SpacetimeDB after this was really really astonishing. And so many were quick to dismiss the overall results based on some handwaving arguments. I wasn't prepared for it.
We should have held off on publishing until we had a deep technical blog on the subject.
If you haven't read https://spacetimedb.com/blog/benchmarking you should give it a read.
It seems like an odd choice perhaps, but I had the numbers to prove that it was simply measurably worse overall performance for most OLTP workloads.
IIRC, we don't let pure reads read the committed state while a transaction is underway. We can certainly do that in the future, but we're so far out ahead of typical database performance for most OLTP workloads we haven't even bothered.