TurboKV: Insanely fast Rust key-value store(github.com) |
TurboKV: Insanely fast Rust key-value store(github.com) |
> Appended to the WAL without a per-write sync
So… it’s not durable? Durable doesn’t mean “survives a process restart”, it means “durably saved to persistent storage”. For example, this “durable” mode wouldn’t survive power loss.
Being able to recover a db without corruption beyound losing the last few writes is a pretty useful feature, and buys a lot of performance, but it would be better to label that clearly, as a reasonable expectation on the durable() preset would be for it to be Durable.
mmap is nice but it doesn’t support durable semantics in the way that we usually mean with databases.
if a write is acknowledged it should not be forgotten, which is not what this is.
However, this doesn't appear to be the case here...
Unsurprisingly, performance goes to crap when sync is enabled.
This is pretty old now but has some useful fsync/sec numbers which can be completely divorced from other I/O performance https://www.percona.com/blog/fsync-performance-storage-devic...
I have nothing against this being a non-default option in a db/kv engine but anything advertising to be durable and not fsyncing by default is something I would stay away from. To me it's like a litmus test of how well the author knows/cares data durability and not destroying users data.
Insanely fast was making 8 bit games possible at all.
https://github.com/ncruces/go-sqlite3
And I'm still having fun.
If not, please just link the repo and its own title, no need for hype.
This DB doesn’t have a transaction API.
> TurboKV's persisted Bloom-filter format uses hardware AES.
Also, built-in LZ4 compression.
I would expect SIMD to be used for scans.
It seems pretty ubiquitous yet not a part of the core rust libs.
I'm not trying to be pedantic, but this split over async runtimes was what originally turned me off of rust years ago and it still seems to be an issue.
Sharding would be so easy! Just run "split" and spread the chunks over a bunch of servers, then run "recutils" with "parallel" to retrieve!
Truly webscale.
There is no query primitives in the CODASYL Data Model. Only a Data Manipulation Language.
Because "relational" databases didn't really happen until the early '80s - nearly a decade later. (SQL came directly out of work on CODASYL).
We had a Database Task Group, long, long, before we had the concept of a query primitive.
Of course, if you need SIL2 type reliability then you need to assume any given hardware component can spontaneously combust and become a total loss, at which point the data loss caused by a power cut is a rounding error.
Several years after they become commercially available?
My experience with small UPSes is they tend to cook the batteries and you don't find out until they switch the load and the battery doesn't hold up.
Large facility UPSes tend to do better, but automatic transfer switches have a tendancy to fail ocassionally. If you're hosted in many locations, it's not unusual to have a couple ATS failures per decade.
All that said, unexpected power loss is certainly one reason that writes may be lost, but OSes crash too. Disk firmware can also crash, but if thst bricks the disk, writes in progress don't really matter. Sometimes cabling fails. Or you get a uncorrectable ECC error (which will typically cause an OS panic... unless you're running a very fancy OS, but if it's in dirty disk backed page, even a fancy OS wouldn't save you)
Plenty of applications don't need or want to pay the cost for full commit to disk, but calling something durable when it's not committed to disk is inaccurate.
And that's before we get into the whole thing where the OS and the disk like to return success when things haven't quite finished.
Doing a hard shutdown or tripping over power cords seem much likelier local scenarios than any spontaneous combustion of hardware components.
If someone demands that the database should use fsync and only respond with success once the write finished, it is not some arbitrarily high reliability demand that needs to be implemented using reliable hardware. In fact, the entire point of implementing the power loss protection in software is so that you don't need perfectly reliable hardware. The power loss event turns into a downtime event which is often completely acceptable.
The requirement to have infallible hardware only emerged because the software refused to do its job. Infallible hardware is not a requirement decided by the user, it's a requirement decided by the developer of TurboKV to intentionally restrict his software to exclusively operate in a reliable hardware environment.
The fact that the user specified durability of the KV store during power loss does not make the user obsessed over hardware reliability, the software shifted the burden onto the hardware and forced the user to deal with this mess.
I don't know how exactly TurboKV works so let's talk about a hypothetical software instead.
Let's say the software cannot survive a power loss event and just corrupts the database. If the user wants to operate the software, he is forced by the software to operate it in an infallible environment where power loss can never occur. Based on how the software was designed, power loss is a catastrophic event. The SIL2 type reliability you're talking about only makes sense in contexts with catastrophic events.
So how it went is that the user made a reasonable demand with bounded reliability: "please survive power loss with durable writes" and the author says, sure just run the software on a SIL2 type reliability hardware environment.
It's not the user who blew up the hardware requirements.
For example, the best this DB can do is ~3.7Mkeys/s for non durable writes and 162k/s for durable. I have an equivalent DB that’s always durable and does 30M/s* (for 8 byte entries) because it doesn’t use Tokio among other things. For this dataset it would be saturating the disk I/O no problem so I would expect it to be running ~7-14M/s depending on how fast your SSD is (~2-4x faster than non durable mode and 40-80x faster than its “durable”)
* it was running at 70-100mhz at one point but the challenge is keeping the hot path at ~10ns as you add features and other things.
Out of curiosity where do you stand on libraries like this imposing a runtime choice on the user?
Is it standard in the rust ecosystem to have multiple async runtimes in a project?
Use whatever you want. Tokio is a very good work stealing runtime which is what Apple’s GCD popularized 17 years ago. It’s a fine model but sacrifices total throughput for ease of use and “easy” multithreading. Thread per core with pinned cores is what you use when you prioritize throughput and absolute possible latency. Tail latencies can suffer if you make a mistake and have imbalanced work on a single thread
iced does that quite well.
It has absolutely paid off, there are many people not using tokio.
Agreed. Good design is when the things do what you expect them to do without reading the manual, don't reuse wording with other meaning in the wrong way. That way if you do encounter nee wording, you know you should read the manual.
If your system has a reasonable tolerance for power failure (multi-az multi-cloud), this can provide much better throughput