Reply to Aphyr attack on Redis Sentinel(antirez.com) |
Reply to Aphyr attack on Redis Sentinel(antirez.com) |
My opinion of him and the Redis project increases further every time.
That said, I agree that DB reliability should be taken with the same rigor as net security...but I was kind of under the impression that it already was, in that DBs are pretty serious business. Also, "attack" has the connotation of, well, an "attack"...here, some of the failures happen in regular business operations, which is a problem different from when the system is under "attack".
But at least the OP took the criticism graciously. When I read what the case actually was, I then worried that the OP was having a bunker mentality.
In the PostgreSQL evaluation[0], Aphyr noticed that, if a packet confirming a transaction is dropped, the client ends up in a deadlock.
Does PostgreSQL keep a record of the past transactions, and their success or failure. If so, is it possible to query it?
The transaction ID does wrap around, so there's a time limit depending on your transaction throughput. You can also ask for certain transactional properties on rows, though this won't allow you to recover in all (most?) cases.
All in all, I am not sure if anyone should find this surprising: if anyone has ever had a network stall when clicking the 'confirm' button at a web-based store, they are familiar with the uncertainty as to whether the order has been submitted or not (resolved typically by browsing the history or waiting for an email, or no).
I would guess modern e-commerce vendors would send you a UUID or moral equivalent to de-dup cart resubmissions these days...but if not, it'd be interesting to know why not.
Thanks for the explanation.
I've had good results using Redis as a lock server, but I live in (perhaps misplaced) fear of a client hanging or crashing leaving a lock stranded. Not that this is really Redis's problem.
Even given this, bad lock timing (not that likely) or a crash (more likely) could let inconsistency in.
Shrugs
Like I said, my problem is not really Redis's. If I can't trust everything that uses a lock not to crash 99.99% of the time I should really be looking at our jobs and not at Redis.
Even then, though, it's probably more a matter of me not trusting things than it is said things not actually being trustworthy.
https://github.com/jamwt/dreadlock
It will release the lock when the client dies (disclaimer: I wrote it).
Or you can go whole hog and use zookeeper + ephemeral nodes. More robust but quite a bit more complex.
A great alternative to thinking about things in terms of CAP that Justin brings up is harvest-yield, where yield is the probability of completing a request and harvest is the fraction of your data that the response actually represents. Here's the paper: http://lab.mscs.mu.edu/Dist2012/lectures/HarvestYield.pdf
It looks bad, HN. We all know that VMWare is litigious as (try looking up benchmarks sometime.) But to (presumably) cave so quickly and effortlessly suggests... well, I'm not sure.
The other possibility is that Aphyr yanked them himself, probably under duress (or else there'd just be an 'update' at the bottom of the research's page.) Aphyr, is this what happened? I figure you probably can't talk freely if so, but say something.
1) I no longer work for VMware, but Pivotal. Redis is open source and copyright is of the original guys that wrote the code: I, Pieter Noordhuis, other contributors.
2) I posted the link to the original article in the first very lines of my reply. Actually thanks to my reply the exposure the Aphyr research had about Redis is the greatest, compared to the other data stores mentioned. I publicly said thank you to Aphyr on Twitter, and posted its blog post.
So I really don't understand your theories here.
It wouldn't be the first time a reputable news site was forced to bury a story by a litigious company. Sponsoring FOSS does not make any organization beyond doubt. Especially if they, say, have a history of suing anyone who benchmarks them.
HN stories on my original posts are still there, as far as I can tell. They just never hit frontpage.
http://aphyr.com/posts/287-asynchronous-replication-with-fai...
I'd be interested to hear about your implementation if you can share (email is HN username at gmail.com)
Talking about things like the FLP impossibility result, CAP theorem and specifying protocols with TLA+ may be a bit over the heads of many HN readers - clearly, people would rather read stories about the latest funding round, acquisition or frontend UI framework than a substantive article on distributed systems.