I think that here the approach is very wise: every time you have a problem involving a big amount of data, Redis may be costly, since it stores data in memory. Sometimes this is unavoidable, for instance I may see Twitter switching from Redis to another in-memory store to cache timelines, but hardly to something using disk. So it is very legit to say, I'm running a number of hosts since my memory requirements are high, let's try a cheaper on disk solution.

However that said it is also true that memory starts to be seriously cheap, and the real limit may be to run stuff on EC2 or similar platforms.

So sometimes the cost of RAM is a virtual cost associated with the platform you are running your services in, compared to the actual cost of that amount of memory.

Just to be clear -- this doesn't mean we have eliminated Redis. It's still a very useful tool.

The article compares Redis to Cassandra which means comparing apples to oranges for their use case. It would have been meaningful if they explained why they chose Casssandra, instead of for example Hbase.

Interesting that Instagram is still (at least partially) on EC2 rather than in Facebook's datacenters.

I wonder if being part of Facebook (from which Cassandra originated) has anything to do with the choice for Cassandra over other databases.

I'm interested if @antirez has thought of extending the (admittedly fun) redis api to a disk-backed version for when you don't need the performance of in-memory, but appreciate the simplicity of the redis api.

Maybe be able to assign a specific DB number to disk?

I realize that might be kitchen sinking Redis more than he intends, but it might be worth it.

As others have noted, this capability was intentionally removed. From http://redis.io/topics/virtual-memory

"Redis VM is now deprecated. Redis 2.4 will be the latest Redis version featuring Virtual Memory (but it also warns you that Virtual Memory usage is discouraged). We found that using VM has several disadvantages and problems. In the future of Redis we want to simply provide the best in-memory database (but persistent on disk as usual) ever, without considering at least for now the support for databases bigger than RAM. Our future efforts are focused into providing scripting, cluster, and better persistence."

I assume Instagram was a serious user of Redis. If you can switch from Redis to Cassandra in a matter of days, even after presumably years of Redis-usage being ground in your code base (because no matter how much you try to abstract out your fundamental data store, it always comes poking out in your architecture), it's probably not useful for Redis to try to out-Cassandra Cassandra. Let Redis be the best Redis it can be, and let Cassandra be the best Cassandra it can be.

The ArDb project might fill this gap "A redis-protocol compatible persistent storage server, support LevelDB/KyotoCabinet/LMDB as storage engine." https://github.com/yinqiwen/ardb

IIRC, antirez experimented with that for a version or two before deciding it wasn't right for redis, deprecating then removing those added features.

My understanding is that being single threaded and assuming that every operation will complete very quickly is how Redis sidesteps a lot of complexity around atomicity and transactions. This is probably why swap as an option was quickly removed.

I've been working on a server that implements the Redis protocol and stores the data in LevelDB: https://github.com/cupcake/setdb

You might be interested in this:

http://www.anchor.com.au/blog/2013/04/redis-rethought-exciti...;

http://github.com/inaka/edis is here now for that purpose