What I read (I'm usually the pessimist in the room):

1. We had Solr but couldn't scale it

2. We decided to use Redis

3. We migrated all code to use Redis and wrote our own indexer

4. Redis didn't work (and I love Redis) out of the box for our problem so we build our solution on top of Redis, we had an index and an intersection server and put effort into ops

5. Then we moved to Elastic

6. Then we moved to Algolia

I wonder

a.) what value all of this brought to the user

b.) what ROI the engineering investment had

I wonder how much engineering time went in to the memory/performance tuning by using the slave? And I wonder how much they saved on their hosting bill?

I don't know the answers to these, but the addition of the slave and the patches to Redis set off my "not worth it" alarm. If they are constrained on the development resource, that would be even worse.

Has there been any attempt to incorporate these changes upstream? Always seems a bit of a tragedy when its not even attempted, and perhaps comes across as a little lazy. From an engineering PoV, maintaining a fork is great at first but either an explicit pain later on or (worse) contains unknown unknowns.

Hehe, must have been what, 7 years ago I used Redis for full text search instead of using PostgreSQL on Heroku.

RedisToGo had a soft tier free service and Heroku offered 20(?) mb PostgreSQL back then. It was fun

tl;dr version: Intersection operation took too long, forked redis to enable key retrievals/expirations on a slave instance and ran the CPU intensive intersection on the slave.