ApsaraCache: A Redis branch originated from Alibaba Group

ApsaraCache: A Redis branch originated from Alibaba Group(github.com)

43 points by laithshadeed 8 years ago | 41 comments

antirez 8 years ago |

Hi all, I'm here in Hangzhou and had the pleasure to have a meeting with the team working at that. It's a great team and we'll work into merging stuff into upstream Redis when it makes sense. It's cool that companies other than Redis Labs are starting to contribute to the development!

Note that this branch is based on Redis 2.8 so of course porting stuff requires some work.

tyingq 8 years ago | |

>Note that this branch is based on Redis 2.8

Hmm. The Github repo says it's based on 4.0: "ApsaraCache is based on the Redis official release 4.0" (from https://github.com/alibaba/ApsaraCache/blob/develop/README.m...)

antirez 8 years ago | | |

Oh probably I mistaken other patches I discussed that will need to be ported. Thanks for the reply.

ddorian43 8 years ago | |

Can I ask what you think of pedis ? (see my other comment on the thread)

antirez 8 years ago | | |

Hey, I never tested it, I've no idea. In general I think that threading Redis is a good idea and I'm going to add more threading very soon. I'm less sold on making disk the Redis primary storage...

gregoriol 8 years ago |

Reminds me of HHVM: a specific solution developed by a specific team for their specific problem.

Might help Redis on the long run though: some ideas might be brought back to Redis, or at least it will bring some challenge to the devs.

Meanwhile, keep calm and use Redis ;-)

ianlevesque 8 years ago | |

Agreed, though HHVM went on to be deployed at some other small websites [1]. Maybe this will too.

1. http://hhvm.com/blog/7205/wikipedia-on-hhvm

mintplant 8 years ago | | |

On the other hand, Wikipedia is going to have to migrate back off of HHVM now that Facebook is planning to break compatibility with PHP and focus totally on Hack [0]. So hopping on board someone else's solution can bite you if their priorities change.

[0] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html

danjf 8 years ago |

30% faster by which measurements? Not much info on the docs....

0T-T0 8 years ago | |

yeah, one of the few red flags

> In short connection scenario, ApsaraCache makes 30% performance increase compared with the vanilla version.

sounds like a cherry picked case to me

detaro 8 years ago | | |

Cherry-picked IMHO isn't necessarily the right label for that. They don't claim they made an universally 30% faster redis, they clearly say one of the things they did improved performance for a specific case.

If anything, the OP here is misleading by quoting it as a general improvement.

ehllo 8 years ago | | |

for them it is a real need.

(Talk with Daniel Zang CEO Alibaba Group) https://youtu.be/YhlvtcBwxPM?t=326

olavgg 8 years ago | |

I just tested it with redis-benchmark. The performance is the same as with Redis.

tyingq 8 years ago | | |

It mentions "in short connection scenario".

I take that to mean repeated connect/set-get/disconnect. Like you might get from a client in a fastcgi scenario...php for example. Maybe add the -k 0 flag to the benchmark and a low-ish number of concurrent clients.

danjf 8 years ago | | |

Oh...

daotoad 8 years ago |

This looks pretty interesting to me. But it leads to a lot of questions that are unanswered on the webpage.

The claims of 30% faster for a particular workload are reasonable, but what configuration is needed and what exact workload are we talking about? I'd love to see some benchmarking code.

What are the limitations imposed by running in memcached mode? Can you still do clustering?

What other changes are they planning? Site just says "many". Are we talking about memory allocation or persistence changes? Maybe additional interface compatibility modes? It's hard to bet on this when we don't have any idea what else it will bring to the table.

Where possible, I'd really like to see these improvements factored into distinct redis modules and made available as plugins to the core.

stonewhite 8 years ago |

Goals: For both support Redis and Memcached protocol with no client code need to be modified. For highly stable and efficient in production environment.

At face value it appears to be a NIH driven development. It is not only a redis fork it is also a persistent memcached port, which is _only_ selectable on launch. Maybe there is a catch, but lack of a roadmap and/or design document certainly doesn't really help.

ksec 8 years ago | |

I remember some time ago Alibaba was the largest Memcached user on the planet, may be they wanted to smooth the operation and use Redis whenever possible?

pmontra 8 years ago |

Why forking instead of submitting pull requests to Redis?

lugg 8 years ago | |

The readme states they have a few extra unreleased features.

I'm guessing their changes are for internal use and may be out of scope for redis or simply they want to share their internal fork and get feedback/help.

I know you didn't say anything to the contrary but I feel it needs to be said: forks aren't inherently bad, they're a sign of a mature and healthy open source ecosystem.

And I'm sure if it's worth it to someone, perf improvements will eventually get upstreamed.

helb 8 years ago | |

It seems they made a lot of changes, and some would be probably unsuitable for Redis. "Memcached mode" for example.

arthursilva 8 years ago |

I wonder if Redis throughput is even a problem in practice. People usually add nodes to get more capacity (memory).

ihsw2 8 years ago | |

For most use-cases, Redis nodes will saturate network IO before CPU, but it should be noted that Redis is single-threaded so YMMV.

joneholland 8 years ago | | |

We do heavy pipelining and tend to max out the single thread, so we run multiple redis instances per host.

mangatmodi 8 years ago |

One of the most useless FAQ I saw. I would have been more interested in knowing the improvements or changes made when compared to Redis. I had some trouble with large scale master slave Redis configuration. I was hoping if they have solved the issues.

detaro 8 years ago | |

They list the two things they changed compared to redis in the readme.

mangatmodi 8 years ago | | |

I went back and read again. "ApsaraCache is based on the Redis official release 4.0 and has many features and performance enhancements" - I could find only memcache and performance improvement. Now Membache has nothing to do with Redis, its something not improving Redis, and 30% performance improvement is very vague statement with no other supporting benchmarks and documents. When they say many, I can't see any other than these two.

ddorian43 8 years ago |

I think they are also working on: https://github.com/fastio/pedis (redis implemented on top of seastar-framework / scylladb )

tuyguntn 8 years ago | |

any experience using it in production? looks promising results

ddorian43 8 years ago | | |

still missing many features so I don't think it's ready, you should ask on their issue queue

tyingq 8 years ago |

Diffs of what they changed in /src from Redis 4.0: https://github.com/asaradiffs/apsaradiffs

tuananh 8 years ago |

documentation and readme of alibaba projects are scary. Have anyone use this or any alibaba open source projects for production?

CyberDildonics 8 years ago |

Does it still run on only one thread?

rvnx 8 years ago |

This project is some kind of a joke;

Look at the commit history; https://github.com/alibaba/ApsaraCache/commits/develop

15 are "writing of README / adding my copyright", and 2 are actual feature (and not long)

zokier 8 years ago | |

I think this is what you are missing: https://github.com/alibaba/ApsaraCache/commit/07518063b17b13...

detaro 8 years ago | |

Kind of sad how things can't be partial and in the process of being open-sourced, no they have to be complete or they are "jokes".

dang 8 years ago | |

Please don't post uncharitable dismissals of other people's work to HN. If you read https://news.ycombinator.com/newsguidelines.html properly you'll see that doing this breaks more than one of the site guidelines.