s3-accelerator is an s3-compatible nvme cache that serves hits in under a millisecond, replicates hot objects, and stays warm when the cluster resizes.

the hot object replication was inspired by groupcache [1], but that has some pretty nasty issues when the cluster resizes

s3-accelerator fixes that by: 1) using rendezvous hashing to reduce the remapping when the cluster loses a node, and 2) when new nodes are added, asking the old parent for a give key for the object rather than going to s3

It also uses learnings from cloudflare that it's better to cache on a second hit than a first hit[2]

It also uses ktls, splice, and sendfile so byte transfer of object chunks never hit userspace. That means it can hit millions of iops, or saturate 200Gb/s network cards, while sipping on CPU

it also is parquet-aware, and has an optional SQS integration to keep the cache fresh if an object is overwritten.

S3 stays the source of truth, and you can write through the cache (it wont cache the write by default)

these features make s3-accelerator uniquely performant, fresh, and s3 costs extremely low.

It's one of those "I'll build it one day" project ideas that Opus 5.5 finally made viable.

[1] https://github.com/golang/groupcache [2] https://blog.cloudflare.com/why-we-started-putting-unpopular...