- SignalDesk2小时前
Original Summary
s3-accelerator is an s3-compatible nvme cache that serves hits in under a millisecond, replicates hot objects, and stays warm when the cluster resizes.<p>the hot object replication was inspired by groupcache [1], but that has some pretty nasty issues when the cluster resizes<p>s3-accelerator fixes that by: 1) using rendezvous hashing to reduce the remapping when the cluster loses a node, and 2) when new nodes are added, asking the old parent for a give key for the object rather than going to s3<p>It also uses learnings from cloudflare that it's better to cache on a second hit than a first hit[2]<p>It also uses ktls, splice, and sendfile so byte transfer of object chunks never hit userspace. That means it can hit millions of iops, or saturate 200Gb/s network cards, while sipping on CPU<p>it also is parquet-aware, and has an optional SQS integration to keep the cache fresh if an object is overwritten.<p>S3 stays the source of truth, and you can write through the cache (it wont cache the write by default)<p>these features make s3-accelerator uniquely performant, fresh, and s3 costs extremely low.<p>It's one of those "I'll build it one day" project ideas that Opus 5.5 finally made viable (next up, paper2code).<p>[1] <a href="https://github.com/golang/groupcache" rel="nofollow">https://github.com/golang/groupcache</a> [2] <a href="https://blog.cloudflare.com/why-we-started-putting-unpopular-assets-in-memory" rel="nofollow">https://blog.cloudflare.com/why-we-started-putting-unpopular...</a>
- 情报分类：硬件与数码
- 分类依据：内容涉及硬件、数码产品或通信卡
- 信息来源：Hacker News 新项目
- 发布时间：2026/10/3 07:46:02
- 暂无回复