Original Summary

I’m trying to build a side project in the inference space. I’ve been talking to a few inference engineers and startups and I’ve been hearing how annoying it is to not have manual control over the KV cache at times and just constantly being subject to the black box caching methods of their inference providers. It is particularly annoying for agent swarms when you want to fork agents from the same cached prefix or manually store a cache for a longer period for a future agent to hit later.<p>Im curious if this problem is consistent across multiple people and if there are any solutions for it that people know about.

  • 情报分类：商业与市场研究
  • 分类依据：内容涉及商业、投资或市场动态
  • 信息来源：Hacker News 新项目
  • 发布时间：2026/9/21 12:26:20