We built an open-source sub-microsecond Rust engine and gateway for LLM epistemic uncertainty quantification and hallucination gating:<p>GitHub: https:&#x2F;&#x2F;github.com&#x2F;Adarshent&#x2F;Spnda Python SDK: pip install spnda<p>Why we built this: Semantic Entropy (Nature 2024) is great at detecting hallucinations, but the quadratic NLI cross-encoder bottleneck (90ms GPU overhead with DeBERTa) makes it unusable for real-time production serving.<p>We found that exact-match normalized entropy (R_sc) achieves the same discriminative AUROC on structured reasoning (GSM8K) while running in under 1 microsecond on CPU (760 nanoseconds).<p>Benchmarking against Python proxies: - Kernel latency: 760 ns (Rust) vs 15 ms (Python) - Memory RSS: ~3 MB vs 230 MB - Cold start: 3.6 ms vs 1.17 s<p>Happy to

  • 情报分类：技术价值
  • 命中依据：Rust实现LLM亚微秒级不确定性估计，有明确技术对象与参考价值
  • 来源：Hacker News 新项目
  • 原作者：liquidngas
  • 发布时间：2026/9/12 02:48:16