Original Summary

Jev from TypeSafe AI made me wonder how far typed decisions could be pushed with a genuinely small local model, so I built an independent open-source experiment called Decision Lab. The model reads shared context once, reuses the cache for every field, scores a fixed set of allowed answers, then normal code assembles the typed JSON. What surprised me was not the headline score. It was the gap between a clean benchmark and a realistic workflow: 83% on 1,600 held-out public examples 59.6% field accuracy on a realistic workflow suite 19% exact records 92 to 148 ms for four warm fields on my M4 Mac mini about 296 MB to load before shader compilation A schema guarantees valid output, not a correct decision. Independent field scoring can also create contradictions when fields depend on one another. My current take is that this pattern is useful for many small, bounded judgments over shared context. It is much less convincing for sequential or stateful reasoning. I open-sourced the INT4 model, training and export pipeline, TypeScript SDK, browser playground, tests, evaluation data, and failure cases: https://github.com/khalilelghoul01/decision-lab I would especially value criticism of the evaluation design and the independent-field assumption.   submitted by   /u/Glad-Bend6933 [link]   [comments]

  • 情报分类：技术学习与提效
  • 分类依据：内容涉及技术、AI、软件工具或工程实践
  • 信息来源：Reddit · SideProject
  • 发布时间：2026/9/19 03:41:50