Original Summary

A while ago I started building TUFF because I wanted to run massive LLMs on my old 16GB M2 MacBook Air. I ended up writing my own inference engine in Swift and Metal, with SSD streaming to run models much larger than my available RAM. Now TUFF has reached version 8.0, and it’s turned into a lot more than I originally expected. I’ve added web search, local file search, conversation caching, better memory management, a local API, and Homebrew installation. The funniest part is that adding web search has probably been the most useful improvement for me personally. I used to think I’d always need frontier models for chat, but even small local models are now good enough for most of my everyday questions. I’m hoping to turn TUFF into a real community project with more people contributing, rather than just something I work on myself. It’s completely free and open source. https://github.com/rexmhall09/TUFF Would love any feedback, especially from other people building open-source tools!   submitted by   /u/Weekly-Dentist-8302 [link]   [comments]

  • 情报分类：硬件与数码
  • 分类依据：内容涉及硬件、数码产品或通信卡
  • 信息来源：Reddit · SideProject
  • 发布时间：2026/10/9 13:07:46