Original Summary

I thought I would drop this one out again, the first harness I made, but still my best worker for resources. Currently occupying a 4060 laptop (harness + models), single handedly running one of my businesses customer service, social media, email and personal assistant to me. I have gemma3n:e4b occupying the CPU as the intake (quick response), qwen3.5:9b as the worker, and nomic-embed-text for encoding occupying the GPU. Queue the 9b hate, but at this scale the harness is what makes the big difference, they are quite effective when you keep the individual workloads granular. This is the leanest combination I have found effective, and it does occasionally fail, but it does the job sufficiently, I was running a few additional LLMs attached over network but needed them for my latest project. It is useful to have workers like this when you want consistent AI support, but need your big VRAM pools available to other tasks. It does have the option to run as many LLMs as you can throw at it in parallel, which does make it heaps more powerful, but I think it really excels at lightweight agent setups. If anyone is looking for a light weight option for your spare or old hardware, it may be an option for you. Open source, modular, hackable, with a basic plugin ecosystem, built in IoT, enjoy. https://github.com/doctarock/local-ai-home-assistant   submitted by   /u/Electronic-Space-736 [link]   [comments]

  • 情报分类：工作与职业机会
  • 分类依据：内容涉及招聘、求职或职业发展
  • 信息来源：Reddit · SideProject
  • 发布时间：2026/9/22 18:58:48