Original Summary

I run local AI models at home, but sometimes need more compute than I own. Cloud GPUs solve that problem, except I hated paying for an expensive GPU to idle while I wasn't actually running anything. So I started building Summon. The basic idea is to keep the cheap infrastructure and model cache around, but treat the GPU worker as disposable. Ask for compute, create the worker, attach the model cache, start inference, then destroy the expensive part when you're finished. My current GCP prototype can go from no worker at all to an L4 serving a model through llama.cpp in about 90 seconds. The unexpectedly interesting problem has been capacity. A GCP zone can support L4s while having none available, so I'm now building the control layer to discover capability and hunt for available capacity automatically. This started as something I wanted for myself, but I'm trying to figure out whether it's a problem other people actually have before I disappear down the infrastructure rabbit hole for another six months. If you occasionally need more compute than you own, how do you handle it now? Would waiting ~90 seconds for a GPU in exchange for not paying for it while idle be useful, or is that already a dealbreaker? summon.protectorate.net   submitted by   /u/Legal-Value2541 [link]   [comments]

  • 情报分类：商业与市场研究
  • 分类依据：内容涉及商业、投资或市场动态
  • 信息来源：Reddit · SideProject
  • 发布时间：2026/9/17 13:10:34