Original Summary

Hey everyone, I’ve been working on a small open-source Python library called callm and would genuinely love feedback from people using LLMs in production. The idea is simple: instead of replacing the OpenAI, Anthropic, or Google GenAI SDKs with another abstraction, it wraps the functions where you already make those calls. from callm import callm @callm( cache=True, retry=3, fallback=["anthropic/claude-sonnet-5"], block_pii=True, output_schema=Summary, ) def summarize(text: str): return client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": text}], ) Inside the decorated function, it can handle: retries with backoff and provider retry-after / rate-limit headers exact-match caching (SQLite by default, Redis optional) fallbacks across providers or models while returning the response type your code expects per-call cost limits and shared budgets, checked before a request is sent PII masking and prompt-injection scoring Pydantic output validation, with automatic re-asking when validation fails local call records plus a callm stats CLI Outside decorated functions, the SDKs continue working normally. I know tools like LiteLLM, Instructor, Guardrails AI, and a combination of Tenacity + caching already cover parts of this space. My goal with callm is specifically to keep native SDK calls intact and provide the common production safeguards entirely in-process, without needing to deploy a gateway. It’s still early, currently v0.1.0. The test suite has 274 tests across Python 3.10–3.13 on Linux, macOS, and Windows, using real SDKs with mocked HTTP. But real workloads are where the interesting problems show up. Repo: https://github.com/TanbirRamim/callm Docs: https://tanbirramim.github.io/callm/ Install: pip install "callm-toolkit[openai,validation]" No API key needed for the offline demo: python examples/offline_demo.py I’d especially appreciate feedback on whether this solves a problem you actually have, what feels unnecessary, and what would make you trust or adopt it in a real service.   submitted by   /u/MassiveFinish3510 [link]   [comments]

  • 情报分类：商业与市场研究
  • 分类依据：内容涉及商业、投资或市场动态
  • 信息来源：Reddit · SideProject
  • 发布时间：2026/9/16 21:10:20