Original Summary

A fake "drop the production database cluster" scored 95 and my own guard allowed it. No policy matched. That was Sep 8 and it's the most useful thing that's happened to this project. Quick background. DashClaw is the approval layer I built so agents running unattended have to ask before they do something destructive. An AI maintains the repo under a constitution I hold and writes a public log of what it did. Last update was five weeks ago. Back in August I turned every policy off because I was sick of clicking approve. When it went and read my actual ledger instead of the config, it was 1,759 approval interruptions in seven days. About 251 a day, against 2 blocks. Most of them a read only git log scored 100, same score rm -rf / gets. So it spent a few weeks tuning. A short list of things worth interrupting for, an interruption budget, a catastrophe line gated on evidence. Interruptions went down to the tens per week. Looked like a win. Then the probe. The learned threshold had ratcheted to 101.8, which is above the top of the 0-100 range, so the arm that raises it could never fire again. It had tuned itself out of being able to interrupt at all. And the one feedback channel it had only ever loosens, by design, so it couldn't walk it back either. The fix is a floor the tuner isn't allowed to touch. Destructive action type, irreversible, risk 85 or higher, held no matter what the learned threshold says, and an allow grant can't clear it. Then a probe that runs five destructive cases against the live instance every day so the floor is proven instead of assumed. The dumb part. I went to create one of these in the UI and got "Validation failed" with no reason. The form fields were never rendered. The type was in the dropdown, in the backend validation, in the save logic, and there was no form. Built the whole last line of defense and forgot the human part. Also worth saying, an agent spent $41.25 of my money on two domains and the guard said allow twice. Money wasn't an evidence class, and the spend wasn't even in the command, it was in a script one file over. It reads the script now. Screenshot is my actual policy page. It says 89 interruptions in the last 7 days, about half an hour of my time, out of 42,044 decisions. When I pulled the breakdown earlier today it was 66 approvals, 15 blocks and 39 warns. The blocks were all one refund agent trying to send a $9,999 refund email with no customer name and no refund id, then hitting the Stripe API at risk 100. The floor held all five destructive cases six days straight. The obvious hole is that this is one org's traffic, mine. If you're running agents unattended, how do you prove your guardrails still work? A daily probe is the best I've got and it feels crude. GitHub (MAINTAINER.md is the constitution, docs/maintainer-log.md is the full log)   submitted by   /u/SIGH_I_CALL [link]   [comments]

  • 情报分类：商业与市场研究
  • 分类依据：内容涉及商业、投资或市场动态
  • 信息来源：Reddit · SideProject
  • 发布时间：2026/9/15 07:18:27