Original Summary

My AI agent told a user "nothing was cancelled." The booking was already gone. I build Verb, an assistant that lives inside other people's products and does things for the user, like cancelling or rescheduling a booking, using that user's own access. Last week on a demo, someone asked it to reschedule. The cancel went through, the action returned nothing, and the assistant said the opposite of what had happened to their data. That is the worst kind of bug an agent can have, because it sounds confident. It made me look at everything the agent was holding, and I did not like it. So I am rebuilding the boundary, and I want you to poke holes in it. BEFORE - The model was handed the real id of every booking or order and was trusted not to repeat it. - The approval card showed raw fields: "Booking uid" over a long id, and the internal action name as its title. - Anyone who got hold of a conversation id could approve a pending action or read the answer. - "Verified" was whatever the user's browser reported about itself. - If a write returned nothing, we guessed done or failed. - Refresh the page and the messages stayed but the record of what the agent did vanished. NOW (built and testing locally, not live yet) - The model never sees a real id. It sees "booking_1, titled Demo with John". The real id is swapped back in a split second before the action runs. If the model never holds the secret, it cannot leak it, however it is tricked. - The approval card is plain words: "Cancel a booking? Booking: Demo with John." - A conversation is bound to the person who started it. A stranger with the id gets a plain 404. - "Verified" is judged by my server from the reply, never taken from the browser. - A write that returns nothing is "unknown". The assistant looks the record up again and tells you what it saw, or says it could not confirm. - A final filter scrubs ids and internal names from whatever the model writes, as a backstop and not the plan. - Before anything that changes data runs, it re-checks the thing you approved. That one came from someone on my last post, who asked: for cancellations the extra click is worth it, but what if the booking is moved while the card is open? Now, if it moved or vanished, nothing runs and the assistant tells you what is different and asks again. What it does not do: it narrows that window, it is not a lock. A page can still see the request its own browser makes. And I cannot judge meaning, so the output filter catches obvious leaks, not clever ones. I would love your opinion on three things: Is the re-check worth the occasional extra question? A trivial change can cost the user one more click. I think that is the right trade, but I might be wrong. Chat memory. Right now a visitor's chat lives in the tab. A refresh keeps it, closing the tab clears it. I did that on purpose so nothing follows an anonymous visitor around a site they never signed

  • 情报分类：技术学习与提效
  • 分类依据：内容涉及技术、AI、软件工具或工程实践
  • 信息来源：Reddit · SaaS
  • 发布时间：2026/10/8 01:52:33