Original Summary

I run a small services business. We recently took on a paid pilot for an ecommerce business. The plan was straightforward on paper: connect an assistant to product information and let it handle a defined set of customer questions. Then we opened the data. Their website and spreadsheet didn't match in a lot of places. A large number of products had no useful descriptions. Some IDs were not unique. Product attributes that customers actually ask about were missing or sitting in different places. We asked their developer what the source of truth was and basically got: "this is all we have." So we did more than I expected. Over nearly a month, we cleaned and indexed the catalogue, filled in a lot of missing product information, and built a safer knowledge base instead of letting it make things up. That took real delivery time. When the first version was ready, the client tested it in real chats. Across about 41 minutes and eight conversations, they found real issues: - answers were sometimes too long; - photo-based questions needed another setup pass; - some questions could not be answered because the source data did not contain the answer; - a few scenarios needed clearer human handoff. None of that is something I want to hide. A pilot is supposed to expose those things. But then the conversation became awkward. Their position was basically: "we wanted it to work without us changing anything or getting involved. If we have to improve the data or help train it, the service is not useful to us." They also wanted to discuss getting money back. And honestly, I can see both sides. They bought a result, not a lesson in how their catalogue is broken. We should have done a better job of making the pilot boundaries painfully explicit before taking the payment. At the same time, I don't think "we only used it for 41 minutes" makes nearly a month of real setup, data work, configuration and testing worth zero. We did not sell a free demo. We sold a paid pilot, and the data condition was much worse than it looked at the start. The bigger lesson for me is that "pilot" is too vague unless you write down the boring parts: - what data is the source of truth; - what the assistant is allowed to answer versus escalate; - what counts as a defect versus missing client data; - how many correction rounds are included; - what is a new request; - what happens if the client removes half the original scope halfway through. What I am changing for the next client: No build starts until there is one written source of truth for every customer-facing fact. Data audit and cleanup get a visible scope, cap, and price instead of being implied setup work. The pilot has written test cases, a named feedback owner, and a fixed number of consolidated correction rounds. We define which situations go to a human before launch, rather than treating human handoff as a surprise. For agency owners who have dealt with this: would you sell the data audi

中文概览

中文标题： 代理机构老板：付费AI试点遇到混乱客户数据，随后仅41分钟测试，下次你会如何设计？

小型服务商为电商客户做付费AI试点，将助手接入产品信息。客户数据混乱：描述缺失、ID不唯一、关键属性缺失，团队花近一个月清洗建库。客户用41分钟、8段对话测试，发现答案过长、图片问题需额外配置、部分问题因源数据无法回答、人工转接不清。客户不愿改数据或参与并要求退款。作者认为应事先写清试点边界，下次先确立唯一事实来源、限定数据审计范围与报价、写明测试用例和修正轮次。

  • 情报分类：商业与市场研究
  • 分类依据：内容涉及代理机构付费AI试点中的客户数据清洗、范围与验收争议，属商业与市场研究。
  • 信息来源：Reddit · SaaS
  • 发布时间：2026/9/18 07:10:35