the hardest engineering problem in my AI product turned out to be making the model wrong on purpose
- SignalDesk1小时前
Original Summary
I'm building an AI math tutor that's deliberately incorrect. The character (a robot called Errol) works through a problem, gets it wrong somewhere, and the kid's job is to catch the mistake and explain why it's wrong. Premise being that every other AI tutor is racing to be correct, and what kids are actually going to need is the ability to tell when the machine isn't. I assumed the "being wrong" part would be the easy half. It was by far the hardest thing in the build. LLMs are genuinely bad at being wrong on purpose. Ask for a subtly incorrect solution and you get one of two failure modes. Either the error is so obvious a six year old spots it instantly, or the "wrong" answer is accidentally correct. That second one will kill the product. Show a kid a correct answer, tell them it's wrong, and you've taught them to distrust their own good reasoning. That is worse than teaching them nothing. So nothing reaches a kid unchecked. Every generated problem gets independently verified as actually wrong before it's allowed anywhere near a user, and if it doesn't hold up it gets thrown away and regenerated. I eat the cost on every discarded one. Non negotiable though, because the failure is silent. Nobody reports the bug where the app told a kid they were wrong when they weren't. They just quietly stop using it. Difficulty calibration is the product, not a feature. Too obvious and they're bored in two minutes. Too subtle and they feel stupid, which is the one thing you cannot do to a 12 year old and expect them back tomorrow. I've got tiers roughly mapped to age bands and it still needed an adaptive layer per kid per topic on top. This is the part I'd rebuild first if I had to start over, and honestly the only thing here that's hard to copy. Stop treating "the AI" as one thing. The biggest structural win was splitting it into separate jobs that don't trust each other. The thing that writes the content is not the thing that decides whether the content is fit to ship, and neither of those is the thing that reads what the kid wrote back. Once they're separate you can put real effort where the quality actually matters and something cheap and fast everywhere else, and you can change your mind about any one of them without touching the rest. Retrofitting that later would have been miserable. Where I actually am: feature complete, pre launch, zero paying customers. So read all of that as engineering notes and not a success story. I've built the thing, I haven't proven anyone wants it. The bit I keep going round in circles on, and would genuinely take input on: the buyer is a parent, the user is a kid. Conventional wisdom says sell to the wallet, so pitch the parent hard on outcomes and progress reports. But retention lives entirely with the kid, and the kid does not care about a weekly email to their mum. Anyone here shipped something where the payer and the user
- 情报分类:工作与职业机会
- 分类依据:内容涉及招聘、求职或职业发展
- 信息来源:Reddit · SaaS
- 发布时间:2026/9/23 05:48:48
- 暂无回复