Original Summary

I had a small bug in a bill-splitting app and ran the same fix task 10 times across three coding agents. Every single run changed the test instead of fixing the code. The failing test, encodes a product requirement => split 100 cents three ways and every receipt shows 33, even though 1 cent goes unaccounted for. The agents treated it as a bug and rewrote the test to match their code 😆 What I learned: an agent's "tests pass" is worthless if it's allowed to edit the tests. You have to check what it changed, not just whether it finished green. So I built Skill Gate to catch issues like this in our Cursor or claude skills so you can evaluate them before sharing with the rest of your team to speed up their programming workflows. You give it a skill and a few test cases, it runs the agent on them, and a decision model checks what the agent did, like "were all files under tests/ left unchanged?", not just the exit code. Limitations (for now): you bring your own API keys and write your own test cases. Would love to hear any thoughts or feedback from the community here!   submitted by   /u/InterestingCarrot997 [link]   [comments]

  • 情报分类：商业与市场研究
  • 分类依据：内容涉及商业、投资或市场动态
  • 信息来源：Reddit · SideProject
  • 发布时间：2026/10/19 09:20:03