Original Summary

Claude Code is great at planning but slow at clicking: every step is a screenshot and a think. So I split the job. Claude writes the steps, and Jev (TypeSafe's small decision model) finds each target from a screenshot in about 0.6s and makes the move. Plain steps like typing, pressing keys, picking from a dropdown or attaching a file skip the model entirely. A few things I cared about: - It reads the macOS accessibility tree first when it can, and only falls back to pixels when it has to. - It stops before anything that looks risky (send, delete, pay) and waits for me. - If I touch the mouse, it stops. On a test page of 12 random buttons it hit 47/48, at about 1.2s per click. Free and open source (MIT), macOS. Code and a real-time demo GIF: https://github.com/zacfink/screen-intelligence I'd love feedback, especially on apps where it gets lost.   submitted by   /u/Frosty-Hunt-7697 [link]   [comments]

  • 情报分类：工作与职业机会
  • 分类依据：内容涉及招聘、求职或职业发展
  • 信息来源：Reddit · SideProject
  • 发布时间：2026/10/19 04:26:19