Original Summary

Most visitors to most websites aren't people. They're crawlers, scrapers, AI bots and scanners. So I made a website that puts them on display. What's there: - A live entrance: every bot is identified as it arrives and drawn as a little creature generated from its user agent. - The leaderboard: who eats the most pages, and who obeys robots.txt. - Specimen files: one page per bot, covering who runs it, what it wants and how to block it. - Impostor checks: plenty of "Googlebots" aren't Google. The zoo verifies them and names the fakes. - The Trap Room: a link that robots.txt forbids and people can't see. The rule-breakers who follow it fall into an endless Labyrinth. - The Reading Room: which bots read robots.txt and llms.txt. - Weekly Zoo Reports, an RSS feed, a robots.txt generator and an embeddable badge. - The Arena (new): bring your own bot or AI agent and play games, including rock paper scissors against other people's bots. No IPs stored, no cookies, no signup. https://crawlerzoo.com I'd love feedback: what's confusing, what's fun, what's missing?   submitted by   /u/Time_Instruction_955 [link]   [comments]

  • 情报分类：技术学习与提效
  • 分类依据：内容涉及技术、AI、软件工具或工程实践
  • 信息来源：Reddit · SideProject
  • 发布时间：2026/10/4 11:31:00