Original Summary

I've been spending my weekends building this localized flight deal aggregator. Nothing crazy, just trying to track price drops for obscure routes in eastern europe because google flights kinda sucks for regional airlines. But man, the big carriers have their bot protection dialed up to absolute insanity right now. I can’t even run a basic cron job to check a route twice a day without my cheap VPS getting instantly perma-banned. They really treat public ticket prices like classified state secrets lol Ended up having to rip out my whole fetching logic last night and route the requests through shifter so the target sites think I'm just normal people checking prices from different mobile devices. It finally got the scraper running stable without dropping connections every 5 minutes, but the amount of infrastructure you need just to build a basic comparison tool these days is wild. are any of you guys building aggregators right now? How are you dealing with the absolute state of data collection in 2026? i feel like half my codebase is just anti-anti-bot logic at this point.   submitted by   /u/CurrencyPopular8550 [link]   [comments]

  • 情报分类：工作与职业机会
  • 分类依据：内容涉及招聘、求职或职业发展
  • 信息来源：Reddit · SideProject
  • 发布时间：2026/10/6 16:27:07