Cloudflare 为何要封锁 AI 代理爬虫
cloudflare 从 9 月 15 日起默认封锁 ai 代理爬虫。这迫使开发者必须协商获取权限，可能会增加构建代理的成本。
cloudflare started blocking ai agent crawlers by default on september 15. the policy affects any bot attempting to fetch pages in real-time for a user, requiring operators to negotiate access rights with site owners. this creates a new barrier for ai labs building agents that rely on live web access, distinguishing them from passive training crawlers. the move signals a shift where web infrastructure companies act as gatekeepers for ai data flows.