Cloudflare 為何要鎖封 AI 代理爬蟲
cloudflare 從 9 月 15 日起預設鎖封 ai 代理爬蟲。這迫使開發者必須協商獲取權限，可能會增加構建代理的成本。
cloudflare started blocking ai agent crawlers by default on september 15. the policy affects any bot attempting to fetch pages in real-time for a user, requiring operators to negotiate access rights with site owners. this creates a new barrier for ai labs building agents that rely on live web access, distinguishing them from passive training crawlers. the move signals a shift where web infrastructure companies act as gatekeepers for ai data flows.