웹은 인간을 위한 것인가, AI 봇을 위한 것인가? Cloudflare가 새로운 기본값을 설정하다
cloudflare가 세분화된 AI 제어 기능을 출시하여 광고 기반 페이지에서 학습/에이전트 봇을 기본적으로 차단하고, 봇 운영자에게 자신의 목적을 밝히도록 강제합니다.
cloudflare introduced a new bot management framework allowing site owners to toggle access for search indexing, model training, and user agents independently, rather than applying a blanket block. the system defaults to denying training and agent access on ad-supported pages starting september 15, 2026, reserving that traffic for human visitors, while still permitting search crawlers. cloudflare ceo matthew prince cited that bot traffic recently surpassed human traffic for the first time as a driver for these stricter controls. this move pressures ai labs to stop bundling training functionality with search crawlers, forcing greater transparency about how content is used. enterprise users gain botbase, a dashboard database detailing known bots and their specific content usage to aid in enforcement.