Cloudflare、2026年までに複合用途AIクローラーをブロックへ
Cloudflareは2026年までに、広告掲載ページに対する複合用途クローラーをブロックする方針。AIラボはトレーニング用ボットを検索用途から分離するか、コンテンツの対価を出版社に支払うことを迫られる。
cloudflare announced a september 2026 deadline to block mixed-use crawlers from ad-supported pages unless site owners opt-in, targeting bots that bundle search indexing with ai training. the policy responds to non-human traffic surpassing human traffic and aims to stop ai companies from scraping intellectual property for free. cloudflare is also evolving its pay per crawl marketplace into pay per use, partnering with ceramic.ai and you.com to let publishers charge when content generates value. this forces ai providers to clarify bot intent or establish commercial agreements to access training data.