Cloudflare 2026年起默认拦截混合AI爬虫
cloudflare将于2026年9月默认拦截混合用途爬虫，迫使ai公司将训练爬虫与搜索分离，或通过按需付费模式向发布者付费。
cloudflare announced a september 2026 deadline to block mixed-use crawlers from ad-supported pages unless site owners opt-in, targeting bots that bundle search indexing with ai training. the policy responds to non-human traffic surpassing human traffic and aims to stop ai companies from scraping intellectual property for free. cloudflare is also evolving its pay per crawl marketplace into pay per use, partnering with ceramic.ai and you.com to let publishers charge when content generates value. this forces ai providers to clarify bot intent or establish commercial agreements to access training data.