Big AI players back new scraping opt-out control

a new standard for opting out of ai scraping has been backed by major ai companies including openai, google, microsoft, and anthropic. proposed by kevin kelly, a technical lead for standards at anthropic, the control will be implemented in the robots.txt file of a website. this allows publishers to signal their ai training preferences across multiple crawlers at once, rather than individually for each ai company. the move comes as publishers face increasing numbers of ai crawlers scraping their content. microsoft will offer more granular controls, allowing publishers to opt out of ai training or ai grounding. openai will implement the standard in the coming months, while google and microsoft plan to support it next year.