OpenAI, Google, Microsoft and Anthropic to honor unified robots.txt AI training opt-out
On October 30, 2026, Google announced plans to support a shared AI training control in robots.txt, with OpenAI, Microsoft and Anthropic agreeing to honor the same opt-out, based on IETF AIPREF working group work.
Read the full story at OpenAI News ↗
Timeline · 13 reports
- 2026-10-31 14:17 · DEV Community — AI
Check robots.txt before your AI agent fetches a page: an 80-line RFC 9309 parser in Node - 2026-10-31 13:09 · DEV Community — AI
One robots.txt Rule to Opt Out of AI Training: What Developers Need to Know (2027) - 2026-10-31 12:26 · The Next Web
AI training opt-out: publishers ‘should not have to choose’ - 2026-10-31 12:24 · The Next Web
Google, Microsoft, OpenAI and Anthropic back one AI training opt-out - 2026-10-31 11:30 · Unite.AI
Google Plans Support for Shared Robots.txt AI Training Control - 2026-10-31 06:39 · DEV Community — AI
Google plans one robots.txt token for AI training opt-outs - 2026-10-31 04:33 · DEV Community — AI
One robots.txt Line That Speaks to Every AI Lab: Implementing the Shared Training Opt-Out - 2026-10-31 03:47 · DEV Community — AI
OpenAI, Google, Microsoft and Anthropic to support a unified AI training opt-out token in robots.txt - 2026-10-31 02:00 · SiliconANGLE AI
Google, Anthropic and others to support new AI crawling opt-out standard - 2026-10-31 01:19 · DEV Community — AI
One robots.txt Line to Say No to AI Training: What OpenAI, Google, Microsoft and Anthropic Just Agreed - 2026-10-30 21:05 · Techmeme
OpenAI, Google, Microsoft, and Anthropic say they will adopt a new standard that publishers can use to opt out of being scraped by AI across multiple crawlers - 2026-10-30 18:27 · Unite.AI
OpenAI to Honor Shared Robots.txt Signal for AI Training Preferences - 2026-10-30 09:30 · OpenAI News
Making it easier for websites to express AI training preferences