OpenAI and Google back shared AI training control

Web publishers may soon be able to express one AI-training preference to several major AI developers, instead of maintaining separate instructions for each company. On October 30, OpenAI, Google and Microsoft backed a proposed common signal. This is an agreement to pursue a shared mechanism, not evidence that it already works everywhere.

The Internet-Draft written by Anthropic's K. Kelley uses the existing robots.txt file and a new policy token, __usage_ai-training. A website operator could use familiar Allow and Disallow rules to express preferences for the use of individual paths in training generative AI models. The proposed token changes no ordinary crawling rules: a preference about training does not itself prevent a bot from visiting a page.

Adoption schedules differ. OpenAI says it aims to implement the signal in the coming months and will keep its existing crawler controls. Google plans support in 2027, with its separate Google-Extended preference still available. Microsoft says its controls will begin to take effect as pages are crawled from February 1, 2027. Microsoft also announced separate page-level choices governing whether content may be used to inform AI-generated answers, sometimes called grounding. Sites using its older NoArchive or NoCache preferences may need to take transition steps by March 1, 2027.

That distinction matters to publishers. Choosing to withhold content from training is not the same as preventing traditional search indexing or preventing AI-generated answers from citing it. Microsoft says search eligibility is separate, although restrictions on grounding can reduce visibility in its AI experiences. Neither choice promises citations or referral traffic.

The IETF document remains a working draft, not a finished Internet standard. We have not tested any provider's implementation or compliance. The announcements do not establish that every AI company will honor the preference or that it guarantees compensation for content.