Instead of juggling a separate opt-out for every AI company, website owners may soon be able to say “don’t train on my content” once. OpenAI, Google, Microsoft and Anthropic have backed a shared signal that lives in a site’s robots.txt file, according to reports published Oct. 30.
The catch: it is a commitment to build something, not a switch you can flip today for most of the companies involved.
What the signal is
The proposal is a special robots.txt label, written as __usage_ai-training. A site owner would add a rule under that label, such as a blanket Disallow: /, with optional Allow: lines for sections they are happy to share. Participating AI companies would read that one line as the owner’s training preference.
It is not a new crawler. It is a policy label, described in an individual Internet-Draft titled “Machine-Readable Training Preference Signal.” Individual drafts are not adopted internet standards and carry no formal IETF endorsement.
Who is doing what, and when
- Anthropic: has reportedly started implementing it, with no completion date given.
- OpenAI: says it aims to honor the signal “in the coming months.” Its existing controls stay in place.
- Microsoft: targets Feb. 1, 2027, and plans a separate setting for AI “grounding,” the step where a chatbot pulls live web content into an answer.
- Google: says 2027. If a site uses the shared label without naming Google-Extended, Google says it will treat that as a Google-Extended disallow.
What it won’t do
Three limits matter for anyone running a blog, shop or local-business site.
- It covers training, not crawling. The label does not by itself stop a bot from fetching your pages. Your existing crawler rules still do that job.
- It is a request, not a lock. OpenAI itself calls it a coordination mechanism rather than a security control, so a scraper that ignores robots.txt can ignore this too.
- It looks forward. Opting out does not pull your content out of datasets or models that already used it.
What to do now
Today the working controls are the ones you can already use. OpenAI separates training from search: GPTBot collects potential training material, while OAI-SearchBot powers ChatGPT search. Blocking GPTBot will not block every ChatGPT visit, because ChatGPT-User handles some requests that a person triggers, and robots.txt rules may not apply to those.
There is also a trade-off. Keeping your pages out of AI training is a different decision from keeping them out of AI answers and search. Blocking one can affect the other in ways that differ by company, so decide which you care about before editing the file.
If you manage a site, a sensible low-effort step is to open your robots.txt (usually at yoursite.com/robots.txt), note which AI crawlers you already address, and wait until each company confirms it reads the shared label before relying on it. Many website platforms and hosting dashboards edit this file for you.
Related: see how AI companies are already being pressed over data and safety in our report on the FTC subpoenas and state lawsuits.
More on Contoh
- FTC Subpoenas OpenAI and Anthropic as 8 States Sue Over ChatGPT Safety
- Schwab and JPMorgan Block Outside AI Agents From Logging In to Your Accounts
- Claude Opus 5.5 Is Cutting Off Reverse-Engineering Work. Here’s the Likely Reason