According to OpenAI’s announcement on October 30, 2026, OpenAI Blog, websites can now use a shared robots.txt directive to indicate their preferences regarding inclusion in AI training datasets. This update allows site operators to communicate directly with OpenAI and other participating providers about whether their content should be used for AI training or not, similar to how robots.txt manages crawler access.
To apply this in your work, you should:
- Review the new AI-related directives added to the robots.txt standard as outlined by OpenAI.
- Advise site owners or your DevOps team how to set up or adjust robots.txt entries to reflect their data-sharing choices for AI training.
- Expect OpenAI and other participating AI firms to respect these preferences as a matter of policy moving forward.
This change will make AI training data practices more transparent and controllable for webmasters and developers.