OpenAI to honor robots.txt signals for AI training preferences
Website operators can now use a shared robots.txt standard to indicate whether AI models may use their content for training, with OpenAI as the first listed adopter.
1 source · cross-referenced
- OpenAI will recognize a shared robots.txt directive to signal AI training preferences.
- Website operators can now explicitly opt out of AI training via a standardized robots.txt entry.
- This is the first public commitment by a major model provider to honor such signals.
OpenAI announced it will honor a shared robots.txt signal that allows website operators to indicate their preferences regarding AI model training on their content. The directive, which can be added to a site’s robots.txt file, enables operators to opt out of AI training by listing a specific user agent and disallow rule. OpenAI states it will recognize this signal as part of its data sourcing practices.
The company describes this as the first public commitment by a major model provider to adopt a standardized method for respecting website operators’ AI training preferences. The shared directive is intended to simplify compliance for websites that wish to exclude their content from AI training datasets.
The change does not alter OpenAI’s existing legal positions or terms of service but provides an additional, machine-readable signal for websites to communicate their preferences. Website operators can add the directive to their robots.txt files to indicate that participating AI providers should not crawl or use their content for training purposes.
- Nov 11, 2026 · Google AI — Blog
Google launches Gemini Decisions API with new Gemini Decision 1 model for fast, calibrated multimodal decisions
Trust78 - Nov 11, 2026 · OpenAI — News
OpenAI offers free ChatGPT Plus to more than 20,000 veterans and servicemembers
Trust75 - Nov 11, 2026 · Latent Space — swyx
Anthropic releases Claude Fable 5.2 and Mythos 5.2, claiming top LLM spot on Anthropic’s IPO roadshow day
Trust76