Skip to content
Models · Oct 30, 2026

OpenAI to honor robots.txt signals for AI training preferences

Website operators can now use a shared robots.txt standard to indicate whether AI models may use their content for training, with OpenAI as the first listed adopter.

Trust79
HypeLow hype

1 source · cross-referenced

ShareXLinkedInEmail
TL;DR
  • OpenAI will recognize a shared robots.txt directive to signal AI training preferences.
  • Website operators can now explicitly opt out of AI training via a standardized robots.txt entry.
  • This is the first public commitment by a major model provider to honor such signals.

OpenAI announced it will honor a shared robots.txt signal that allows website operators to indicate their preferences regarding AI model training on their content. The directive, which can be added to a site’s robots.txt file, enables operators to opt out of AI training by listing a specific user agent and disallow rule. OpenAI states it will recognize this signal as part of its data sourcing practices.

The company describes this as the first public commitment by a major model provider to adopt a standardized method for respecting website operators’ AI training preferences. The shared directive is intended to simplify compliance for websites that wish to exclude their content from AI training datasets.

The change does not alter OpenAI’s existing legal positions or terms of service but provides an additional, machine-readable signal for websites to communicate their preferences. Website operators can add the directive to their robots.txt files to indicate that participating AI providers should not crawl or use their content for training purposes.

Sources
  1. 01OpenAI — News — Making it easier for websites to express AI training preferences
Also on Models

Stories may contain errors. Dispatch is assembled with AI assistance and curated by human editors; despite the trust-score filter, mistakes happen. We correct publicly — every article links to its revision history. Nothing here is financial, legal, or medical advice. Verify before relying on any claim.

© 2026 Dispatch. No ads. No sponsorships. No paid placement. Reader-supported via Ko-fi.

Built by a person who cares about honest AI news.