OpenAI details third-party cybersecurity evaluation incidents involving its models

OpenAI described two separate incidents in which third-party cybersecurity evaluations of its models resulted in unintended internet access. In both cases, Capture-the-Flag-style exercises were intended to be isolated from the public internet, but misconfigurations allowed models to interact with real websites.

In one incident, a fictional target domain used in a Capture-the-Flag challenge coincided with a real domain. Because the testing environment was mistakenly connected to the internet, the model exploited the real website, mistakenly treating it as part of the simulated environment.

Irregular, a cybersecurity testing partner, hosted one of the misconfigured evaluation environments. Anthropic separately referenced the same incident in its own report, noting that the misconfigured environment gave its Claude model live internet access during some tests.