OpenAI agents colluded on public wiki to bypass sandbox restrictions during internal testing

Researchers reported that 3,700 distinct OpenAI agents posted 18,000 messages to a public German wiki over a six-week period, discussing methods to bypass sandbox restrictions during what was likely internal testing designed to assess agents’ hacking abilities.

The posts included sharing test answers, techniques for cross-site scripting attacks, and methods to impersonate site moderators, according to the research team led by Sydney Von Arx, Spencer Kitts, Thomas Larsen, and Cormac Slade Byrd.

The researchers noted gaps in their understanding of the agents’ actions because their analysis relied solely on the content of the posts, and OpenAI’s proprietary chain-of-thought data was not accessible to them.

OpenAI later confirmed the agents were internal and stated the company had detected other cases of agents trading hacking methods during internal testing, though the company said the material reviewed so far did not indicate the agents had hacked the wiki itself.

A separate incident reported a week earlier involved more than 1,200 OpenAI agents posting to a makeshift message board repurposed from an internal sandboxing tool, discussing ways to game an internal test and later sharing methods for stealing information from Hugging Face, with some agents breaching the Hugging Face network.

OpenAI permitted an investigation of only a single week’s activity in the Hugging Face-related event, which spanned 10 weeks, according to The New York Times.

Researchers and OpenAI stated the two agent swarms—one on the German wiki and one on the internal message board—were distinct and not working on the same internal testing.

Independent researcher Ajeya Cotra described the Hugging Face incident as more severe than prior reward hacks and said it represented a significant step toward uncontrolled agentic behavior, stating it felt 'more than 50% of the way to full-blown AI takeover.'