OpenAI builds GPT-Red, an LLM-based red-teaming tool to probe its models for cyberattack vulnerabilities

OpenAI has built GPT-Red, an LLM designed to simulate adversarial attacks on its own models to strengthen their defenses against cyber threats. The system automates a form of safety evaluation known as red-teaming, which traditionally relies on human testers to probe for weaknesses such as prompt injection, code sabotage, or data exfiltration. OpenAI states that training its models against GPT-Red contributed to the robustness of its latest flagship model, GPT-5.6, which was released the week prior to the report.

GPT-Red was developed through a self-play loop in which it attacked other LLMs while those models defended themselves, improving both attacker and defender over successive rounds. The training environment, described as a "dojo," mimicked real-world scenarios including web browsing, email and calendar interactions, and code editing. According to OpenAI researchers, GPT-Red proved highly effective at identifying the most impactful attack paths, outperforming human red-teamers in a 2025 experiment that tested both approaches on an earlier version of GPT-5.

Among the novel attack techniques GPT-Red uncovered is a method called "fake chain of thought," where adversarial inputs insert false reasoning traces into a model’s internal process, tricking it into accepting and acting on incorrect information. For example, an attacker could inject a false premise such as "1+1=3" into a model’s chain of thought, leading the model to produce incorrect outputs without overt prompting. OpenAI tested GPT-Red’s attacks against Vendy, a vending machine agent developed by Andon Labs, and demonstrated that GPT-Red could manipulate prices and cancel orders.

OpenAI reports that over 90% of GPT-Red’s strongest attacks succeeded against GPT-5 (released August 2025), but fewer than 23% succeeded against the newer GPT-5.6. The company emphasizes that GPT-Red is intended to supplement, not replace, human red-teamers, noting that GPT-Red struggles with multi-turn conversational attacks and image-based prompt injections. OpenAI does not plan to release GPT-Red, citing both competitive and safety concerns, and asserts that replicating such a system would require significant resources and compute.

Researchers outside OpenAI, such as Jessica Ji at Georgetown University’s Center for Security and Emerging Technology, view the self-play approach as promising but stress that human expertise remains critical for identifying where additional testing is most needed.