OpenAI GPT-Red automated red-teaming model

GPT-Red operates inside what OpenAI calls a 'dojo' - a simulated environment replicating real-world agentic tasks including web browsing, email, calendar management, and code editing. In each training round, GPT-Red attacks a collection of diverse defender LLMs simultaneously. The reward structure is adversarial by design: GPT-Red earns credit for eliciting a valid failure such as a successful prompt injection, while each defender earns credit for resisting the attack and completing its original task. ^[1]

The flywheel effect is the key innovation. As defenders become more robust through training, GPT-Red is forced to discover stronger and more diverse attacks to keep earning rewards. This creates a capability floor that rises automatically - the system does not plateau at today's known attack library but actively searches for tomorrow's. OpenAI Research Scientist Dylan Hunn framed this explicitly: 'As more capable models become available, we will have already designed the system that can discover new modes of attack.' ^[2]The implication is structural: GPT-Red is infrastructure that compounds with model capability rather than a static test suite that goes stale.

The most consequential finding from GPT-Red's training run is an attack class that OpenAI researchers say they had not previously encountered: fake chain-of-thought (CoT) injection. In this attack, a malicious prompt inserts a fabricated reasoning step into the model's visible chain of thought - effectively spoofing an entry in the model's internal deliberation log. Because the model is trained to trust its own prior reasoning as verified context, it treats the forged premise as established fact and acts accordingly. Research Scientist Chris Choquette-Choo described the mechanic: 'It's like if I told you that 1+1=3 and that you have verified this already.' ^[2]

This attack surface did not meaningfully exist before reasoning models became the dominant paradigm. Chain-of-thought outputs were originally an interpretability aid; they have since become load-bearing components of how frontier models make decisions. GPT-Red's discovery reveals that externalizing a model's reasoning trace - a feature intended to improve transparency - simultaneously creates a writable attack surface. On GPT-5.1, fake CoT attacks succeeded in over 95% of attempts. On GPT-5.6 Sol, trained against GPT-Red, that rate fell below 10%. ^[2]

The performance gap between GPT-Red and human red-teamers is not incremental - it is structural. In internal evaluation scenarios against novel safety environments, GPT-Red succeeded 84% of the time. Human red-teamers in the same tests succeeded 13% of the time. ^[3]That six-to-one ratio reflects something beyond speed or scale: the model is better at generating attacks that actually work, not merely generating more of them.

The downstream impact on GPT-5.6 Sol is equally striking. Prompt injection attack success rates dropped from over 90% against GPT-5 to under 23% against GPT-5.6 - a reduction of more than three-quarters. GPT-5.6 Sol also recorded six times fewer failures on direct prompt injection benchmarks compared to its predecessor from just four months prior. ^[3]Critically, OpenAI reports that models trained with GPT-Red showed no increased refusal rates or decreased performance on legitimate tasks - meaning the robustness gains came without the usual capability-safety tradeoff that has historically constrained this work. ^[4]

GPT-Red's limitations are as informative as its strengths. The system struggles with two attack categories that human red-teamers handle without difficulty: multi-turn conversational attacks and image-based prompt injections. ^[2]Multi-turn attacks require strategic patience - building rapport, establishing false context across several exchanges, then exploiting accumulated trust. This is a distinctly human social-engineering skill that does not map cleanly onto single-pass optimization.

Image-based injections exploit the visual modality: instructions embedded in images that the model processes as content rather than commands. Both gaps point to a common underlying weakness - GPT-Red was trained primarily in text-based agentic task environments, and its attack vocabulary reflects that domain. Georgetown CSET's Jessica Ji acknowledged the results while noting that 'human expertise will still be very important,' a view borne out by these specific gaps. ^[2]OpenAI's decision to keep GPT-Red internal also raises a related accountability question: the same capabilities that make it an effective internal hardening tool would make it dangerous in adversarial hands - and without external access, independent researchers cannot audit whether the claimed improvements hold under conditions OpenAI did not test. ^[3]