TL;DR: GPT-Red in 30 Seconds
  • GPT-Red is an automated red-teaming model OpenAI trained to break its own models; it is not public and never will be.
  • It was trained with self-play: attacker GPT-Red and defender models played against each other, both getting stronger over time.
  • In scenarios where human red-teamers succeeded only 13% of the time, GPT-Red hit 84%.
  • GPT-5.6 was trained against GPT-Red, making it OpenAI’s most robust model against prompt injections (only a 0.05% failure rate).

OpenAI just announced something unusual: not a model for users, but an “AI hacker” trained to break its own models. Meet GPT-Red! According to the research publication “Unlocking Self-Improvement for Robustness” released today, GPT-Red automates the work of human security testers (red teams) and does it far better than humans.

Why would a company train an AI to attack its own models? The answer makes perfect sense: to find the vulnerabilities before malicious actors do. Let’s dive in!

What Is GPT-Red?

GPT-Red is OpenAI’s automated red-teaming model. Its job is to launch prompt injection attacks against GPT models and expose their weak spots. It works just like a human hacker: it sends a prompt to the target, observes how the model responds, and refines its attack step by step.

Here is the critical part: GPT-Red is not public and never will be. Because OpenAI deliberately trained attack capabilities into it, the model is kept completely separate from anything that gets deployed. The goal is to harden production models without handing those capabilities to adversarial actors.

What Is Red-Teaming?
Red-teaming means trying to break a system before release to discover its weaknesses. It is normally done by teams of human experts. But human testing is slow and expensive, and it cannot generate the volume of adversarial data needed to train models. GPT-Red solves exactly that bottleneck.

Self-Play: An Endless Duel of Attackers and Defenders

GPT-Red’s training method is fascinating. OpenAI used self-play reinforcement learning:

  • GPT-Red (attacker): Earns a reward when it pulls off a successful prompt injection.
  • Defender models: Earn rewards by resisting the attack and completing their original tasks.

As the defenders get tougher, GPT-Red is forced to discover stronger and more creative attacks. The loop creates an arms race that continuously improves both sides.

The training scenarios come straight from real life: instructions hidden inside third-party data such as part of a local file, a webpage banner, an email body, or a tool output. OpenAI says GPT-Red was trained at the compute scale of some of its largest post-training runs. That is a massive amount of compute spent purely on safety!

How Strong Is It? 84 to 13 Against Humans

The numbers are striking. By the end of training, GPT-Red can break nearly every model it faces, up to and including GPT-5.5.

Even more impressive, it succeeds in scenarios it has never seen. On a replicated version of the indirect prompt injection arena from Dziemian et al. (2025), both human red-teamers and GPT-Red designed attacks against GPT-5.1. The result:

  • GPT-Red: produced successful attacks in 84% of scenarios
  • Human red-teamers: only 13%
Real-World Test: It Broke a Vending Machine Agent
OpenAI pitted GPT-Red against Vendy, the AI-powered vending machine agent (built by Andon Labs) in its office. GPT-Red first refined its attacks in simulation, then deployed them against the live system and achieved all three of its malicious objectives: it dropped the price of an expensive item to $0.50, ordered a new $100+ item and listed it for $0.50, and canceled another customer’s order.

GPT-Red was also tested against a Codex CLI agent (based on GPT-5.4 mini) on 10 held-out data exfiltration scenarios. Compared to a prompted GPT-5.5 baseline, it succeeded in more scenarios while using fewer tokens.

Fake Chain-of-Thought: An Attack Nobody Had Seen Before

GPT-Red’s most remarkable discovery is a brand-new attack class called “Fake Chain-of-Thought”. In this attack, the model injects a fake entry into the target AI’s chain of thought. The target mistakes the fake entry for its own reasoning and acts on the bogus “system rule” embedded inside it.

In the example OpenAI shared, the attack hides fake metadata inside a file search result and tricks GPT-5.1 into uploading sensitive files to an external server. The numbers are alarming:

  • Success rate against GPT-5.1: above 95%
  • Against GPT-5.6 Sol: below 10%

The drop is no accident: GPT-5.6 was trained specifically against these attacks.

So That’s Why GPT-5.6 Is So Robust

The secret behind the record robustness scores of the recently introduced GPT-5.6 series is now out. OpenAI has been adversarially training every production model since GPT-5.3 against GPT-Red’s precursors. The results:

  • GPT-5.6 Sol shows 6x fewer failures on OpenAI’s hardest direct prompt injection benchmark compared to its best production model from just four months earlier.
  • Against GPT-Red’s direct prompt injections, GPT-5.6 Sol fails only 0.05% of the time.
  • Several indirect prompt injection benchmarks targeting developer tools and browsing are now saturated at over 97% accuracy.
Did Robustness Kill Capability?
A model can look “safer” simply by refusing more requests or doing less. OpenAI is aware of this trap: it evaluated both general frontier capabilities and targeted over-refusal tasks. The verdict: normal capabilities remain unaffected while robustness improved significantly. The gains come from resisting malicious instructions, not from refusing legitimate requests.

AI That Improves Its Own Safety

GPT-Red matters beyond a single model. OpenAI describes it as a safety flywheel: today’s models are used to make tomorrow’s models safer. As capabilities grow, safety testing scales right along with them.

The company also announced that a pre-print with more technical details will be released later this week.

Conclusion

GPT-Red signals a real paradigm shift in AI safety: where human testers were the bottleneck, AI now handles both the attack and the defense. GPT-5.6’s impressive robustness numbers show the approach is working.

What do you think? Is training an AI to attack your own models a genius move or a risky gamble? Let’s meet in the comments! 👇

AI-Generated Content Notice
This blog post is entirely generated by artificial intelligence. While AI enables content creation, it may still contain errors or biases. Please verify any critical information before relying on it.