OpenAI’s GPT-Red Uses Self-Play to Harden GPT-5.6 Against Prompt Injection
OpenAI says its internal GPT-Red attacker helped cut GPT-5.6 Sol’s direct prompt-injection failure rate to 0.05% in held-out tests.
OpenAI has disclosed GPT-Red, an internal model trained to find prompt-injection vulnerabilities in other AI systems and convert successful attacks into training data for production models. The company says this adversarial training helped GPT-5.6 Sol reduce failures on its hardest direct prompt-injection benchmark to one-sixth the rate of OpenAI’s best production model four months earlier.
On a broader collection of held-out environments, OpenAI reports that GPT-5.6 Sol failed on just 0.05% of GPT-Red’s direct prompt-injection attempts. The company describes Sol as its most prompt-injection-resistant production model to date.
GPT-Red is not a ChatGPT option, API model, or security product that customers can deploy. OpenAI is keeping it separate from publicly available models because it was deliberately trained to discover effective attacks. Its output instead feeds an internal testing and adversarial-training pipeline that OpenAI says has been used for successive production models since GPT-5.3.
How GPT-Red trains against defending models
GPT-Red uses self-play reinforcement learning. During training, it attacks a collection of defender language models across scenarios designed to resemble the environments in which agents encounter untrusted information.
The attacker receives a goal, sends a prompt to a defender, observes the response, and iterates. It earns a reward when it causes a valid failure, such as persuading an agent to follow an injected instruction. The defender is rewarded for resisting the attack while still completing the user’s legitimate task.
This second requirement matters. A model can reduce its apparent attack surface simply by refusing more requests or avoiding tool use, but such behavior would also make it less useful. OpenAI says it evaluated GPT-5.6 on general capabilities and targeted over-refusal tests and found that its normal capabilities were unaffected by the robustness training. The company has not yet published the underlying results needed to assess that claim independently.
The training environments specify both an attacker’s control and the condition for success. GPT-Red might be allowed to manipulate part of a local file, an email body, a banner on a webpage, or the output returned by a tool. As the defender population improves, the attacker must discover different or more effective instructions to continue receiving rewards.
OpenAI says GPT-Red was trained at a compute scale comparable to some of its largest post-training runs. By the end of training, it could break nearly every internal and production model placed against it, including GPT-5.5. The resulting attacks were then used to train GPT-5.6 rather than being incorporated into a publicly deployed attacker.
GPT-Red found attacks humans missed
To test whether GPT-Red could generalize beyond its training environments, OpenAI reproduced an indirect prompt-injection arena developed by Mateusz Dziemian and other researchers. GPT-Red and human red-teamers separately attempted to compromise GPT-5.1 in scenarios that OpenAI says were distinct from those used during GPT-Red’s training.
GPT-Red found a successful attack in 84% of the scenarios, compared with 13% for the human participants. That result measures the share of scenarios in which an attacker found at least one successful strategy; it should not be read as an 84% probability that any single injection will work.
The underlying public competition illustrates why automated testing is attractive. Its researchers recorded 272,000 attack attempts from 464 participants against 13 frontier models. They found 8,648 successful attacks across 41 scenarios, covering tool use, coding, and computer-control agents. Every evaluated model was vulnerable, with per-attempt attack success rates ranging from 0.5% to 8.5%.
Those public results are not directly comparable with OpenAI’s 84% figure. OpenAI used an internal mirror of the arena, a different target model, and a scenario-level success measure. The comparison nevertheless tests an important property: whether an automated attacker trained in one set of environments can transfer its methods to previously unseen tasks.
An earlier GPT-Red system also discovered what OpenAI calls “Fake Chain-of-Thought” attacks. These direct injections present fabricated reasoning as if the target model had already produced or verified it. OpenAI says the attacks succeeded more than 95% of the time against GPT-5.1 but now succeed in fewer than 10% of attempts against GPT-5.6 Sol.
Several separate indirect-injection benchmarks targeting browsing and developer tools have now exceeded 97% defensive accuracy, according to OpenAI. That creates another reason to use an adaptive attacker: a fixed benchmark can stop revealing meaningful differences once newer models learn to defeat its known attacks.
Attacks transferred from simulation to live agents
OpenAI also tested GPT-Red against systems that could take real actions rather than merely produce unsafe text.
In one case, GPT-Red targeted Vendy, an autonomous agent operating a vending machine in an OpenAI office. The attacker first experimented against a simulated system that closely reproduced the production agent and exposed its tool calls. It then transferred the resulting attacks to the live deployment.
OpenAI says GPT-Red achieved three assigned objectives: it reduced the price of an expensive in-stock item to the system’s minimum of $0.50, ordered a new item worth more than $100 and offered it for $0.50, and cancelled another customer’s order. The vulnerabilities were disclosed to the system’s developer, Andon Labs, and OpenAI said additional safeguards were being tested.
A second case targeted a Codex CLI agent powered by GPT-5.4 mini. GPT-Red attempted to make the coding agent exfiltrate sensitive information across 10 held-out scenarios. OpenAI reports that it succeeded in more scenarios and used fewer tokens than a prompted GPT-5.5 baseline, although the announcement does not provide the underlying success rates or token counts.
These cases demonstrate the difference between an undesirable model response and an agent-security failure. Once a model can access files, send requests, edit records, or execute commands, an injected instruction can cross from text into an external system. The severity then depends not only on the model’s behavior but also on the permissions, confirmation rules, and server-side controls surrounding it.
The reported gains still need independent scrutiny
The published figures are OpenAI’s internal evaluation results. The company has not disclosed the number of environments behind every metric, the sampling budget available to GPT-Red, confidence intervals, the precise human testing conditions, or enough technical detail to reproduce the training process.
OpenAI said a more detailed preprint would follow later in the week, but no GPT-Red preprint was available at the time of publication. Until that material appears, the 0.05% failure rate should be understood as performance against OpenAI’s specified held-out attacks, not proof that GPT-5.6 Sol is immune to prompt injection in arbitrary deployments.
Independent reporting by SiliconANGLE also noted limitations in multi-turn conversational attacks and image-based prompt injection, citing coverage based on interviews with GPT-Red’s creators. Those gaps leave roles for human specialists who can invent new threat models, combine vulnerabilities across systems, and examine modalities not well represented in automated training.
OpenAI itself continues to describe prompt injection as an open security problem. Its broader defense includes model training, automated monitors, sandboxing, user confirmations for sensitive actions, internal and external red teams, and a bug-bounty program. GPT-Red strengthens the training component; it does not replace those other controls.
For users, the immediate benefit is therefore indirect: GPT-5.6 incorporates adversarial examples generated by GPT-Red. Developers cannot call GPT-Red to audit their own agents, and they should not treat model-level resistance as a substitute for least-privilege tools, explicit authorization checks, restricted data access, or confirmation before consequential actions.
Sources
Share