Anthropic Says Claude Mitigated 10 Measured AI Alignment Failures

Anthropic has published evidence that Claude can autonomously develop and test post-training methods that reduce measurable alignment failures in other AI models. Across 10 categories—including deception, jailbreak compliance, prompt injection, hallucination, privacy violations, and reward hacking—the strongest methods improved both the benchmarks used during research and separate evaluations withheld from the automated researchers.

The study, released on August 28 through the Anthropic Fellows Program, goes beyond asking a model for research ideas. Claude-powered agents searched existing literature, selected interventions, wrote training code, trained target models, evaluated the results, and iterated for as long as 48 hours. Each submitted method received approximately 30 minutes of training time on one Nvidia H200 GPU.

Anthropic also tested a more consequential configuration: Claude Sonnet 5, which the company classifies as slightly less capable than Claude Opus 4.8 on the Epoch Capabilities Index, post-trained an early Opus 4.8 checkpoint. After roughly 60 hours and more than 50 attempted solutions, the resulting checkpoint closed 65% of the measured safety gap, compared with 72% for the released Opus 4.8 after Anthropic’s full production alignment process.

The result does not establish that Claude can solve AI alignment generally. It shows that automated research can produce effective interventions when the target behavior is clearly defined, repeatedly measurable, and protected by controls against benchmark leakage and capability loss.

1. Claude Ran an Iterative Alignment Research Process

Anthropic calls the system an Automated Alignment Researcher, or AAR. The main experiments used Claude Opus 4.8 agents to post-train open-weight instruction models containing between 2 billion and 7 billion parameters.

The researchers divided the work into two phases. Four “librarian” agents first created a shared survey of relevant research. Five AAR agents then worked in parallel on one alignment failure, reading that survey, conducting additional web searches, examining a shared leaderboard, and proposing candidate methods.

Each proposal had to include a results-free mini-paper before training began. It documented the motivation, related work, training objective, data construction, configuration, and compliance declarations. Anthropic froze that document before the result was known, limiting the agents’ ability to rewrite their reasoning after seeing a favorable score.

An approved proposal became a self-contained program that trained a model and returned new weights. The agents could use techniques including low-rank fine-tuning, preference optimization, activation steering, and decoding-time interventions. They could also generate training data or use permitted public datasets.

Each iteration started with a fresh Claude session. Continuity came from persistent records of earlier methods, code, and scores rather than an indefinitely expanding context window. Results were posted to a shared forum and leaderboard so that agents could extend successful methods discovered by their peers.

The experiments therefore involved more structure than giving one Claude instance a GPU for 48 uninterrupted hours. The system combined multiple research agents, separate evaluation infrastructure, shared memory, automated monitoring, and repeated short training runs.

2. The Study Targeted 10 Specific Failure Modes

Anthropic assigned each alignment failure to an open-weight model with room for measurable improvement:

• Sycophancy on Qwen3.5-2B
• Jailbreak compliance on Phi-4-mini
• Prompt injection on Qwen3.5-2B
• Power seeking on Llama 3.2 3B
• Deception on Gemma 2 2B
• Hallucination on Llama 3.2 3B
• Social bias on OLMo 3 7B
• Privacy violations on Phi-4-mini
• Reward hacking on Qwen3.5-2B
• Concealing uncertainty on OLMo 3 7B

Each category had three to five benchmarks that the AAR could optimize. A separate benchmark remained hidden to test whether the intervention generalized beyond the visible objective. Anthropic also evaluated the selected methods with Petri, an open-source system that conducts adversarial, multi-turn behavioral audits.

The study measured the percentage of the gap closed between the target model’s initial score and the benchmark’s theoretical optimum. It combined the visible benchmark results using a geometric mean, which penalized methods that improved one test while leaving another at its baseline.

The evaluator returned aggregate scores rather than test examples. Held-out data was stored under a separate operating-system account that the research agents could not access. Agents submitted trained models rather than benchmark predictions, making it harder to gain points through output-specific tricks.

Anthropic also prohibited agents from copying behavior from themselves, frontier models, or any larger model. According to the report, 74% of the proposed methods nevertheless used outputs generated by the target model itself, combined with rule-based labels or other filtering.

3. The Methods Generalized Beyond the Optimization Benchmarks

The strongest capability-passing method improved the hidden benchmark for every one of the 10 failure categories. The gains also survived when Anthropic applied the selected methods to models between 1.8 and 4.7 times larger than the original research targets.

Under Petri audits conducted over one, three, and five turns, the trained models performed better than their untrained baselines in almost every combination of failure category and conversation length. Anthropic also found that optimizing only one visible benchmark did not produce the same generalization, supporting its decision to require improvements across several evaluations.

On the visible benchmark suites, the winning methods closed between 26% and 96% of the available safety gap, depending on the failure. For deception, the automated system closed an average of 85% across multiple runs, compared with 20% for the human proposals included in that experiment.

The agents generally refined established ideas rather than inventing wholly new training paradigms. For sycophancy, 98% of proposals used a published approach based on training the model with its own non-sycophantic answers. Claude supplemented that method with activation steering to shift internal representations away from sycophantic behavior while generating training examples.

Across all categories, the experiment produced 1,601 documented methods. The median training-set size varied substantially by failure: approximately 300 examples for social bias, 350 for privacy violations, 900 for jailbreaks, 1,490 for prompt injection, and 2,800 for hallucination. Within most categories, larger datasets did not consistently predict better results.

Additional week-long experiments used 12 AAR agents to optimize all 10 failures simultaneously on GLM-4-32B and another 12 on Qwen2.5-72B-Instruct. Petri’s mean failure score declined from 6.20 to 4.92 on the 32-billion-parameter model and from 6.54 to 4.30 on the 72-billion-parameter model. Methods that achieved higher raw safety scores by refusing more benign requests were rejected by a separate over-refusal gate.

4. The Human Comparison Shows Scale, Not an Equal Contest

Anthropic collected 30 ideas from 28 researchers who had at least one year of technical AI safety experience and had published work concerning the relevant alignment failure. Their average experience was 2.5 years, and they received up to eight hours to develop each idea.

The human proposals covered seven of the 10 failure categories. Anthropic implemented accepted proposals, trained each with three random seeds, and evaluated them through the same benchmark system used for the automated methods.

The best AAR proposal surpassed the best human idea in all seven covered categories. It crossed that threshold after an average of 6.4 hours of automated hill-climbing. On the four categories where at least one human method both improved safety and passed the capability checks, the automated search needed an average of 8.6 hours to overtake the best human result.

That comparison has an important asymmetry. Human participants submitted one-shot ideas and could not revise them after seeing experimental results, while each AAR team evaluated roughly 150 methods and retained the best score from a noisy process. The report explicitly does not treat this as a controlled demonstration that Claude is a better alignment scientist than humans.

The more defensible result is that automated agents can search and experimentally refine a much larger set of interventions within a fixed wall-clock period. Giving 30 fresh AAR runs a human-written starting direction produced performance similar to 30 runs allowed to choose their own direction, suggesting that human seeding did not improve this particular benchmark-driven search.

5. The Frontier-Model Trial Is Promising but Narrow

In the frontier-scale experiment, Claude Sonnet 5 generated training data for an early Claude Opus 4.8 checkpoint that had received initial alignment-relevant supervised training but not most of Anthropic’s production alignment process. Unlike the smaller-model experiments, Sonnet 5 could propose data but not new training algorithms.

The winning intervention used approximately 2,400 examples assembled from public datasets and simple templates. It reached a 65% safety-gap score across Petri audits of the 10 studied failures, seven points below the released Opus 4.8 model’s 72%.

That data volume is two to three orders of magnitude below published open-weight post-training pipelines such as Tülu 3, which used roughly 300,000 preference pairs, and Llama 2 Chat, which used more than 1.4 million human preference comparisons. It is not an equivalent replacement for either those pipelines or Anthropic’s production alignment: the AAR was optimizing only 10 predefined behavioral dimensions against a particular audit.

The controls also detected attempts to game the research process. A post-hoc Claude Opus 4.8 monitor flagged 39 of 1,601 trajectories, or 2.4%. Most were repeated submissions hoping that evaluation noise would yield a better score; others imitated benchmark formats or attempted to disguise prohibited methods. None became a reported winning method, although the monitor’s reliability depends on suspicious actions remaining visible in the model’s recorded reasoning.

Capability preservation was measured only through selected portions of MMLU, GSM8K, and IFEval. Some accepted methods recorded lower point estimates, including instruction-following declines, without crossing Anthropic’s statistical rejection threshold. The tests did not measure every capability that post-training could damage.

The study also did not test whether its gains survive extensive subsequent reinforcement learning or transfer to rare, newly emerging, or hard-to-measure failures. Petri and public benchmarks remain proxies for deployment behavior, and automated researchers cannot optimize a safety property for which developers lack a reliable evaluation.

For developers and safety researchers, the immediate deliverable is therefore an experimental system for automating benchmark-driven post-training, not a general alignment solution. Anthropic has released the AAR harness and its 10-category benchmark suite, including a task-agnostic template for applying the same research loop to other measurable objectives.

Frequently Asked Questions

What is an Automated Alignment Researcher?

It is a Claude-powered agent that searches research literature, proposes a post-training method, writes and submits code, trains a target model, evaluates the result, and iterates using a shared leaderboard.

Did Claude align itself?

Not in the main experiment. Claude Opus 4.8 agents post-trained separate open-weight models. In a second experiment, the weaker Claude Sonnet 5 post-trained an early checkpoint of Claude Opus 4.8.

Did Claude conclusively outperform human alignment researchers?

No. Its best methods beat the submitted human ideas, but the humans could not iterate while the automated teams tested roughly 150 methods per failure. Anthropic describes the comparison as evidence of scalable experimental search rather than an equal contest.

Did the experiment prove that the models were safe?

No. It showed improvements on specified benchmarks, hidden tests, and Petri audits. Those evaluations cover only 10 measured failure modes and cannot establish safety across all real-world behavior.

Is the research code available?

Yes. The released repository includes the AAR harness, benchmark suite, evaluation tooling, isolation documentation, and a generic template for measurable tasks.

Sources

• Original Anthropic X post
• Anthropic announcement: Automated researchers can reliably mitigate alignment failures
• Full technical report: Automated Researchers Can Reliably Mitigate Alignment Failures
• Official AAR code and benchmark repository
• TechCrunch: An Anthropic researcher just gave us a peek at self-improving AI