Meta's AIRA3 Ranks Eighth in NVIDIA's 4,182-Team Nemotron Challenge

1. A live competition put Meta's research system against thousands of teams

Meta says its AIRA₃ autonomous research system placed eighth in NVIDIA’s Nemotron Model Reasoning Challenge, earning a Kaggle gold medal among 4,182 teams.

The result came from a live competition rather than a retrospective benchmark assembled by Meta. Kaggle records show that the challenge ran from March 16 to June 15, 2026, attracted 5,223 participants and received 71,743 submissions. Meta entered AIRA₃ in June and says its final work was evaluated through the same private test set used to rank the other competitors.

Eighth place is not a competition victory: seven teams finished above AIRA₃, and Kaggle’s cash prizes were reserved for the top three. “Gold” refers to the platform’s medal tier. The distinction matters because descriptions of the result as AIRA₃ “beating 4,000 teams” erase both the higher-ranked entries and the difference between a medal and first place.

Even with that qualification, the placement provides stronger evidence than an internal demonstration. The target model, submission format, evaluation procedure and deadline were set by NVIDIA and Kaggle. Meta could optimize against the published task, but it did not control the final scoring environment.

Meta interprets the result as evidence that AIRA₃ can improve a specific model capability at a level comparable to human experts. The ranking supports the narrower claim that the system produced a highly competitive fine-tuning submission. It does not by itself establish general scientific ability or equivalence to human researchers across open-ended work.

2. What the Nemotron challenge actually measured

The competition asked entrants to improve the reasoning accuracy of NVIDIA-Nemotron-3-Nano-30B while retaining the same base model. The required submission was a compatible low-rank adaptation, or LoRA, adapter with a maximum rank of 32.

That constraint made the contest a targeted model-improvement problem rather than a search for the strongest unrestricted language model. Entrants could change the training data, prompting strategy, synthetic-data pipeline, reinforcement-learning method or lightweight fine-tuning process, but their final adapter had to run on NVIDIA’s designated model and evaluation stack.

NVIDIA’s technical report describes Nemotron 3 Nano as a hybrid Mamba-Transformer mixture-of-experts model. It contains 31.6 billion parameters in total, with 3.2 billion activated during each forward pass, or approximately 3.6 billion when embeddings are included.

According to the winning team’s public write-up, the supplied training set contained 9,500 labeled problems covering bit manipulation, numerical equations, cryptarithms, text ciphers, numeral systems, unit conversion and gravity. Each prompt supplied examples of an unknown transformation and asked the model to infer the rule before answering a new case.

Kaggle loaded each submitted adapter with the vLLM inference engine. Answers were extracted primarily from a LaTeX \boxed{} expression and counted as correct when they matched the reference string exactly or fell within a relative numerical tolerance of \(10^{-2}\). The final score was the proportion of correct answers.

The fixed inference configuration included an 8,192-token model length, up to 7,680 generated tokens, a temperature of zero and top_p set to 1.0. These controls reduced differences caused by sampling behavior and concentrated the ranking on what each adapter had learned.

The private test set limited direct optimization against final answers, but it did not create a controlled human-versus-AI experiment. Kaggle teams could use different compute budgets, external resources and automated tools, and some submissions may themselves have relied heavily on AI. The leaderboard therefore compares completed systems and workflows, not unaided individual researchers under identical laboratory conditions.

3. AIRA3 used multiple models and coding harnesses

Meta says AIRA₃ does not depend on a single agent or a central controller. It runs multiple long-lived agents in isolated environments, with each agent pairing a language model with a coding harness.

The agents coordinate asynchronously through two shared mechanisms. A forum carries hypotheses, experimental findings and discussion, while a shared filesystem stores solution artifacts. Individual agents decide which discoveries to investigate or extend as information accumulates.

This design is intended to let useful findings survive beyond the context window or runtime of one agent. An experiment performed by one model-and-harness pair can become input for another, allowing later work to build on earlier results instead of restarting the search independently.

For the live Nemotron competition, Meta used an ensemble combining GPT 5.5 with OpenCode and Claude 4.8 with Claude Code. The eighth-place result therefore belongs to the complete AIRA₃ workflow, not to one underlying language model.

Meta also reported several post-competition evaluations on the same private test set. A configuration using Muse Spark 1.2 with MuseCode reached what Meta described as gold-medal performance. Muse Spark 1.1 with OpenCode and GLM 5.2 with OpenCode reached the silver-medal range.

Those post-hoc results are useful evidence that the orchestration scheme can work with different model-and-harness combinations, but they are not additional live placements. Only the GPT 5.5 and Claude 4.8 ensemble received the reported eighth-place rank during the competition.

4. How AIRA3 extends Meta's previous research-agent work

AIRA₃ follows the AIRA₂ system that Meta documented in April 2026. That earlier project focused on three bottlenecks in autonomous machine-learning research: limited experimental throughput, unreliable validation signals during long searches and the restricted behavior of fixed, single-turn model operators.

AIRA₂ addressed those problems with an asynchronous multi-GPU worker pool, a Hidden Consistent Evaluation protocol and ReAct agents capable of interactively scoping and debugging their work.

Meta reported that AIRA₂ achieved a mean percentile rank of 81.5 after 24 hours and 83.1 after 72 hours on MLE-bench-30, compared with 72.7 for the strongest baseline. On its separate AIRS-Bench suite, AIRA₂ exceeded the stated human state of the art on six of 20 tasks.

The AIRA₃ announcement moves the evaluation from selected research benchmarks to an active competition with an external deadline and private scoring. Its forum-and-filesystem design also emphasizes collaboration among persistent agents, rather than presenting parallel workers only as a way to increase experiment throughput.

Meta has not yet published an AIRA₃ paper, source release or detailed technical report. The announcement does not disclose the number of agents, total compute expenditure, training recipe, experiment count, intervention by Meta researchers or an ablation separating the value of shared knowledge from the value of running more trials. It is therefore not yet possible to reproduce the eighth-place workflow or determine which component contributed most.

5. What the result establishes—and what remains unverified

The clearest practical result is that an autonomous research workflow produced a top-10 solution to a constrained model-training problem under an external evaluation. For developers building research agents, that demonstrates a workflow capable of managing data generation, experimentation, fine-tuning and selection well enough to compete with experienced Kaggle teams.

The result does not show that AIRA₃ independently chose a valuable research question, designed the evaluation or improved its own underlying agent models. It optimized a predefined capability of a separate Nemotron model against a known competition objective. Describing this as recursive self-improvement would go beyond the evidence currently available.

Meta says the same system can move between domains by changing only its task specification. It reported a 27% latency reduction for production GPU kernels in an internal benchmark and gold-level performance in Kaggle’s Deep Past Challenge, which involved translating transliterated Old Assyrian Akkadian into English.

The Deep Past competition is independently documented by Kaggle, but Meta has not supplied an identifiable AIRA₃ leaderboard entry, score or technical write-up for that run. The GPU-kernel figure is also internal and lacks a disclosed workload, baseline, hardware configuration or measurement method. Both results should therefore be treated as Meta-reported transfer tests rather than independently reproducible evidence.

For companies evaluating autonomous research systems, the Nemotron placement establishes competitive task performance but leaves cost and efficiency unanswered. Without compute usage, human-supervision records and controlled comparisons against independent parallel agents, the result cannot show whether AIRA₃ is more economical than an expert team or whether its shared coordination mechanisms outperform an equivalent amount of uncoordinated search.

Frequently Asked Questions

What is AIRA3?

AIRA₃ is Meta’s autonomous AI research system. Meta describes it as multiple long-running model-and-coding-harness pairs that work in isolated environments and coordinate through a shared forum and filesystem.

Did AIRA3 win the NVIDIA competition?

No. It finished eighth and earned a Kaggle gold medal. Seven teams ranked higher, while the competition’s placement prizes covered the top three.

Which models powered the live submission?

Meta says the live entry combined GPT 5.5 running with OpenCode and Claude 4.8 running with Claude Code.

What did AIRA3 change in Nemotron?

It produced a LoRA adapter, with rank limited to 32, for NVIDIA’s Nemotron-3-Nano-30B base model. The competition measured whether the adapted model answered structured reasoning problems more accurately.

Is AIRA3 publicly available?

Meta’s announcement did not provide an AIRA₃ source release, downloadable system or full technical paper. Its exact training process and compute requirements remain undisclosed.

Sources

• AI at Meta: AIRA₃ Kaggle result and announcement thread
• Kaggle: NVIDIA Nemotron Model Reasoning Challenge
• Kaggle: First-place Nemotron challenge solution
• Meta AI: AIRA₂ — Overcoming Bottlenecks in AI Research Agents
• NVIDIA researchers: Nemotron 3 Nano technical report
• Kaggle: Deep Past Challenge — Translate Akkadian to English