AIMO Proof Pilot evaluates fully open models on expert-graded Olympiad-style proofs

The pilot restricted participants to models whose intermediate checkpoints, training protocols and training data were public. Six teams produced evaluated outputs on six proof problems, and AIMO compared them under the same expert-grading procedure with open-weight and commercial frontier models…

What happened?

AIMO reported that the winning Proof Pilot team led by Yi-Chia Chen scored 29 of 42 across six National-Olympiad-hard to IMO-medium problems using fully open-source models, compared with 1 of 42 for the strongest provided open baseline; every submitted proof was graded independently by at least two expert human graders and the outputs and grading data were released publicly.

The pilot restricted participants to models whose intermediate checkpoints, training protocols and training data were public. Six teams produced evaluated outputs on six proof problems, and AIMO compared them under the same expert-grading procedure with open-weight and commercial frontier models while publishing the submissions and detailed grading data.

Why it matters

The event supplied a concrete proof-quality benchmark for the fully open model ecosystem, where reproducibility and training transparency were part of the competition contract rather than optional release practices, and where success was measured on human-readable proofs instead of only final answers.

Significance rationale: The AIMO Proof Pilot combined full Olympiad-style proof generation, expert human grading and unusually strict openness requirements covering model checkpoints, training protocols and training data, creating a reproducible proof-writing milestone distinct from final-answer competitions and closed frontier-model demonstrations.

Historical context

Primary evidence

Evidence & status at a glance

Significance
H2 · Field milestone
Verification
Claimed
Evidence level
E3 · Inspectable artifact
AI role
Computation only
Formal assurance
Artifact available

How should these labels be read? →

Contribution & verification

Mathematical novelty
Benchmark result
Interface
Informal
AI contribution
Participating AI systems generated complete natural-language proofs and were the objects under evaluation; humans defined the competition protocol, enforced openness requirements and graded proof quality.
Human contribution
Competition organizers selected the proof problems and openness rules, teams trained or adapted open models and systems, and expert graders independently assessed each proof to establish final scores.
Verification
Claimed · E3 · Inspectable artifact · Artifact available
Last reviewed
2026-09-11

Technical details

Systems
aimo-proof-pilot
Organizations
aimo-prize, xtx-markets
Problems
olympiad-proof-generation
Methods
open-source-model-training, proof-generation, expert-grading, reproducible-evaluation

Sources & artifacts

Verification history

  • 2026-07-07 · Claimed · E2 · Research record

    AIMO published the Proof Pilot results, team identities, per-problem scores, comparison models and expert-grading procedure.

  • 2026-09-11 · Claimed · E3 · Inspectable artifact

    AIMO links the anonymized team/model outputs and a public grading dataset on Kaggle. E3 records inspectable competition artifacts and does not imply Chronicle independently regraded every proof or reproduced the teams' training runs.

Corrections

No corrections recorded.