OlympiadBench scales Olympiad-level evaluation to bilingual multimodal science

The benchmark was designed to remain difficult for frontier text and multimodal models and includes detailed evaluation methodology.

What happened?

OlympiadBench introduced 8,476 bilingual multimodal Olympiad-level math and physics problems with expert step-by-step annotations.

Why it matters

It broadened AI4Math evaluation beyond English text-only competition problems.

Significance rationale: OlympiadBench provided a large bilingual benchmark that combined mathematics, physics and multimodal problem statements at Olympiad difficulty.

Historical context

Primary evidence

Evidence & status at a glance

Significance
H2 · Field milestone
Verification
Paper released
Evidence level
E3 · Inspectable artifact
AI role
Computation only
Formal assurance
Artifact available

How should these labels be read? →

Contribution & verification

Mathematical novelty
Tooling only
Interface
Informal, Symbolic, Numeric
AI contribution
Language and multimodal models are evaluated on the benchmark.
Human contribution
Researchers curated problems, annotations and evaluation rules.
Verification
Paper released · E3 · Inspectable artifact · Artifact available
Last reviewed
2026-09-11

Technical details

Systems
olympiadbench
Organizations
tsinghua-university, openbmb
Problems
olympiad-mathematics, olympiad-physics
Methods
benchmarking, multimodal-evaluation

Sources & artifacts

Verification history

  • 2024-02-21 · Paper released · E2 · Research record

    The Chronicle records this event from the cited primary or official source; the status does not imply independent Chronicle verification.

  • 2026-09-11 · Paper released · E3 · Inspectable artifact

    This corpus refresh added a directly inspectable public artifact maintained by the authors or releasing organization. E3 records artifact-level inspectability and does not imply independent verification of the underlying scientific or mathematical claim.

Corrections

No corrections recorded.