Primary evidence
- Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language ModelsS1 · Primary research · Paper · 2024-10-10
- Official Omni-MATH benchmark repositoryS1 · Primary research · Repository
Omni-MATH refreshes Olympiad evaluation after older math benchmarks begin saturating
The benchmark was designed after frontier reasoning models approached saturation on older datasets such as MATH.
What happened?
Omni-MATH introduced 4,428 human-annotated competition problems spanning more than 33 subdomains and more than 10 difficulty levels.
Why it matters
It provided a more discriminating informal-math benchmark during the transition to reasoning-specialized models.
Significance rationale: Omni-MATH responded to rapid benchmark saturation by collecting a larger, finer-grained set of Olympiad-level mathematics problems.
Historical context
Earlier developments
Later developments
—
Related milestones
—
Evidence & status at a glance
- Significance
- H2 · Field milestone
- Verification
- Paper released
- Evidence level
- E3 · Inspectable artifact
- AI role
- Computation only
- Formal assurance
- Artifact available
Contribution & verification
- Mathematical novelty
- Tooling only
- Interface
- Informal, Symbolic
- AI contribution
- Frontier language models are evaluated across difficulty levels and mathematical subdomains.
- Human contribution
- Researchers curated and annotated competition problems and designed the taxonomy and scoring.
- Verification
- Paper released · E3 · Inspectable artifact · Artifact available
- Last reviewed
- 2026-09-11
Technical details
- Systems
- omni-math
- Organizations
- shenzhen-research-institute-of-big-data
- Problems
- olympiad-mathematics
- Methods
- benchmarking, difficulty-stratification
Sources & artifacts
- Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language ModelsS1 · Primary research · Paper · accessed 2026-09-10
- Official Omni-MATH benchmark repositoryS1 · Primary research · Repository · accessed 2026-09-11
- Omni-MATH dataset and evaluation codeArtifact · Dataset
Verification history
- 2024-10-10 · Paper released · E2 · Research record
The Chronicle records this event from the cited primary or official source; the status does not imply independent Chronicle verification.
- 2026-09-11 · Paper released · E3 · Inspectable artifact
This corpus refresh added a directly inspectable public artifact maintained by the authors or releasing organization. E3 records artifact-level inspectability and does not imply independent verification of the underlying scientific or mathematical claim.
Corrections
No corrections recorded.