Primary evidence
- Llemma: An Open Language Model For MathematicsS1 · Primary research · Paper · 2023-10-16
- Llemma models, Proof-Pile-2 data and training code repositoryS1 · Primary research · Repository
Llemma releases open language models and data specialized for mathematics
The project continued pretraining Code Llama on Proof-Pile-2, a mixture of mathematical web text, scientific papers and mathematical code.
What happened?
Llemma released 7B and 34B math-specialized models that outperformed known open base models on MATH and showed tool-use and formal-proof capabilities without task-specific fine-tuning.
Why it matters
It gave the community an open base model and corpus for studying how mathematical specialization transfers across informal, tool and formal settings.
Significance rationale: Llemma established a reproducible open math-language-model stack, including model weights, Proof-Pile-2 and training code.
Historical context
Earlier developments
Later developments
—
Related milestones
—
Evidence & status at a glance
- Significance
- H2 · Field milestone
- Verification
- Paper released
- Evidence level
- E3 · Inspectable artifact
- AI role
- Computation only
- Formal assurance
- Artifact available
Contribution & verification
- Mathematical novelty
- Benchmark result
- Interface
- Informal, Formal, Symbolic
- AI contribution
- The trained models perform mathematical reasoning, tool use and formal proof generation.
- Human contribution
- Researchers assembled Proof-Pile-2, continued pretraining the models and released the full artifacts.
- Verification
- Paper released · E3 · Inspectable artifact · Artifact available
- Last reviewed
- 2026-09-11
Technical details
- Systems
- llemma
- Organizations
- eleutherai, princeton-university
- Problems
- math-benchmark
- Methods
- continued-pretraining, math-corpus, language-model
Sources & artifacts
- Llemma: An Open Language Model For MathematicsS1 · Primary research · Paper · accessed 2026-09-10
- Llemma models, Proof-Pile-2 data and training code repositoryS1 · Primary research · Repository · accessed 2026-09-11
- Llemma code, model and Proof-Pile-2 data linksArtifact · Code
Verification history
- 2023-10-16 · Paper released · E2 · Research record
The Chronicle records this event from the cited primary or official source; the status does not imply independent Chronicle verification.
- 2026-09-11 · Paper released · E3 · Inspectable artifact
This corpus refresh added a directly inspectable public artifact maintained by the authors or releasing organization. E3 records artifact-level inspectability and does not imply independent verification of the underlying scientific or mathematical claim.
Corrections
No corrections recorded.