Natural-language premise selection turns mathematical reference retrieval into a benchmark task

The work treated mathematical discourse as a retrieval problem rather than only an answer-generation problem. Its dataset captures links among conjectures, definitions and supporting propositions in ProofWiki, while accompanying baselines and later graph-based experiments measured how effectively…

What happened?

Deborah Ferreira and André Freitas introduced the natural-language premise-selection task, in which a system receives a mathematical conjecture expressed through prose and formulae and must retrieve definitions or propositions likely to support a proof, together with a ProofWiki-derived dataset and reproducible extraction pipeline.

The work treated mathematical discourse as a retrieval problem rather than only an answer-generation problem. Its dataset captures links among conjectures, definitions and supporting propositions in ProofWiki, while accompanying baselines and later graph-based experiments measured how effectively models could recover proof-relevant references.

Why it matters

Reference selection is a core bottleneck in both informal and formal mathematical reasoning. By defining a benchmark directly on human-facing mathematical text, this work provided a bridge from earlier formal premise-selection research toward later NaturalProofs-style retrieval and reference-grounded proof generation.

Significance rationale: Natural Language Premise Selection made retrieval of supporting definitions and propositions from ordinary mathematical text an explicit machine-learning task and released a ProofWiki-derived dataset for evaluating it. This supplied an early benchmark for reasoning over mixed prose and formulae before later natural-language theorem-proving datasets and grounded proof-generation systems.

Historical context

Earlier developments

—

Related milestones

—

Primary evidence

Evidence & status at a glance

Significance
H2 · Field milestone
Verification
Paper released
Evidence level
E3 · Inspectable artifact
AI role
Computation only
Formal assurance
Artifact available

How should these labels be read? →

Contribution & verification

Mathematical novelty
Tooling only
Interface
Informal, Symbolic
AI contribution
Machine-learning and retrieval models were trained and evaluated to rank proof-relevant mathematical statements; the task definition, ProofWiki extraction and relevance labels were constructed by researchers from existing human-authored mathematical material.
Human contribution
Researchers defined the task, built the ProofWiki-derived corpus and extraction pipeline, designed evaluation splits and baselines, and analyzed the difficulty of retrieving mathematically relevant premises from mixed prose and formulae.
Verification
Paper released · E3 · Inspectable artifact · Artifact available
Last reviewed
2026-09-15

Technical details

Systems
nl-ps, ps-proofwiki
Organizations
university-of-manchester
Problems
natural-language-premise-selection
Methods
premise-selection, information-retrieval, mathematical-language-processing, dataset-construction

Sources & artifacts

Verification history

  • 2020-04-30 · Paper released · E2 · Research record

    The arXiv and LREC research records establish the natural-language premise-selection task and the released ProofWiki-derived dataset.

  • 2026-09-15 · Paper released · E3 · Inspectable artifact

    The public nl-ps repository exposes the dataset, extraction pipeline and baseline tooling. E3 records artifact-level inspectability rather than independent validation of every reported retrieval result.

Corrections

No corrections recorded.