AIME
In plain wordsA hard US high-school math contest whose 15 questions each have a whole-number answer; AI labs use each year's new papers to test their models' mathematical reasoning.
The invitation-only second round of the Mathematical Association of America's American Mathematics Competitions, reached by scoring near the top of the AMC 10 or AMC 12, in which students have three hours for 15 problems. Because each answer is a whole number from 0 to 999, responses can be graded automatically by exact matchexact match, n.: The simplest way to grade an AI's answer: it counts as right only if it is identical to the reference answer, after minor tidying such as ignoring capital letters.. Because two new papers (AIME I and II) are set each February, recent problems offer a fresh and partly contamination-resistantcontamination-resistant benchmark, n.: A test built so that its questions are unlikely to have been seen by an AI during training — kept secret, refreshed often, or written after the model was made. test of competition-level reasoning, usually reported as accuracy on a year's 30 problems averaged over many samples, or with majority voting. With so few problems, results are noisy: one problem is worth more than three percentage points. OpenAI's September 2024 announcement of its o1 reasoning modelreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning. made AIME a headline benchmark, reporting 74% on AIME 2024 for o1 against 12% for GPT-4o; leading models were scoring above 90% by 2025.
On AIME 2025 the model averages 88% over 64 samples, so we use the 2026 papers to rule out contamination.
Origin
The AIME was first held in 1983 as a qualifying step toward the USA Mathematical Olympiad. Its use as an AI benchmark took off in 2024, spread by OpenAI's o1 report and by open datasets of recent papers.
Usage note
'AIME 2025' means that year's 30 contest problems, not a separately designed benchmark. Strong AIME scores show contest problem-solving with short numerical answers, not proof-writing.
Related
- MATHMATH, prop. n.: A set of 12,500 problems from high-school math competitions, each with a full worked solution, used to test language models on genuinely challenging mathematics.
- FrontierMathFrontierMath, prop. n.: Several hundred new math problems, kept unpublished so AI systems cannot have trained on them, written by professional mathematicians to check whether machines can do advanced, even research-level, mathematics.
- IMO Grand ChallengeIMO Grand Challenge, prop. n.: The goal of building an AI that earns a gold medal at the International Mathematical Olympiad, the top contest for school-age mathematicians. AI systems first reached gold-medal scores in 2025, though not under the challenge's strict original rules.
- reasoning modelreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning.
- self-consistencyself-consistency, n.: Asking an AI the same question many times, letting it reason differently each time, then going with the answer it reaches most often — a vote among its own attempts.
- contamination-resistant benchmarkcontamination-resistant benchmark, n.: A test built so that its questions are unlikely to have been seen by an AI during training — kept secret, refreshed often, or written after the model was made.
Sources
- OpenAI (2024). Learning to Reason with LLMs. openai.com/index/learning-to-reason-with-llms
Mentioned in
FrontierMathFrontierMath, prop. n.: Several hundred new math problems, kept unpublished so AI systems cannot have trained on them, written by professional mathematicians to check whether machines can do advanced, even research-level, mathematics., MATHMATH, prop. n.: A set of 12,500 problems from high-school math competitions, each with a full worked solution, used to test language models on genuinely challenging mathematics., contamination-resistant benchmarkcontamination-resistant benchmark, n.: A test built so that its questions are unlikely to have been seen by an AI during training — kept secret, refreshed often, or written after the model was made., IMO Grand ChallengeIMO Grand Challenge, prop. n.: The goal of building an AI that earns a gold medal at the International Mathematical Olympiad, the top contest for school-age mathematicians. AI systems first reached gold-medal scores in 2025, though not under the challenge's strict original rules.