AIME24
AIME 2024 problems
2024mathsaturated
Score in %
Definition
- Version
- 2024
- Category
- math
- Maintainer
- not in the notes
- Status
- saturated
- Home page
- not in the notes
Sources
- huggingface.co/microsoft/Phi-4-mini-flash-reasoning https://huggingface.co/microsoft/Phi-4-mini-flash-reasoning
- s1: Simple test-time scaling arxiv.org https://arxiv.org/html/2501.19393v1
- LIMO: Less is More for Reasoning arxiv.org https://arxiv.org/abs/2502.03387
- benchlm.ai/blog/posts/aime-hmmt-competition-math https://benchlm.ai/blog/posts/aime-hmmt-competition-math
Scores at a Glance
Score in %
Higher is better for this unit.
1
LIMOShanghai Jiao Tong University
s1Stanford University, University of Washington, Allen AI, Contextual AIafter SFT on 1,000 examples (base 26.7)
Phi-4-mini-flash-reasoningMicrosoft AI
Qwen2.5-32B-InstructAlibaba (Qwen)baseline reported in the s1 paper
LIMOShanghai Jiao Tong University63.3 %
vendor-reported2s1Stanford University, University of Washington, Allen AI, Contextual AIafter SFT on 1,000 examples (base 26.7)56.7 %
vendor-reported3Phi-4-mini-flash-reasoningMicrosoft AI52.29 %
vendor-reported4Qwen2.5-32B-InstructAlibaba (Qwen)baseline reported in the s1 paper26.7 %
vendor-reported0255075100
Where this comes from
Each bar is one model, with its most independent recorded score. Hatched bars are vendor-reported; the word beside each bar says who ran it. Settings and dates differ between rows; the Scores table below has every result.
Scores
| Model | Score | Who ran it | Setting | View or population | Date | Source |
|---|---|---|---|---|---|---|
 Olmo 3.1 32B Instruct | 67.8 % | vendor-reported | – | Olmo 3.1 32B Instruct model card | Dec 2025 | huggingface.co |
| LIMO | 63.3 % | vendor-reported | – | – | Feb 2025 | arxiv.org |
| s1 | 56.7 % | vendor-reported | after SFT on 1,000 examples (base 26.7) | – | Jan 2025 | arxiv.org |
| Phi-4-mini-flash-reasoning | 52.29 % | vendor-reported | – | – | – | huggingface.co |
| Qwen2.5-32B-Instruct | 26.7 % | vendor-reported | baseline reported in the s1 paper | – | Jan 2025 | arxiv.org |
Where this comes from
Scores from different settings, harnesses or dates are not directly comparable; the row says which. Sorted by score, highest first, as a reading aid, not a ranking.
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "AIME24", https://aurion.semai.id/benchmarks/aime-2024/, records of October 2026.