MMLU
Multiple-choice knowledge questions (reported 5-shot on the Phi-4-mini card)
knowledge and reasoningsaturated
Score in %
Definition
- Version
- not in the notes
- Category
- knowledge and reasoning
- Maintainer
- not in the notes
- Status
- saturated
- Home page
- not in the notes
Sources
- huggingface.co/microsoft/Phi-4-mini-instruct https://huggingface.co/microsoft/Phi-4-mini-instruct
- Humanity's Last Exam arxiv.org https://arxiv.org/abs/2501.14249
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone arxiv.org https://arxiv.org/abs/2404.14219
Scores at a Glance
Score in %
Higher is better for this unit.
1
GLM-4 (0520)Z.ai (Zhipu)
Nemotron-4 340BNVIDIAInstruct model; 0-shot
Qwen1.5-72BAlibaba (Qwen)5-shot
Command R+ (Apr '24)Cohere, Cohere LabsOpen LLM Leaderboard figures as quoted on the model card
DBRXDatabricksDBRX Instruct; 5-shot
Jamba 1.5 MiniAI21 LabsCoT
Phi-3-miniMicrosoft AIPhi-4-mini-instructMicrosoft AI5-shotQwen-14B-ChatAlibaba (Qwen)5-shot
GLM-4 (0520)Z.ai (Zhipu)83.3 %
vendor-reported2Nemotron-4 340BNVIDIAInstruct model; 0-shot78.7 %
vendor-reported3Qwen1.5-72BAlibaba (Qwen)5-shot77.5 %
vendor-reported4Command R+ (Apr '24)Cohere, Cohere LabsOpen LLM Leaderboard figures as quoted on the model card75.7 %
vendor-reported5DBRXDatabricksDBRX Instruct; 5-shot73.7 %
vendor-reported6Jamba 1.5 MiniAI21 LabsCoT69.7 %
vendor-reported7Phi-3-miniMicrosoft AI69 %
vendor-reported8Phi-4-mini-instructMicrosoft AI5-shot67.3 %
vendor-reported9Qwen-14B-ChatAlibaba (Qwen)5-shot66.5 %
vendor-reported0255075100
Where this comes from
Each bar is one model, with its most independent recorded score. Hatched bars are vendor-reported; the word beside each bar says who ran it. Settings and dates differ between rows; the Scores table below has every result.
Scores
| Model | Score | Who ran it | Setting | View or population | Date | Source |
|---|---|---|---|---|---|---|
 LongCat-Flash (LongCat-Flash-Chat) | 89.71 % | vendor-reported | acc | LongCat-Flash-Chat model card | Sep 2025 | huggingface.co |
| GLM-4 (0520) | 83.3 % | vendor-reported | – | – | 18 Jun 2024 | arxiv.org |
 Olmo 3.1 32B Instruct | 80.9 % | vendor-reported | – | Olmo 3.1 32B Instruct model card | Dec 2025 | huggingface.co |
| Nemotron-4 340B | 78.7 % | vendor-reported | Instruct model; 0-shot | – | Jun 2024 | huggingface.co |
| Qwen1.5-72B | 77.5 % | vendor-reported | 5-shot | – | 4 Feb 2024 | qwenlm.github.io |
| OLMo 2 32B (OLMo-2-0325-32B-Instruct) | 77.3 % | vendor-reported | – | OLMo-2-0325-32B-Instruct model card | Mar 2025 | huggingface.co |
| Command R+ (Apr '24) | 75.7 % | vendor-reported | Open LLM Leaderboard figures as quoted on the model card | – | Apr 2024 | huggingface.co |
| DBRX | 73.7 % | vendor-reported | DBRX Instruct; 5-shot | – | 27 Mar 2024 | databricks.com |
| Jamba 1.5 Mini | 69.7 % | vendor-reported | CoT | – | Aug 2024 | huggingface.co |
| Phi-3-mini | 69 % | vendor-reported | – | – | Apr 2024 | arxiv.org |
 Granite-4.0-H-Micro | 67.43 % | vendor-reported | 5-shot | Granite-4.0-H-Micro model card | 2 Oct 2025 | huggingface.co |
| Phi-4-mini-instruct | 67.3 % | vendor-reported | 5-shot | – | – | huggingface.co |
| Qwen-14B-Chat | 66.5 % | vendor-reported | 5-shot | – | Sep 2023 | huggingface.co |
Where this comes from
Scores from different settings, harnesses or dates are not directly comparable; the row says which. Sorted by score, highest first, as a reading aid, not a ranking.
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "MMLU", https://aurion.semai.id/benchmarks/mmlu/, records of October 2026.