Project AURIONAI Use-Cases Research & Innovation

GSM8K

Grade-school maths word problems

math

Score in %

  1. 1Z.ai (Zhipu)GLM-4 (0520)93.3 %vendor-reported
  2. 2NVIDIANemotron-4 340B92.3 %vendor-reported
  3. 3Mistral AIMixtral 8x22B Instruct90.8 %vendor-reported
13scores
12models
0independent
0maintainer-run
13vendor-reported

Definition

Version
not in the notes
Category
math
Maintainer
not in the notes
Status
not in the notes
Home page
not in the notes

Sources

  1. huggingface.co/microsoft/Phi-4-mini-instruct https://huggingface.co/microsoft/Phi-4-mini-instruct

Scores at a Glance

Score in %

Higher is better for this unit.

1Z.ai (Zhipu)GLM-4 (0520)Z.ai (Zhipu)
93.3 %
vendor-reported2NVIDIANemotron-4 340BNVIDIAInstruct model; 0-shot
92.3 %
vendor-reported3Mistral AIMixtral 8x22B InstructMistral AImaj@8
90.8 %
vendor-reported4Microsoft AIPhi-4-mini-instructMicrosoft AI
88.6 %
vendor-reported5Alibaba (Qwen)Qwen1.5-72BAlibaba (Qwen)
79.5 %
vendor-reported6AI21 LabsJamba 1.5 MiniAI21 Labs
75.8 %
vendor-reported7DatabricksDBRXDatabricksDBRX Instruct; 5-shot, newer harness commit (Table 2)
72.8 %
vendor-reported8CohereCommand R+ (Apr '24)Cohere, Cohere LabsOpen LLM Leaderboard figures as quoted on the model card
70.7 %
vendor-reported
0255075100

Score in %, Claude 3 model card, Table 1 and vision table

Higher is better for this unit.

1AnthropicClaude 3 OpusAnthropic0-shot CoT
95 %
vendor-reported2AnthropicClaude 3 SonnetAnthropic0-shot CoT
92.3 %
vendor-reported
0255075100
Where this comes from
Each bar is one model, with its most independent recorded score. Hatched bars are vendor-reported; the word beside each bar says who ran it. Settings and dates differ between rows; the Scores table below has every result.

Scores

13 of 13 scores shown gsm8k-scores.csv
ModelScoreWho ran itSettingView or populationDateSource
AnthropicClaude 3 Opus95 %vendor-reported0-shot CoTClaude 3 model card, Table 1 and vision tableMar 2024www-cdn.anthropic.com
Z.ai (Zhipu)GLM-4 (0520)93.3 %vendor-reported––18 Jun 2024arxiv.org
AnthropicClaude 3 Sonnet92.3 %vendor-reported0-shot CoTClaude 3 model card, Table 1 and vision tableMar 2024www-cdn.anthropic.com
NVIDIANemotron-4 340B92.3 %vendor-reportedInstruct model; 0-shot–Jun 2024huggingface.co
Mistral AIMixtral 8x22B Instruct90.8 %vendor-reportedmaj@8–17 Apr 2024mistral.ai
Microsoft AIPhi-4-mini-instruct88.6 %vendor-reported–––huggingface.co
Allen AIOLMo 2 32B (OLMo-2-0325-32B-Instruct)87.6 %vendor-reported–OLMo-2-0325-32B-Instruct model cardMar 2025huggingface.co
IBMGranite-4.0-H-Micro81.35 %vendor-reported8-shotGranite-4.0-H-Micro model card2 Oct 2025huggingface.co
Alibaba (Qwen)Qwen1.5-72B79.5 %vendor-reported––4 Feb 2024qwenlm.github.io
AI21 LabsJamba 1.5 Mini75.8 %vendor-reported––Aug 2024huggingface.co
DatabricksDBRX72.8 %vendor-reportedDBRX Instruct; 5-shot, newer harness commit (Table 2)–27 Mar 2024databricks.com
CohereCommand R+ (Apr '24)70.7 %vendor-reportedOpen LLM Leaderboard figures as quoted on the model card–Apr 2024huggingface.co
DatabricksDBRX66.9 %vendor-reportedDBRX Instruct; 5-shot maj@13 (Table 1)–27 Mar 2024databricks.com
Where this comes from
Scores from different settings, harnesses or dates are not directly comparable; the row says which. Sorted by score, highest first, as a reading aid, not a ranking.

To cite this page: Project AURION, AI Use-Cases Research & Innovation, "GSM8K", https://aurion.semai.id/benchmarks/gsm8k/, records of October 2026.