GPQA
Graduate-level Google-proof science questions (full set; subset not stated where cited)
knowledge and reasoning
Score in %
Definition
- Version
- not in the notes
- Category
- knowledge and reasoning
- Maintainer
- Rein et al. (NYU/Cohere/Anthropic authors), USA
- Status
- not in the notes
- Home page
- https://arxiv.org/abs/2311.12022
Sources
- huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
- Qwen/Qwen3.5-397B-A17B , Hugging Face huggingface.co https://huggingface.co/Qwen/Qwen3.5-397B-A17B
- GPQA: A Graduate-Level Google-Proof Q&A Benchmark arxiv.org https://arxiv.org/abs/2311.12022
- GPQA Leaderboard (October 2026): GPT-6 Astra Leads at 96% benchlm.ai https://www.benchlm.ai/benchmarks/gpqa
Scores at a Glance
Score in %
Higher is better for this unit.
1
Qwen3.5-397B-A17BAlibaba (Qwen)
Nemotron 3 Ultra 550B A55BNVIDIA
Gemma 4 31BGoogle, Google DeepMindReported as 'GPQA' by secondary source; subset not statedQwQ-32B-PreviewAlibaba (Qwen)
GLM-4 (0520)Z.ai (Zhipu)
Jamba 1.5 MiniAI21 Labs
Phi-4-mini-instructMicrosoft AI0-shot CoT
Qwen3.5-397B-A17BAlibaba (Qwen)88.4 %
vendor-reported2Nemotron 3 Ultra 550B A55BNVIDIA87 %
vendor-reported3Gemma 4 31BGoogle, Google DeepMindReported as 'GPQA' by secondary source; subset not stated84.3 %
vendor-reported4QwQ-32B-PreviewAlibaba (Qwen)65.2 %
vendor-reported5GLM-4 (0520)Z.ai (Zhipu)39.9 %
vendor-reported6Jamba 1.5 MiniAI21 Labs32.3 %
vendor-reported7Phi-4-mini-instructMicrosoft AI0-shot CoT25.2 %
vendor-reported0255075100
Where this comes from
Each bar is one model, with its most independent recorded score. Hatched bars are vendor-reported; the word beside each bar says who ran it. Settings and dates differ between rows; the Scores table below has every result.
Scores
| Model | Score | Who ran it | Setting | View or population | Date | Source |
|---|---|---|---|---|---|---|
| Qwen3.5-397B-A17B | 88.4 % | vendor-reported | – | – | – | huggingface.co |
| Nemotron 3 Ultra 550B A55B | 87 % | vendor-reported | – | – | – | huggingface.co |
| Gemma 4 31B | 84.3 % | vendor-reported | Reported as 'GPQA' by secondary source; subset not stated | – | Apr 2026 | datanorth.ai |
| Llama 3.1 Nemotron Ultra 253B v1 | 76.01 % | vendor-reported | Reasoning on, pass@1 | Nemotron Ultra model card | Apr 2025 | huggingface.co |
| Llama 3.3 Nemotron Super 49B v1 | 66.67 % | vendor-reported | Reasoning on | Nemotron Super model card | Mar 2025 | huggingface.co |
| QwQ-32B-Preview | 65.2 % | vendor-reported | – | – | 28 Nov 2024 | qwenlm.github.io |
| GLM-4 (0520) | 39.9 % | vendor-reported | – | – | 18 Jun 2024 | arxiv.org |
| Jamba 1.5 Mini | 32.3 % | vendor-reported | – | – | Aug 2024 | huggingface.co |
 Granite-4.0-H-Micro | 32.15 % | vendor-reported | 0-shot, CoT | Granite-4.0-H-Micro model card | 2 Oct 2025 | huggingface.co |
| Phi-4-mini-instruct | 25.2 % | vendor-reported | 0-shot CoT | – | – | huggingface.co |
Where this comes from
Scores from different settings, harnesses or dates are not directly comparable; the row says which. Sorted by score, highest first, as a reading aid, not a ranking.
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "GPQA", https://aurion.semai.id/benchmarks/gpqa/, records of October 2026.