GPQA Diamond
Graduate-level science questions
knowledge and reasoningcrowded
Score in %
Definition
- Version
- not in the notes
- Category
- knowledge and reasoning
- Maintainer
- Rein et al. (NYU/Cohere/Anthropic authors), USA
- Status
- crowded
- Home page
- https://arxiv.org/abs/2311.12022
Sources
- GPT-6 Astra Is Live: OpenAI's New Flagship Open for Calls - API易文档中心 docs.apiyi.com https://docs.apiyi.com/en/news/gpt-6-astra-launch
- OpenAI Launches GPT-6 Astra, Pricing, Benchmarks and Who Gets It First pasqualepillitteri.it https://pasqualepillitteri.it/en/news/14246/openai-launches-gpt-6-astra-pricing-benchmarks
- huggingface.co/moonshotai/Kimi-K3 https://huggingface.co/moonshotai/Kimi-K3
- GPQA: A Graduate-Level Google-Proof Q&A Benchmark arxiv.org https://arxiv.org/abs/2311.12022
- GPQA Leaderboard (October 2026): GPT-6 Astra Leads at 96% benchlm.ai https://www.benchlm.ai/benchmarks/gpqa
- LLM inference prices have fallen rapidly but unequally across tasks epoch.ai https://epoch.ai/data-insights/llm-inference-price-trends
Scores at a Glance
Score in %
Higher is better for this unit.
1
GPT-6 AstraOpenAIGPT-5.6 SolOpenAI
Kimi K3Moonshot AI
Qwen3.8-2.4T-A95BAlibaba (Qwen)
Hy4-previewTencentQwen3.8-Flash-NextAlibaba (Qwen)
DeepSeek-V4.1-FlashDeepSeekHy3TencentDeepSeek-V4-ProDeepSeekDeepSeek-V4-FlashDeepSeekQwen3.8-27BAlibaba (Qwen)
Solar Open 2Upstage
Gemma 4 31BGoogle, Google DeepMind
K-EXAONE 2.0LG AI ResearchQwen3.5-9BAlibaba (Qwen)Not independently verified
Sarvam 105BSarvam AI
Mistral Small 4Mistral AIMistral Large 3 (2512)Mistral AI
Phi-4-mini-flash-reasoningMicrosoft AI
SmolLM3-3BHugging FaceCard 5-model comparison table; mode label not extracted
Tiny Aya globalCohere Labs
Gemma 3n E2BGoogle DeepMindITGemma 3n E4BGoogle DeepMindIT
GPT-6 AstraOpenAI96 %
vendor-reported2GPT-5.6 SolOpenAI94.6 %
vendor-reported3Kimi K3Moonshot AI93.5 %
vendor-reported4Qwen3.8-2.4T-A95BAlibaba (Qwen)92.6 %
vendor-reported5Hy4-previewTencent92.3 %
vendor-reported6Qwen3.8-Flash-NextAlibaba (Qwen)91.7 %
vendor-reported7DeepSeek-V4.1-FlashDeepSeek90.9 %
vendor-reported8Hy3Tencent90.4 %
vendor-reported9DeepSeek-V4-ProDeepSeek90.1 %
vendor-reported10DeepSeek-V4-FlashDeepSeek89.9 %
vendor-reported11Qwen3.8-27BAlibaba (Qwen)89.2 %
vendor-reported12Solar Open 2Upstage86.3 %
vendor-reported13Gemma 4 31BGoogle, Google DeepMind84.3 %
vendor-reported14K-EXAONE 2.0LG AI Research82.2 %
vendor-reported15Qwen3.5-9BAlibaba (Qwen)Not independently verified81.7 %
vendor-reported16Sarvam 105BSarvam AI78.7 %
vendor-reported17Mistral Small 4Mistral AI71.2 %
vendor-reported18Mistral Large 3 (2512)Mistral AI67.17 %
vendor-reported19Phi-4-mini-flash-reasoningMicrosoft AI45.08 %
vendor-reported20SmolLM3-3BHugging FaceCard 5-model comparison table; mode label not extracted35.7 %
vendor-reported21Tiny Aya globalCohere Labs28.28 %
vendor-reported22Gemma 3n E2BGoogle DeepMindIT24.8 %
vendor-reported23Gemma 3n E4BGoogle DeepMindIT23.7 %
vendor-reported0255075100
Score in %, Claude 3 model card, Table 1 and vision table
Higher is better for this unit.
1
Claude 3 OpusAnthropic0-shot CoTClaude 3 SonnetAnthropic0-shot CoT
Claude 3 OpusAnthropic0-shot CoT50.4 %
vendor-reported2Claude 3 SonnetAnthropic0-shot CoT40.4 %
vendor-reported0255075100
Where this comes from
Each bar is one model, with its most independent recorded score. Hatched bars are vendor-reported; the word beside each bar says who ran it. Settings and dates differ between rows; the Scores table below has every result.
Scores
| Model | Score | Who ran it | Setting | View or population | Date | Source |
|---|---|---|---|---|---|---|
| GPT-6 Astra | 96 % | vendor-reported | – | – | – | docs.apiyi.com |
| GPT-6 Astra | 96 % | vendor-reported | – | rank 1, BenchLM aggregator, top score | 10 Oct 2026 | benchlm.ai |
| GPT-5.6 Sol | 94.6 % | vendor-reported | – | – | – | docs.apiyi.com |
| Gemini 3 Deep Think | 93.8 % | vendor-reported | – | Gemini 3 launch post | 18 Nov 2025 | blog.google |
| Kimi K3 | 93.5 % | vendor-reported | – | – | – | huggingface.co |
| Qwen3.8-2.4T-A95B | 92.6 % | vendor-reported | – | – | – | huggingface.co |
| Hy4-preview | 92.3 % | vendor-reported | – | – | – | huggingface.co |
| Qwen3.8-Flash-Next | 91.7 % | vendor-reported | – | – | – | huggingface.co |
| DeepSeek-V4.1-Flash | 90.9 % | vendor-reported | – | – | – | huggingface.co |
| Hy3 | 90.4 % | vendor-reported | – | – | – | huggingface.co |
| DeepSeek-V4-Pro | 90.1 % | vendor-reported | – | – | – | huggingface.co |
| DeepSeek-V4-Flash | 89.9 % | vendor-reported | – | – | – | huggingface.co |
| Qwen3.8-27B | 89.2 % | vendor-reported | – | – | – | huggingface.co |
| Solar Open 2 | 86.3 % | vendor-reported | – | – | – | huggingface.co |
 A.X K2 | 85.6 % | vendor-reported | Thinking mode | A.X K2 model card | Jul 2026 | huggingface.co |
| Gemma 4 31B | 84.3 % | vendor-reported | – | – | – | huggingface.co |
 MiMo-V2-Flash | 83.7 % | vendor-reported | – | MiMo-V2-Flash model card, post-trained model table | 16 Dec 2025 | huggingface.co |
 Motif-3 | 83.4 % | vendor-reported | temperature 1.0, top-p 0.95, max sequence 262,144 | Motif-3 model card | Aug 2026 | huggingface.co |
| K-EXAONE 2.0 | 82.2 % | vendor-reported | – | – | – | huggingface.co |
| Qwen3.5-9B | 81.7 % | vendor-reported | Not independently verified | – | Mar 2026 | rits.shanghai.nyu.edu |
| DeepSeek-V3.1-Terminus | 80.7 % | vendor-reported | Model card comparison table; mode not stated | DeepSeek-V3.1-Terminus model card | 22 Sep 2025 | huggingface.co |
| DeepSeek-V3.1 | 80.1 % | vendor-reported | Thinking mode, Pass@1 | DeepSeek-V3.1 model card | 21 Aug 2025 | huggingface.co |
| Sarvam 105B | 78.7 % | vendor-reported | – | – | – | huggingface.co |
 Trinity-Large-Thinking | 76.3 % | vendor-reported | – | Trinity-Large-Thinking model card | 1 Apr 2026 | huggingface.co |
| EXAONE 4.0 (32B) | 75.4 % | vendor-reported | Reasoning mode | EXAONE 4.0 model card | Jul 2025 | huggingface.co |
| Claude Opus 4 | 74.9 % | vendor-reported | Without extended thinking | Anthropic announcement | 22 May 2025 | anthropic.com |
 LongCat-Flash (LongCat-Flash-Chat) | 73.23 % | vendor-reported | acc | LongCat-Flash-Chat model card | Sep 2025 | huggingface.co |
| Mistral Small 4 | 71.2 % | vendor-reported | – | – | – | huggingface.co |
| Mistral Large 3 (2512) | 67.17 % | vendor-reported | – | – | – | huggingface.co |
| Claude 3 Opus | 50.4 % | vendor-reported | 0-shot CoT | Claude 3 model card, Table 1 and vision table | Mar 2024 | www-cdn.anthropic.com |
| Phi-4-mini-flash-reasoning | 45.08 % | vendor-reported | – | – | – | huggingface.co |
| Claude 3 Sonnet | 40.4 % | vendor-reported | 0-shot CoT | Claude 3 model card, Table 1 and vision table | Mar 2024 | www-cdn.anthropic.com |
| SmolLM3-3B | 35.7 % | vendor-reported | Card 5-model comparison table; mode label not extracted | – | – | huggingface.co |
| Tiny Aya global | 28.28 % | vendor-reported | – | – | – | huggingface.co |
| Gemma 3n E2B | 24.8 % | vendor-reported | IT | – | – | ai.google.dev |
| Gemma 3n E4B | 23.7 % | vendor-reported | IT | – | – | ai.google.dev |
Where this comes from
Scores from different settings, harnesses or dates are not directly comparable; the row says which. Sorted by score, highest first, as a reading aid, not a ranking.
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "GPQA Diamond", https://aurion.semai.id/benchmarks/gpqa-diamond/, records of October 2026.