SWE-bench Pro
Software engineering tasks; vendor figures are harness-specific and OpenAI claims about 30% of tasks are broken
codingactive
Score in %
Definition
- Version
- not in the notes
- Category
- coding
- Maintainer
- Scale AI
- Status
- active
- Home page
- https://labs.scale.com/leaderboard/swe_bench_pro_public
Sources
- SWE-Bench Pro labs.scale.com https://labs.scale.com/leaderboard/swe_bench_pro_public
- GPT-5.6 Release: Pricing, ChatGPT Work, and the Codex Merger heyuan110.com https://www.heyuan110.com/posts/ai/2026-07-10-gpt-5-6-general-availability/
- Qwen/Qwen3.8-2.4T-A95B , Hugging Face huggingface.co https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B
- LICENSE , Qwen/Qwen3.8-2.4T-A95B at main huggingface.co https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B/blob/main/LICENSE
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? arxiv.org https://arxiv.org/abs/2509.16941
- SWE-bench Pro Leaderboard: Claude Opus 5.5 Leads at 89.9% benchlm.ai https://www.benchlm.ai/benchmarks/swePro
Scores at a Glance
Score in %
Higher is better for this unit.
1
Claude Opus 5.5Anthropicsystem cardClaude Fable 5Anthropicpublic board (per heyuan110)
Qwen3.8-2.4T-A95BAlibaba (Qwen)
Hy4-previewTencent
GPT-5.6 SolOpenAIpublic board (per heyuan110)Qwen3.8-Flash-NextAlibaba (Qwen)
GLM-5.2Z.ai (Zhipu)max effortQwen3.8-27BAlibaba (Qwen)
MiniMax-M3MiniMaxscaffold not statedGPT-5.5OpenAIGPT-5.3-CodexOpenAIxhigh
Step-3.7-FlashStepFun
Gemini 3.1 Pro (preview)Google DeepMind, GoogleScale SEAL public, mini-swe-agentQwen3-Coder-NextAlibaba (Qwen)scaffold not statedQwen3-Coder-480B-A35B-InstructAlibaba (Qwen)
Mellum2.1-12B-A2.5B-ThinkingJetBrainsthinking mode
Claude Opus 5.5Anthropicsystem card89.9 %
vendor-reported2Claude Fable 5Anthropicpublic board (per heyuan110)80 %
vendor-reported3Qwen3.8-2.4T-A95BAlibaba (Qwen)67.7 %
vendor-reported4Hy4-previewTencent65.7 %
vendor-reported5GPT-5.6 SolOpenAIpublic board (per heyuan110)64.6 %
vendor-reported6Qwen3.8-Flash-NextAlibaba (Qwen)62.5 %
vendor-reported7GLM-5.2Z.ai (Zhipu)max effort62.1 %
vendor-reported8Qwen3.8-27BAlibaba (Qwen)61.7 %
vendor-reported9MiniMax-M3MiniMaxscaffold not stated59 %
vendor-reported10GPT-5.5OpenAI58.6 %
vendor-reported11GPT-5.3-CodexOpenAIxhigh56.8 %
vendor-reported12Step-3.7-FlashStepFun56.3 %
vendor-reported13Gemini 3.1 Pro (preview)Google DeepMind, GoogleScale SEAL public, mini-swe-agent46.1 %
maintainer-run14Qwen3-Coder-NextAlibaba (Qwen)scaffold not stated44.3 %
vendor-reported15Qwen3-Coder-480B-A35B-InstructAlibaba (Qwen)38.7 %
vendor-reported16Mellum2.1-12B-A2.5B-ThinkingJetBrainsthinking mode28 %
vendor-reported0255075100
Where this comes from
Each bar is one model, with its most independent recorded score. Hatched bars are vendor-reported; the word beside each bar says who ran it. Settings and dates differ between rows; the Scores table below has every result.
Scores
| Model | Score | Who ran it | Setting | View or population | Date | Source |
|---|---|---|---|---|---|---|
| Claude Opus 5.5 | 89.9 % | vendor-reported | Self-report | rank 1, BenchLM aggregator, top score | 10 Oct 2026 | benchlm.ai |
| Claude Opus 5.5 | 89.9 % | vendor-reported | system card | – | – | llm-stats.com |
| Claude Fable 5 | 80 % | vendor-reported | public board (per heyuan110) | – | – | heyuan110.com |
| Claude Fable 5 | 77.8 % | vendor-reported | – | – | – | ecosistemastartup.com |
| Qwen3.8-2.4T-A95B | 67.7 % | vendor-reported | – | – | – | huggingface.co |
| Hy4-preview | 65.7 % | vendor-reported | – | – | – | huggingface.co |
| GPT-5.6 Sol | 64.6 % | vendor-reported | public board (per heyuan110) | – | – | heyuan110.com |
| Qwen3.8-Flash-Next | 62.5 % | vendor-reported | – | – | – | huggingface.co |
| GLM-5.2 | 62.1 % | vendor-reported | max effort | – | – | rits.shanghai.nyu.edu |
| Qwen3.8-27B | 61.7 % | vendor-reported | – | – | – | huggingface.co |
| Qwen3.8-27B | 61.7 % | vendor-reported | – | – | – | huggingface.co |
 Muse Spark 1.1 | 61.5 % ±3.10 | maintainer-run | Scale SEAL public, mini-swe-agent, +/-3.10 | rank 1, Scale SEAL SWE-Bench Pro public | – | labs.scale.com |
| MiniMax-M3 | 59 % | vendor-reported | scaffold not stated | – | – | huggingface.co |
| GPT-5.5 | 58.6 % | vendor-reported | – | – | – | rits.shanghai.nyu.edu |
| GPT-5.3-Codex | 56.8 % | vendor-reported | xhigh | – | 5 Feb 2026 | openai.com |
 Ling 3.0 Flash (Ling-3.0-flash) | 56.6 % | vendor-reported | – | Ling-3.0-flash model card eval results | Jul 2026 | huggingface.co |
| Step-3.7-Flash | 56.3 % | vendor-reported | – | – | – | huggingface.co |
| Gemini 3.1 Pro (preview) | 46.1 % ±3.60 | maintainer-run | Scale SEAL public, mini-swe-agent | – | – | labs.scale.com |
| Qwen3-Coder-Next | 44.3 % | vendor-reported | scaffold not stated | – | – | huggingface.co |
| Qwen3-Coder-480B-A35B-Instruct | 38.7 % | vendor-reported | – | – | – | huggingface.co |
| Mellum2.1-12B-A2.5B-Thinking | 28 % | vendor-reported | thinking mode | – | – | huggingface.co |
Where this comes from
Scores from different settings, harnesses or dates are not directly comparable; the row says which. Sorted by score, highest first, as a reading aid, not a ranking.
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "SWE-Bench Pro", https://aurion.semai.id/benchmarks/swe-bench-pro/, records of October 2026.