MedQA
Medical licensing-style multiple-choice QA
domain
Score in %
Definition
- Version
- not in the notes
- Category
- domain
- Maintainer
- not in the notes
- Status
- not in the notes
- Home page
- not in the notes
Sources
- MedGemma Technical Report arxiv.org https://arxiv.org/html/2507.05201v2
- google/medgemma-27b-text-it , Hugging Face huggingface.co https://huggingface.co/google/medgemma-27b-text-it
- huggingface.co/m42-health/Llama3-Med42-70B https://huggingface.co/m42-health/Llama3-Med42-70B
Scores at a Glance
Score in %
Higher is better for this unit.
1
MedGemma 27B textGoogle27B, best-of-5Med-PaLM 2Google
Llama3-Med42-70BM42 HealthMedGemma 4BGoogleLlama3-Med42-8BM42 HealthGemma 3 4BGoogleBaseline in MedGemma report
MedGemma 27B textGoogle27B, best-of-589.8 %
vendor-reported2Med-PaLM 2Google86.5 %
vendor-reported3Llama3-Med42-70BM42 Health79.1 %
vendor-reported4MedGemma 4BGoogle64.4 %
vendor-reported5Llama3-Med42-8BM42 Health62.84 %
vendor-reported6Gemma 3 4BGoogleBaseline in MedGemma report50.7 %
vendor-reported0255075100
Where this comes from
Each bar is one model, with its most independent recorded score. Hatched bars are vendor-reported; the word beside each bar says who ran it. Settings and dates differ between rows; the Scores table below has every result.
Scores
| Model | Score | Who ran it | Setting | View or population | Date | Source |
|---|---|---|---|---|---|---|
| MedGemma 27B text | 89.8 % | vendor-reported | 27B, best-of-5 | – | Jul 2025 | arxiv.org |
| MedGemma 27B text | 87.7 % | vendor-reported | 27B, 0-shot with test-time scaling | – | Jul 2025 | arxiv.org |
| Med-PaLM 2 | 86.5 % | vendor-reported | – | – | 2023 | blog.google |
| Llama3-Med42-70B | 79.1 % | vendor-reported | – | – | Aug 2024 | huggingface.co |
| MedGemma 4B | 64.4 % | vendor-reported | – | – | Jul 2025 | arxiv.org |
| Llama3-Med42-8B | 62.84 % | vendor-reported | – | – | Aug 2024 | huggingface.co |
| Gemma 3 4B | 50.7 % | vendor-reported | Baseline in MedGemma report | – | Jul 2025 | arxiv.org |
Where this comes from
Scores from different settings, harnesses or dates are not directly comparable; the row says which. Sorted by score, highest first, as a reading aid, not a ranking.
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "MedQA", https://aurion.semai.id/benchmarks/medqa/, records of October 2026.