CMMLU
67 subjects, including China-specific ones; contamination check for new entries since 2023-12-16
Chinese languageactive
Score in %, CMMLU leaderboard
Definition
- Version
- not in the notes
- Category
- Chinese language
- Maintainer
- Haonan Li, Timothy Baldwin et al. (MBZUAI authors), UAE/China
- Status
- active
- Home page
- https://github.com/haonan-li/CMMLU
Sources
- GitHub - haonan-li/CMMLU: CMMLU: Measuring massive multitask language understanding in Chinese github.com https://github.com/haonan-li/CMMLU
Scores at a Glance
Score in %, CMMLU leaderboard
Higher is better for this unit.
0255075100
Where this comes from
Each bar is one model, with its most independent recorded score. Hatched bars are vendor-reported; the word beside each bar says who ran it. Settings and dates differ between rows; the Scores table below has every result.
Scores
| Model | Score | Who ran it | Setting | View or population | Date | Source |
|---|---|---|---|---|---|---|
| Spark 4.0-2024-10-14 | 90.97 % | maintainer-run | Zero-shot | rank 1, CMMLU leaderboard | – | github.com |
| Lingzhi-72B-chat | 90.26 % | maintainer-run | Five-shot | rank 1, CMMLU leaderboard | – | github.com |
 Qwen1.5-72B | 83.5 % | vendor-reported | – | – | 4 Feb 2024 | qwenlm.github.io |
Where this comes from
Scores from different settings, harnesses or dates are not directly comparable; the row says which. Sorted by score, highest first, as a reading aid, not a ranking.
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "CMMLU", https://aurion.semai.id/benchmarks/cmmlu/, records of October 2026.