SEA-HELM
SEA average and per-language (incl. Indonesian) scores for open-weight models
Southeast Asian and Indonesianactive
Score, SEA-HELM leaderboard, 58 models, SEA average
Definition
- Version
- not in the notes
- Category
- Southeast Asian and Indonesian
- Maintainer
- AI Singapore
- Status
- active
- Home page
- https://leaderboard.sea-lion.ai/detailed/SEA
Sources
- SEA-HELM Leaderboard leaderboard.sea-lion.ai/detailed/SEA https://leaderboard.sea-lion.ai/detailed/SEA
- SEA-HELM Leaderboard leaderboard.sea-lion.ai/detailed/ID https://leaderboard.sea-lion.ai/detailed/ID
- SEA-HELM Leaderboard leaderboard.sea-lion.ai https://leaderboard.sea-lion.ai/
- SEA-HELM: Southeast Asian Holistic Evaluation of Language Models arxiv.org https://arxiv.org/abs/2502.14301
- Indonesia Tech Leaders Team With NVIDIA and Partners to Launch Nation’s AI blogs.nvidia.com https://blogs.nvidia.com/?p=75648
- GoToCompany/Llama-Sahabat-AI-v2-70B-IT , Hugging Face huggingface.co https://huggingface.co/GoToCompany/Llama-Sahabat-AI-v2-70B-IT
- aisingapore/Qwen-SEA-LION-v4.5-27B-IT , Hugging Face huggingface.co https://huggingface.co/aisingapore/Qwen-SEA-LION-v4.5-27B-IT
Scores at a Glance
Score, SEA-HELM leaderboard, 58 models, SEA average
Higher is better for this unit.
1
Gemma 4 26B-A4BGoogleCheckpoint gemma-4-26B-A4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps
Qwen3.6-35B-A3BAlibaba (Qwen)Checkpoint Qwen3.6-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3.5-35B-A3BAlibaba (Qwen)Checkpoint Qwen3.5-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3.8-Flash-NextAlibaba (Qwen)Checkpoint Qwen3.8-Flash-Next as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsGemma 4 12BGoogleCheckpoint gemma-4-12B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3.5-9BAlibaba (Qwen)Checkpoint Qwen3.5-9B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsGemma 4 E4BGoogle, Google DeepMindCheckpoint gemma-4-E4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsgemma-3-27b-itGoogleCheckpoint gemma-3-27b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsGemma-SEA-LION-v4-27B-ITAI SingaporeCheckpoint Gemma-SEA-LION-v4-27B-IT as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3.5-4BAlibaba (Qwen)Checkpoint Qwen3.5-4B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps
Llama 4 ScoutMetaCheckpoint Llama-4-Scout-17B-16E-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps
gpt-oss-20bOpenAICheckpoint gpt-oss-20b as listed, quantization mxfp4; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsLlama-3.3-70B-InstructMetaCheckpoint Llama-3.3-70B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3-8BAlibaba (Qwen)Checkpoint Qwen3-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3-VL-8B-InstructAlibaba (Qwen)Checkpoint Qwen3-VL-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen-SEA-LION-v4-8B-VLAI SingaporeCheckpoint Qwen-SEA-LION-v4-8B-VL as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps
Apertus v1.5 8BSwiss AI (EPFL, ETH Zurich, CSCS)Checkpoint Apertus-v1.5-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps
GLM 4.7 FlashZ.ai (Zhipu)Checkpoint GLM-4.7-Flash as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps
Mistral Small 4Mistral AICheckpoint Mistral-Small-4-119B-2603 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsGemma-3-4B-itGoogleCheckpoint gemma-3-4b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps
Nemotron 3 Nano 30B-A3BNVIDIACheckpoint NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsLlama-3.1-8B-InstructMetaCheckpoint Llama-3.1-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps
Tiny Aya waterCohere LabsCheckpoint tiny-aya-water as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsTiny Aya globalCohere LabsCheckpoint tiny-aya-global as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps
Gemma 4 26B-A4BGoogleCheckpoint gemma-4-26B-A4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps74.81 score
maintainer-run2Qwen3.6-35B-A3BAlibaba (Qwen)Checkpoint Qwen3.6-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps73.86 score
maintainer-run3Qwen3.5-35B-A3BAlibaba (Qwen)Checkpoint Qwen3.5-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps73.55 score
maintainer-run4Qwen3.8-Flash-NextAlibaba (Qwen)Checkpoint Qwen3.8-Flash-Next as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps73.04 score
maintainer-run5Gemma 4 12BGoogleCheckpoint gemma-4-12B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps69.31 score
maintainer-run6Qwen3.5-9BAlibaba (Qwen)Checkpoint Qwen3.5-9B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps69.13 score
maintainer-run7Gemma 4 E4BGoogle, Google DeepMindCheckpoint gemma-4-E4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps66.3 score
maintainer-run8gemma-3-27b-itGoogleCheckpoint gemma-3-27b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps64.54 score
maintainer-run9Gemma-SEA-LION-v4-27B-ITAI SingaporeCheckpoint Gemma-SEA-LION-v4-27B-IT as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps64.35 score
maintainer-run10Qwen3.5-4BAlibaba (Qwen)Checkpoint Qwen3.5-4B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps63.82 score
maintainer-run11Llama 4 ScoutMetaCheckpoint Llama-4-Scout-17B-16E-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps63.43 score
maintainer-run12gpt-oss-20bOpenAICheckpoint gpt-oss-20b as listed, quantization mxfp4; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps60.64 score
maintainer-run13Llama-3.3-70B-InstructMetaCheckpoint Llama-3.3-70B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps60.44 score
maintainer-run14Qwen3-8BAlibaba (Qwen)Checkpoint Qwen3-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps60.33 score
maintainer-run15Qwen3-VL-8B-InstructAlibaba (Qwen)Checkpoint Qwen3-VL-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps56.6 score
maintainer-run16Qwen-SEA-LION-v4-8B-VLAI SingaporeCheckpoint Qwen-SEA-LION-v4-8B-VL as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps56.41 score
maintainer-run17Apertus v1.5 8BSwiss AI (EPFL, ETH Zurich, CSCS)Checkpoint Apertus-v1.5-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps54.16 score
maintainer-run18GLM 4.7 FlashZ.ai (Zhipu)Checkpoint GLM-4.7-Flash as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps53.76 score
maintainer-run19Mistral Small 4Mistral AICheckpoint Mistral-Small-4-119B-2603 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps52.07 score
maintainer-run20Gemma-3-4B-itGoogleCheckpoint gemma-3-4b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps47.84 score
maintainer-run21Nemotron 3 Nano 30B-A3BNVIDIACheckpoint NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps46.89 score
maintainer-run22Llama-3.1-8B-InstructMetaCheckpoint Llama-3.1-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps37.2 score
maintainer-run23Tiny Aya waterCohere LabsCheckpoint tiny-aya-water as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps19.75 score
maintainer-run24Tiny Aya globalCohere LabsCheckpoint tiny-aya-global as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps19.19 score
maintainer-run020406080
Score, SEA-HELM leaderboard, 58 models, Indonesian (ID)
Higher is better for this unit.
1Qwen3.8-Flash-NextAlibaba (Qwen)Checkpoint Qwen3.8-Flash-Next as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsGemma 4 26B-A4BGoogleCheckpoint gemma-4-26B-A4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3.5-35B-A3BAlibaba (Qwen)Checkpoint Qwen3.5-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3.6-35B-A3BAlibaba (Qwen)Checkpoint Qwen3.6-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsGemma 4 12BGoogleCheckpoint gemma-4-12B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3.5-9BAlibaba (Qwen)Checkpoint Qwen3.5-9B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsLlama-3.3-70B-InstructMetaCheckpoint Llama-3.3-70B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsGemma-SEA-LION-v4-27B-ITAI SingaporeCheckpoint Gemma-SEA-LION-v4-27B-IT as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsgemma-3-27b-itGoogleCheckpoint gemma-3-27b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3.5-4BAlibaba (Qwen)Checkpoint Qwen3.5-4B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsGemma 4 E4BGoogle, Google DeepMindCheckpoint gemma-4-E4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3-8BAlibaba (Qwen)Checkpoint Qwen3-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsgpt-oss-20bOpenAICheckpoint gpt-oss-20b as listed, quantization mxfp4; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsLlama 4 ScoutMetaCheckpoint Llama-4-Scout-17B-16E-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen3-VL-8B-InstructAlibaba (Qwen)Checkpoint Qwen3-VL-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsQwen-SEA-LION-v4-8B-VLAI SingaporeCheckpoint Qwen-SEA-LION-v4-8B-VL as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsNemotron 3 Nano 30B-A3BNVIDIACheckpoint NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsMistral Small 4Mistral AICheckpoint Mistral-Small-4-119B-2603 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsApertus v1.5 8BSwiss AI (EPFL, ETH Zurich, CSCS)Checkpoint Apertus-v1.5-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsGemma-3-4B-itGoogleCheckpoint gemma-3-4b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsLlama-3.1-8B-InstructMetaCheckpoint Llama-3.1-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsTiny Aya waterCohere LabsCheckpoint tiny-aya-water as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstrapsTiny Aya globalCohere LabsCheckpoint tiny-aya-global as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps
Qwen3.8-Flash-NextAlibaba (Qwen)Checkpoint Qwen3.8-Flash-Next as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps79.04 score
maintainer-run2Gemma 4 26B-A4BGoogleCheckpoint gemma-4-26B-A4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps78.26 score
maintainer-run3Qwen3.5-35B-A3BAlibaba (Qwen)Checkpoint Qwen3.5-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps77.55 score
maintainer-run4Qwen3.6-35B-A3BAlibaba (Qwen)Checkpoint Qwen3.6-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps77.29 score
maintainer-run5Gemma 4 12BGoogleCheckpoint gemma-4-12B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps74.51 score
maintainer-run6Qwen3.5-9BAlibaba (Qwen)Checkpoint Qwen3.5-9B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps74.23 score
maintainer-run7Llama-3.3-70B-InstructMetaCheckpoint Llama-3.3-70B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps71.37 score
maintainer-run8Gemma-SEA-LION-v4-27B-ITAI SingaporeCheckpoint Gemma-SEA-LION-v4-27B-IT as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps70.55 score
maintainer-run9gemma-3-27b-itGoogleCheckpoint gemma-3-27b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps70.41 score
maintainer-run10Qwen3.5-4BAlibaba (Qwen)Checkpoint Qwen3.5-4B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps69.66 score
maintainer-run11Gemma 4 E4BGoogle, Google DeepMindCheckpoint gemma-4-E4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps69.65 score
maintainer-run12Qwen3-8BAlibaba (Qwen)Checkpoint Qwen3-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps69.36 score
maintainer-run13gpt-oss-20bOpenAICheckpoint gpt-oss-20b as listed, quantization mxfp4; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps68.38 score
maintainer-run14Llama 4 ScoutMetaCheckpoint Llama-4-Scout-17B-16E-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps67.86 score
maintainer-run15Qwen3-VL-8B-InstructAlibaba (Qwen)Checkpoint Qwen3-VL-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps67.46 score
maintainer-run16Qwen-SEA-LION-v4-8B-VLAI SingaporeCheckpoint Qwen-SEA-LION-v4-8B-VL as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps67.21 score
maintainer-run17Nemotron 3 Nano 30B-A3BNVIDIACheckpoint NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps65.36 score
maintainer-run18Mistral Small 4Mistral AICheckpoint Mistral-Small-4-119B-2603 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps63.14 score
maintainer-run19Apertus v1.5 8BSwiss AI (EPFL, ETH Zurich, CSCS)Checkpoint Apertus-v1.5-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps62.23 score
maintainer-run20Gemma-3-4B-itGoogleCheckpoint gemma-3-4b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps57.71 score
maintainer-run21Llama-3.1-8B-InstructMetaCheckpoint Llama-3.1-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps54.48 score
maintainer-run22Tiny Aya waterCohere LabsCheckpoint tiny-aya-water as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps33.23 score
maintainer-run23Tiny Aya globalCohere LabsCheckpoint tiny-aya-global as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps32.12 score
maintainer-run020406080
Score, SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps
Higher is better for this unit.
1Gemma 4 31BGoogle, Google DeepMindQwen3.6-27BAlibaba (Qwen)Qwen3.5-122B-A10BAlibaba (Qwen)Qwen-SEA-LION-v4.5-27B-ITAI SingaporeMuse Glimmer 30BMetaNemotron-SEA-LION-v4.8-120B-A12BAI Singaporegpt-oss-120bOpenAINemotron 3 SuperNVIDIAMistral Medium 3.5 (v26.04)Mistral AIGLM 4.7 FlashZ.ai (Zhipu)Nemotron-SEA-LION-v4.8-30B-A3BAI SingaporeApertus v1.5 70BSwiss AI (EPFL, ETH Zurich, CSCS)Nemotron 3.5 Lightning 30B-A3BNVIDIAGemma 4 E2BGoogle, Google DeepMindGemma-SEA-LION-v4.5-E2B-ITAI Singapore
Gemma 4 31BGoogle, Google DeepMind79.75 score
maintainer-run2Qwen3.6-27BAlibaba (Qwen)77.96 score
maintainer-run3Qwen3.5-122B-A10BAlibaba (Qwen)77.93 score
maintainer-run4Qwen-SEA-LION-v4.5-27B-ITAI Singapore76.51 score
maintainer-run5Muse Glimmer 30BMeta75.61 score
maintainer-run6Nemotron-SEA-LION-v4.8-120B-A12BAI Singapore73.1 score
maintainer-run7gpt-oss-120bOpenAI72.28 score
maintainer-run8Nemotron 3 SuperNVIDIA70.78 score
maintainer-run9Mistral Medium 3.5 (v26.04)Mistral AI70.09 score
maintainer-run10GLM 4.7 FlashZ.ai (Zhipu)67.89 score
maintainer-run11Nemotron-SEA-LION-v4.8-30B-A3BAI Singapore65.87 score
maintainer-run12Apertus v1.5 70BSwiss AI (EPFL, ETH Zurich, CSCS)64.69 score
maintainer-run13Nemotron 3.5 Lightning 30B-A3BNVIDIA64.34 score
maintainer-run14Gemma 4 E2BGoogle, Google DeepMind64.13 score
maintainer-run15Gemma-SEA-LION-v4.5-E2B-ITAI Singapore63.96 score
maintainer-run020406080
Score, SEA-HELM SEA average, open-weight models
Higher is better for this unit.
1Gemma 4 31BGoogle, Google DeepMindQwen3.5-122B-A10BAlibaba (Qwen)Qwen3.6-27BAlibaba (Qwen)Qwen-SEA-LION-v4.5-27B-ITAI SingaporeMuse Glimmer 30BMetagpt-oss-120bOpenAIMistral Medium 3.5 (v26.04)Mistral AINemotron-SEA-LION-v4.8-120B-A12BAI SingaporeGemma-SEA-LION-v4.5-E2B-ITAI SingaporeGemma 4 E2BGoogle, Google DeepMindApertus v1.5 70BSwiss AI (EPFL, ETH Zurich, CSCS)Nemotron-SEA-LION-v4.8-30B-A3BAI SingaporeNemotron 3 SuperNVIDIANemotron 3.5 Lightning 30B-A3BNVIDIA
Gemma 4 31BGoogle, Google DeepMind76.17 score
maintainer-run2Qwen3.5-122B-A10BAlibaba (Qwen)74.66 score
maintainer-run3Qwen3.6-27BAlibaba (Qwen)74.64 score
maintainer-run4Qwen-SEA-LION-v4.5-27B-ITAI Singapore73.24 score
maintainer-run5Muse Glimmer 30BMeta72.13 score
maintainer-run6gpt-oss-120bOpenAI66.14 score
maintainer-run7Mistral Medium 3.5 (v26.04)Mistral AI64.71 score
maintainer-run8Nemotron-SEA-LION-v4.8-120B-A12BAI Singapore63.44 score
maintainer-run9Gemma-SEA-LION-v4.5-E2B-ITAI Singapore59.23 score
maintainer-run10Gemma 4 E2BGoogle, Google DeepMind58.82 score
maintainer-run11Apertus v1.5 70BSwiss AI (EPFL, ETH Zurich, CSCS)55.2 score
maintainer-run12Nemotron-SEA-LION-v4.8-30B-A3BAI Singapore51.57 score
maintainer-run13Nemotron 3 SuperNVIDIA49.3 score
maintainer-run14Nemotron 3.5 Lightning 30B-A3BNVIDIA46.06 score
maintainer-run020406080
Score, SEA-HELM Indonesian
Higher is better for this unit.
1Gemma 4 31BGoogle, Google DeepMindIndonesianQwen3.6-27BAlibaba (Qwen)IndonesianQwen3.5-122B-A10BAlibaba (Qwen)IndonesianSEA-LION v4.5 (Qwen) 27BAI SingaporeIndonesian
Gemma 4 31BGoogle, Google DeepMindIndonesian79.75 index
maintainer-run2Qwen3.6-27BAlibaba (Qwen)Indonesian77.96 index
maintainer-run3Qwen3.5-122B-A10BAlibaba (Qwen)Indonesian77.93 index
maintainer-run4SEA-LION v4.5 (Qwen) 27BAI SingaporeIndonesian76.51 index
maintainer-run020406080
Where this comes from
Each bar is one model, with its most independent recorded score. Hatched bars are vendor-reported; the word beside each bar says who ran it. Settings and dates differ between rows; the Scores table below has every result.
Scores
| Model | Score | Who ran it | Setting | View or population | Date | Source |
|---|---|---|---|---|---|---|
| Gemma 4 31B | 79.75 index 95% CI [-1.25, +1.20] | maintainer-run | Indonesian | rank 1, SEA-HELM Indonesian | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma 4 31B | 79.75 score 95% CI [-1.25, +1.20] | maintainer-run | – | rank 1, SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.8-Flash-Next | 79.04 score 95% CI [-1.23, +1.16] | maintainer-run | Checkpoint Qwen3.8-Flash-Next as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma 4 26B-A4B | 78.26 score 95% CI [-1.29, +1.32] | maintainer-run | Checkpoint gemma-4-26B-A4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.6-27B | 77.96 index 95% CI [-1.22, +1.20] | maintainer-run | Indonesian | rank 2, SEA-HELM Indonesian | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.6-27B | 77.96 score 95% CI [-1.22, +1.20] | maintainer-run | – | rank 2, SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.5-122B-A10B | 77.93 index 95% CI [-1.29, +1.25] | maintainer-run | Indonesian | rank 3, SEA-HELM Indonesian | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.5-122B-A10B | 77.93 score 95% CI [-1.29, +1.25] | maintainer-run | – | rank 3, SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.5-35B-A3B | 77.55 score 95% CI [-1.18, +1.21] | maintainer-run | Checkpoint Qwen3.5-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.6-35B-A3B | 77.29 score 95% CI [-1.22, +1.21] | maintainer-run | Checkpoint Qwen3.6-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen-SEA-LION-v4.5-27B-IT | 76.51 score 95% CI [-1.23, +1.23] | maintainer-run | – | rank 4, SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| SEA-LION v4.5 (Qwen) 27B | 76.51 index 95% CI [-1.23, +1.23] | maintainer-run | Indonesian | rank 4, SEA-HELM Indonesian | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma 4 31B | 76.17 score 95% CI [-0.60, +0.59] | maintainer-run | – | rank 1, SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Muse Glimmer 30B | 75.61 score 95% CI [-1.13, +1.10] | maintainer-run | – | rank 5, SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma 4 26B-A4B | 74.81 score 95% CI [-0.59, +0.62] | maintainer-run | Checkpoint gemma-4-26B-A4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.5-122B-A10B | 74.66 score 95% CI [-0.64, +0.61] | maintainer-run | – | rank 2, SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.6-27B | 74.64 score 95% CI [-0.65, +0.60] | maintainer-run | – | SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma 4 12B | 74.51 score 95% CI [-1.27, +1.29] | maintainer-run | Checkpoint gemma-4-12B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.5-9B | 74.23 score 95% CI [-1.29, +1.28] | maintainer-run | Checkpoint Qwen3.5-9B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.6-35B-A3B | 73.86 score 95% CI [-0.60, +0.57] | maintainer-run | Checkpoint Qwen3.6-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.5-35B-A3B | 73.55 score 95% CI [-0.62, +0.60] | maintainer-run | Checkpoint Qwen3.5-35B-A3B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen-SEA-LION-v4.5-27B-IT | 73.24 score 95% CI [-0.56, +0.57] | maintainer-run | – | rank 4, SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Nemotron-SEA-LION-v4.8-120B-A12B | 73.1 score 95% CI [-1.16, +1.21] | maintainer-run | – | rank 6, SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.8-Flash-Next | 73.04 score 95% CI [-0.53, +0.53] | maintainer-run | Checkpoint Qwen3.8-Flash-Next as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| gpt-oss-120b | 72.28 score 95% CI [-1.35, +1.33] | maintainer-run | – | rank 7, SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Muse Glimmer 30B | 72.13 score 95% CI [-0.59, +0.56] | maintainer-run | – | SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Llama-3.3-70B-Instruct | 71.37 score 95% CI [-1.50, +1.54] | maintainer-run | Checkpoint Llama-3.3-70B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Nemotron 3 Super | 70.78 score 95% CI [-1.22, +1.20] | maintainer-run | – | rank 8, SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma-SEA-LION-v4-27B-IT | 70.55 score 95% CI [-1.42, +1.46] | maintainer-run | Checkpoint Gemma-SEA-LION-v4-27B-IT as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| gemma-3-27b-it | 70.41 score 95% CI [-1.47, +1.49] | maintainer-run | Checkpoint gemma-3-27b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Mistral Medium 3.5 (v26.04) | 70.09 score 95% CI [-1.18, +1.13] | maintainer-run | – | rank 9, SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.5-4B | 69.66 score 95% CI [-1.30, +1.32] | maintainer-run | Checkpoint Qwen3.5-4B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma 4 E4B | 69.65 score 95% CI [-1.40, +1.30] | maintainer-run | Checkpoint gemma-4-E4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3-8B | 69.36 score 95% CI [-1.32, +1.36] | maintainer-run | Checkpoint Qwen3-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma 4 12B | 69.31 score 95% CI [-0.64, +0.67] | maintainer-run | Checkpoint gemma-4-12B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.5-9B | 69.13 score 95% CI [-0.64, +0.63] | maintainer-run | Checkpoint Qwen3.5-9B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| gpt-oss-20b | 68.38 score 95% CI [-1.34, +1.35] | maintainer-run | Checkpoint gpt-oss-20b as listed, quantization mxfp4; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| GLM 4.7 Flash | 67.89 score 95% CI [-1.32, +1.27] | maintainer-run | – | rank 10, SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Llama 4 Scout | 67.86 score 95% CI [-1.58, +1.56] | maintainer-run | Checkpoint Llama-4-Scout-17B-16E-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3-VL-8B-Instruct | 67.46 score 95% CI [-1.55, +1.49] | maintainer-run | Checkpoint Qwen3-VL-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen-SEA-LION-v4-8B-VL | 67.21 score 95% CI [-1.64, +1.54] | maintainer-run | Checkpoint Qwen-SEA-LION-v4-8B-VL as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma 4 E4B | 66.3 score 95% CI [-0.65, +0.65] | maintainer-run | Checkpoint gemma-4-E4B-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| gpt-oss-120b | 66.14 score 95% CI [-0.64, +0.66] | maintainer-run | – | SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Nemotron-SEA-LION-v4.8-30B-A3B | 65.87 score 95% CI [-1.29, +1.33] | maintainer-run | – | SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Nemotron 3 Nano 30B-A3B | 65.36 score 95% CI [-1.30, +1.30] | maintainer-run | Checkpoint NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Mistral Medium 3.5 (v26.04) | 64.71 score 95% CI [-0.59, +0.59] | maintainer-run | – | SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Apertus v1.5 70B | 64.69 score 95% CI [-1.37, +1.36] | maintainer-run | – | SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| gemma-3-27b-it | 64.54 score 95% CI [-0.74, +0.72] | maintainer-run | Checkpoint gemma-3-27b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma-SEA-LION-v4-27B-IT | 64.35 score 95% CI [-0.72, +0.71] | maintainer-run | Checkpoint Gemma-SEA-LION-v4-27B-IT as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Nemotron 3.5 Lightning 30B-A3B | 64.34 score 95% CI [-1.36, +1.37] | maintainer-run | – | SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma 4 E2B | 64.13 score 95% CI [-1.39, +1.41] | maintainer-run | – | SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma-SEA-LION-v4.5-E2B-IT | 63.96 score 95% CI [-1.38, +1.37] | maintainer-run | – | SEA-HELM Indonesian, open-weight models, average of 2,000 bootstraps | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3.5-4B | 63.82 score 95% CI [-0.65, +0.64] | maintainer-run | Checkpoint Qwen3.5-4B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Nemotron-SEA-LION-v4.8-120B-A12B | 63.44 score 95% CI [-0.63, +0.61] | maintainer-run | – | SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Llama 4 Scout | 63.43 score 95% CI [-0.78, +0.79] | maintainer-run | Checkpoint Llama-4-Scout-17B-16E-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Mistral Small 4 | 63.14 score 95% CI [-1.42, +1.50] | maintainer-run | Checkpoint Mistral-Small-4-119B-2603 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Apertus v1.5 8B | 62.23 score 95% CI [-1.32, +1.32] | maintainer-run | Checkpoint Apertus-v1.5-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| gpt-oss-20b | 60.64 score 95% CI [-0.64, +0.63] | maintainer-run | Checkpoint gpt-oss-20b as listed, quantization mxfp4; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Llama-3.3-70B-Instruct | 60.44 score 95% CI [-0.70, +0.70] | maintainer-run | Checkpoint Llama-3.3-70B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3-8B | 60.33 score 95% CI [-0.66, +0.67] | maintainer-run | Checkpoint Qwen3-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma-SEA-LION-v4.5-E2B-IT | 59.23 score 95% CI [-0.66, +0.67] | maintainer-run | – | SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma 4 E2B | 58.82 score 95% CI [-0.64, +0.66] | maintainer-run | – | SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
 MERaLiON-2-10B | 58.01 score 95% CI -1.66 to +1.65 | maintainer-run | Indonesian overall; average of 2000 bootstraps | rank 16 of 18, SEA-HELM Indonesian leaderboard, 18 models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma-3-4B-it | 57.71 score 95% CI [-1.76, +1.67] | maintainer-run | Checkpoint gemma-3-4b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen3-VL-8B-Instruct | 56.6 score 95% CI [-0.79, +0.75] | maintainer-run | Checkpoint Qwen3-VL-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Qwen-SEA-LION-v4-8B-VL | 56.41 score 95% CI [-0.79, +0.77] | maintainer-run | Checkpoint Qwen-SEA-LION-v4-8B-VL as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Apertus v1.5 70B | 55.2 score 95% CI [-0.66, +0.61] | maintainer-run | – | SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Llama-3.1-8B-Instruct | 54.48 score 95% CI [-1.57, +1.56] | maintainer-run | Checkpoint Llama-3.1-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Apertus v1.5 8B | 54.16 score 95% CI [-0.62, +0.62] | maintainer-run | Checkpoint Apertus-v1.5-8B as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| GLM 4.7 Flash | 53.76 score 95% CI [-0.57, +0.57] | maintainer-run | Checkpoint GLM-4.7-Flash as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Mistral Small 4 | 52.07 score 95% CI [-0.61, +0.64] | maintainer-run | Checkpoint Mistral-Small-4-119B-2603 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Nemotron-SEA-LION-v4.8-30B-A3B | 51.57 score 95% CI [-0.52, +0.52] | maintainer-run | – | SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Nemotron 3 Super | 49.3 score 95% CI [-0.51, +0.53] | maintainer-run | – | SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Gemma-3-4B-it | 47.84 score 95% CI [-0.75, +0.74] | maintainer-run | Checkpoint gemma-3-4b-it as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Nemotron 3 Nano 30B-A3B | 46.89 score 95% CI [-0.50, +0.55] | maintainer-run | Checkpoint NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Nemotron 3.5 Lightning 30B-A3B | 46.06 score 95% CI [-0.54, +0.54] | maintainer-run | – | SEA-HELM SEA average, open-weight models | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Llama-3.1-8B-Instruct | 37.2 score 95% CI [-0.65, +0.67] | maintainer-run | Checkpoint Llama-3.1-8B-Instruct as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Tiny Aya water | 33.23 score 95% CI [-1.13, +1.13] | maintainer-run | Checkpoint tiny-aya-water as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Tiny Aya global | 32.12 score 95% CI [-1.13, +1.13] | maintainer-run | Checkpoint tiny-aya-global as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, Indonesian (ID) | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Tiny Aya water | 19.75 score 95% CI [-0.40, +0.43] | maintainer-run | Checkpoint tiny-aya-water as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
| Tiny Aya global | 19.19 score 95% CI [-0.40, +0.41] | maintainer-run | Checkpoint tiny-aya-global as listed; release view (data last updated 18 Sep 2026); score is the average of 2000 bootstraps | SEA-HELM leaderboard, 58 models, SEA average | 18 Sep 2026 | leaderboard.sea-lion.ai |
Where this comes from
Scores from different settings, harnesses or dates are not directly comparable; the row says which. Sorted by score, highest first, as a reading aid, not a ranking.
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "SEA-HELM", https://aurion.semai.id/benchmarks/sea-helm/, records of October 2026.