Low-Resource and Regional Language Resources

Datasets, benchmarks and adaptation for under-served languages.

Tier Grades

Tier A: Frontier AI labs

feasible

medium confidence

Qwen3's growth to 119 languages shows frontier labs can cover regional languages.

Tier B: Indonesian AI labs

feasible

medium confidence

Sahabat-AI covers five Indonesian languages, and the work is data-heavy rather than compute-heavy.

Tier C: Institution units such as ITB

feasible

high confidence

The Indonesian NLP community built NusaX and NusaCrowd, which are data efforts rather than compute efforts.

Tier D: Organisations such as SEMAI (PoT, PoC, PoV)

stretch

low confidence

Months-to-years data work is long for a proof of concept; the notes give no proof-of-concept anchor.

Where this comes from

Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.

Anchors

ComputeData effort dominates compute; compute not sourced.
DataNusaX: 10 Indonesian local languages; NusaCrowd: 137 datasets, 118 loaders, first multilingual ASR benchmark for Indonesian languages.
Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): not sourced in the notes.
Time horizonMonths to years.
Shelf-life signalsQwen3 went from 29 to 119 languages and dialects, frontier lab overlap with regional coverage
Pipeline roleTier C and Tier B build local datasets and benchmarks that Tier A and Tier B train on and Tier D uses to show local-language value.

Where this comes from

Team size is a proxy (author counts of exemplar papers), not a headcount.

Who and What

Exemplars

NusaX NusaCrowd AI Singapore

SEA-LION

Observed actors

Indonesian NLP community AI Singapore

AI Singapore

Linked models

SEA-LION v3LLMexemplar

SEA-LION-E5-Embedding-600Membeddingproduced by

SEA-LION ModernBERT 300Membeddingproduced by

SEA-LION ModernBERT 600Membeddingproduced by

SEA-LION ModernBERT-Embedding 300Membeddingproduced by

SEA-LION ModernBERT-Embedding 600Membeddingproduced by

Nemotron-SEA-LION-v4.8-120B-A12BLLMproduced by

Nemotron-SEA-LION-v4.8-30B-A3BLLMproduced by

Apertus v1.5 70Bmultimodal LLMproduced by

Apertus v1.5 8Bmultimodal LLMproduced by

Gemma-SEA-LION-v4.5-E2B-ITedge LLMproduced by

Qwen-SEA-LION-v4.5-27B-ITLLMproduced by

Gemma 4 E2Bedge LLMused by

Qwen3.6-27Bmultimodal LLMused by

Tiny AyaLLMproduced by

Tiny Aya baseedge LLMproduced by

Tiny Aya earthedge LLMproduced by

Tiny Aya fireedge LLMproduced by

Tiny Aya globaledge LLMproduced by

Tiny Aya wateredge LLMproduced by

Gemma-SEA-Guard-12B-2602guardproduced by

Llama-SEA-Guard-8B-2602guardproduced by

Qwen-SEA-Guard-8B-2602guardproduced by

SEA-Guardguardproduced by

+59

Qwen3.5-397B-A17Bmultimodal LLMused by

Nemotron 3 SuperLLMused by

SeaLLMs-Audio-7Bmultimodal LLMproduced by

Nemotron 3 Nano 30B-A3BLLMused by

Qwen-SEA-LION-v4-8B-VLmultimodal LLMproduced by

Gemma-SEA-LION-v4-27BLLMproduced by

Gemma-SEA-LION-v4-27B-ITLLMproduced by

Llama-Sahabat-AI-v2-70B-ITLLMproduced by

A*STAR (Institute for Infocomm Research, I2R)

MERaLiON-2-3Bmultimodal LLMproduced by

MERaLiON-2multimodal LLMproduced by

MERaLiON-2-10Bmultimodal LLMproduced by

Aya Visionmultimodal LLMproduced by

Aya Vision 32Bmultimodal LLMproduced by

Aya Vision 8Bmultimodal LLMproduced by

gemma2-9b-cpt-sahabatai-v1LLMproduced by

llama3-8b-cpt-sahabatai-v1LLMproduced by

Sahabat-AI v1LLMproduced by

Aya Expanse 32BLLMproduced by Vintern-1B v2OCR and documentproduced by

SeaLLMs v3LLMproduced by

SeaLLMs v3 1.5BLLMproduced by

SeaLLMs v3 7BLLMproduced by

Gemma 2 9BLLMused by

SeaLMMM 7B v0.1multimodal LLMproduced by CendolLLMproduced by

Komodo-7BLLMproduced by

Aya 101LLMproduced by

UMT5LLMproduced by

BLOOMZLLMproduced by

BLOOMZ 7B1LLMproduced by

BLOOMLLMproduced by

BLOOM 176BLLMproduced by

BLOOM 560MLLMproduced by

BLOOM 7B1LLMproduced by

ViT5 baseLLMproduced by

wav2vec2-indonesian-javanese-sundanesespeech recognitionproduced by IndoBERT baseembeddingproduced by IndoBERT largeembeddingproduced by IndoBERT liteembeddingproduced by

fastText word vectorsembeddingproduced by

Grab custom vision LLM 1BOCR and documentproduced by

Grab custom vision LLM 2BOCR and documentproduced by

Flax Community (Hugging Face community event)

Indonesian RoBERTa baselanguage encoderproduced by

OPUS-MT en-idtranslationproduced by

OPUS-MT id-entranslationproduced by

gemma2-9b-cpt-sea-lionv3-base (SEA-LION v3)LLMproduced by

SEA-LIONLLMproduced by

SEA-LION v4.5 (Qwen) 27BLLMproduced by

Command A TranslateLLMproduced by

GemmaLLMused by

PaLM 2LLMused by

gemma2-9b-cpt-sahabatai-v1-base (Sahabat-AI v1 base)LLMproduced by

gemma2-9b-cpt-sahabatai-v1-instruct (Sahabat-AI v1 instruct)LLMproduced by

Sahabat-AILLMproduced by IndoBERTembeddingproduced by

Llama 3.1 70BLLMused by

Llama 3 8BLLMused by

LLM-jp-4-VL-9Bmultimodal LLMproduced by T5 base Indonesian summarizationLLMproduced by

Links

R&D TopicLow-Resource and Regional Language Resources

Models

SEA-LION v3AI Singapore, LLM

Grab custom vision LLM 1BGrab, OCR and document

Grab custom vision LLM 2BGrab, OCR and document Vintern-1B v2OCR and document+79 more

Use CasesIndonesian and Regional-Language LLMsPublic Sector Indonesian NLU (Classification, NER)Public Sector Translation Including IndonesianPublic Sector Embeddings and Retrieval for SEA LanguagesPublic Sector+5 more

Who Runs It

PT Bank Tabungan Negara (BTN)In production, 1 Jul 2026

PT PegadaianIn production, 5 Jun 2026

Bank Central Asia (BCA)In production, 16 Mar 2026

Bank Central Asia (BCA)In production, 16 Mar 2026+13 more

Graph

Models 83

SEA-LIONAI Singapore, LLMproduced by

Gemma-SEA-LION-v4.5-E2B-ITAI Singapore, edge LLMproduced by

Llama-Sahabat-AI-v2-70B-ITGoTo Gojek Tokopedia, Indosat Ooredoo Hutchison, AI Singapore, LLMproduced by

llama3-8b-cpt-sahabatai-v1GoTo Gojek Tokopedia, Indosat Ooredoo Hutchison, AI Singapore, LLMproduced by

Sahabat-AIGoTo Gojek Tokopedia, AI Singapore, LLMproduced by

BLOOMBigScience Workshop, LLMproduced by

Gemma 2 9BGoogle, LLMused by

Gemma 4 E2BGoogle, Google DeepMind, edge LLMused by

gemma2-9b-cpt-sahabatai-v1GoTo Gojek Tokopedia, Indosat Ooredoo Hutchison, AI Singapore, LLMproduced by

Grab custom vision LLM 1BGrab, OCR and documentproduced by

Llama 3 8BMeta, LLMused by

Qwen-SEA-LION-v4.5-27B-ITAI Singapore, LLMproduced by

+71

Tiny Aya baseCohere Labs, edge LLMproduced by CendolIndoNLP, LLMproduced by

MERaLiON-2-10BA*STAR (Institute for Infocomm Research, I2R), multimodal LLMproduced by

MERaLiON-2-3BA*STAR (Institute for Infocomm Research, I2R), multimodal LLMproduced by

Nemotron 3 SuperNVIDIA, LLMused by

Qwen-SEA-LION-v4-8B-VLAI Singapore, multimodal LLMproduced by

Qwen3.5-397B-A17BAlibaba (Qwen), multimodal LLMused by

SEA-LION-E5-Embedding-600MAI Singapore, embeddingproduced by

Tiny AyaCohere Labs, LLMproduced by

Apertus v1.5 70BSwiss AI (EPFL, ETH Zurich, CSCS), multimodal LLMproduced by

Apertus v1.5 8BSwiss AI (EPFL, ETH Zurich, CSCS), multimodal LLMproduced by

Gemma-SEA-LION-v4-27BAI Singapore, LLMproduced by

gemma2-9b-cpt-sahabatai-v1-base (Sahabat-AI v1 base)GoTo Gojek Tokopedia, LLMproduced by IndoBERTIndoNLP, embeddingproduced by

MERaLiON-2A*STAR (Institute for Infocomm Research, I2R), multimodal LLMproduced by

Nemotron 3 Nano 30B-A3BNVIDIA, LLMused by

Qwen3.6-27BAlibaba (Qwen), multimodal LLMused by

Sahabat-AI v1GoTo Gojek Tokopedia, Indosat Ooredoo Hutchison, LLMproduced by

SEA-GuardAI Singapore, guardproduced by

SEA-LION ModernBERT 300MAI Singapore, embeddingproduced by

SEA-LION ModernBERT 600MAI Singapore, embeddingproduced by

SEA-LION ModernBERT-Embedding 300MAI Singapore, embeddingproduced by

SEA-LION ModernBERT-Embedding 600MAI Singapore, embeddingproduced by

SeaLLMs v3Alibaba DAMO, LLMproduced by

Aya VisionCohere Labs, multimodal LLMproduced by

BLOOMZBigScience Workshop, LLMproduced by

BLOOMZ 7B1BigScience Workshop, LLMproduced by

GemmaGoogle, LLMused by

Gemma-SEA-LION-v4-27B-ITAI Singapore, LLMproduced by

gemma2-9b-cpt-sea-lionv3-base (SEA-LION v3)AI Singapore, LLMproduced by

Grab custom vision LLM 2BGrab, OCR and documentproduced by IndoBERT baseIndoNLP, embeddingproduced by IndoBERT largeIndoNLP, embeddingproduced by IndoBERT liteIndoNLP, embeddingproduced by

Komodo-7BYellow.ai, LLMproduced by

Llama 3.1 70BMeta, LLMused by

Nemotron-SEA-LION-v4.8-120B-A12BAI Singapore, LLMproduced by

Nemotron-SEA-LION-v4.8-30B-A3BAI Singapore, LLMproduced by

OPUS-MT en-idUniversity of Helsinki (Helsinki-NLP), translationproduced by

OPUS-MT id-enUniversity of Helsinki (Helsinki-NLP), translationproduced by

PaLM 2Google, LLMused by

Qwen-SEA-Guard-8B-2602AI Singapore, guardproduced by

Tiny Aya earthCohere Labs, edge LLMproduced by

Tiny Aya fireCohere Labs, edge LLMproduced by

Tiny Aya globalCohere Labs, edge LLMproduced by

Tiny Aya waterCohere Labs, edge LLMproduced by

Aya 101Cohere Labs, LLMproduced by

Aya Vision 32BCohere Labs, multimodal LLMproduced by

Aya Vision 8BCohere Labs, multimodal LLMproduced by

BLOOM 176BBigScience Workshop, LLMproduced by

BLOOM 560MBigScience Workshop, LLMproduced by

BLOOM 7B1BigScience Workshop, LLMproduced by

Gemma-SEA-Guard-12B-2602AI Singapore, guardproduced by

gemma2-9b-cpt-sahabatai-v1-instruct (Sahabat-AI v1 instruct)GoTo Gojek Tokopedia, LLMproduced by

Indonesian RoBERTa baseFlax Community (Hugging Face community event), language encoderproduced by

Llama-SEA-Guard-8B-2602AI Singapore, guardproduced by

LLM-jp-4-VL-9BNII LLM-jp, multimodal LLMproduced by

SeaLLMs v3 1.5BAlibaba DAMO, LLMproduced by

SeaLLMs v3 7BAlibaba DAMO, LLMproduced by

SeaLMMM 7B v0.1Alibaba DAMO, multimodal LLMproduced by T5 base Indonesian summarizationCahya Wirawan, LLMproduced by

ViT5 baseVietAI, LLMproduced by

wav2vec2-indonesian-javanese-sundaneseIndonesian NLP, speech recognitionproduced by

Aya Expanse 32BCohere, LLMproduced by

Command A TranslateCohere, LLMproduced by

fastText word vectorsMeta, embeddingproduced by

SEA-LION v3AI Singapore, LLMexemplar

SEA-LION v4.5 (Qwen) 27BAI Singapore, LLMproduced by

SeaLLMs-Audio-7BAlibaba DAMO, multimodal LLMproduced by

UMT5Google, LLMproduced by Vintern-1B v2OCR and documentproduced by

Papers

IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language UnderstandingBryan Wilie et al., arXiv preprint, 2020 founding
SEA-LION: Southeast Asian Languages in One NetworkRaymond Ng et al., ACL 2025 (per arXiv comment), 2025 state of the art
Aya Dataset: An Open-Access Collection for Multilingual Instruction TuningShivalika Singh et al., arXiv preprint, 2024 state of the art
Aya Model: An Instruction Finetuned Open-Access Multilingual Language ModelAhmet Üstün et al., arXiv preprint, 2024 state of the art
NusaCrowd: Open Source Initiative for Indonesian NLP ResourcesSamuel Cahyawijaya et al., arXiv preprint, 2022 state of the art

Where this comes from

The papers to read first: where the direction began, its current best, and a survey when one exists.

Sources

Qwen3 Technical Report arxiv.org https://arxiv.org/abs/2505.09388
Sahabat-AI Gets Smarter: Indosat and GoTo Launch New 70 Billion Parameter Model with Multilingual Chat Service gotocompany.com https://www.gotocompany.com/en/news/press/sahabat-ai-gets-smarter-indosat-and-goto-launch-new-70-billion-parameter-model-with-multilingual-chat-service
SEA-LION: Southeast Asian Languages in One Network arxiv.org https://arxiv.org/abs/2504.05747
NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages arxiv.org https://arxiv.org/abs/2205.15960
NusaCrowd: Open Source Initiative for Indonesian NLP Resources arxiv.org https://arxiv.org/abs/2212.09648

To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Low-Resource and Regional Language Resources", https://aurion.semai.id/topics/low-resource-regional-language-resources/, records of October 2026.