Speech Recognition for Indonesian and Regional Languages
Adapting speech recognition to Indonesian and regional languages such as Javanese and Sundanese, including spontaneous, informal and noisy speech.
Tier Grades
Tier A: Frontier AI labs
Frontier labs cover Indonesian inside multilingual models such as Whisper and MMS, which the IDSV study evaluated zero-shot.
Tier B: Indonesian AI labs
Fine-tuning Whisper and MMS on a compiled Indonesian dataset, as the IDSV study did, is within an Indonesian lab's means.
Tier C: Institution units such as ITB
The IDSV study itself is institution work by ITB and UI with partners, with 6 authors.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
A delivery organisation can fine-tune a released model on client speech, since the IDSV study found the fine-tuned Whisper gave the lowest error.
Where this comes from
Anchors
- ComputeNot in the sources read: the IDSV study abstract gives no compute; it fine-tunes existing Whisper and MMS models.
- DataIDSV study: Indonesian data is dominated by read, formal and clean speech, so the authors compiled a dataset with speaking-style, context and noise variability.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 6 authors on the IDSV study (ITB, UI, JAIST, NAIST).
- Time horizonMonths.
- Shelf-life signalsMultilingual models reaching low error on spontaneous Indonesian without fine-tuning, Public Indonesian corpora with spontaneous and noisy speech
- Pipeline roleTier A supplies multilingual base models; Tier C builds Indonesian speech datasets and fine-tunes; Tier B scales them; Tier D brings domain recordings and tests them in proofs of value.
Where this comes from
Who and What
Links
FT-whisper (IDSV)Institut Teknologi Bandung (ITB), speech recognitionMMS fine-tuned with LM (IDSV)Institut Teknologi Bandung (ITB), speech recognition
wav2vec2-indonesian-javanese-sundaneseIndonesian NLP, speech recognitionMMS fine-tuned, no LM (IDSV)Institut Teknologi Bandung (ITB), speech recognitionModels 4
FT-whisper (IDSV)Institut Teknologi Bandung (ITB), Universitas Indonesia, JAIST, NAIST, speech recognitionexemplarMMS fine-tuned with LM (IDSV)Institut Teknologi Bandung (ITB), Universitas Indonesia, JAIST, NAIST, speech recognitionexemplarMMS fine-tuned, no LM (IDSV)Institut Teknologi Bandung (ITB), Universitas Indonesia, JAIST, NAIST, speech recognitionproduced bywav2vec2-indonesian-javanese-sundaneseIndonesian NLP, speech recognitionexemplarPapers
- Enhancing Indonesian Automatic Speech Recognition: Evaluating Multilingual Models with Diverse Speech VariabilitiesAulia Adila et al., arXiv preprint, 2024 state of the art
- Scaling Speech Technology to 1,000+ LanguagesVineel Pratap et al., arXiv preprint, 2023 state of the art
- NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource LanguagesSamuel Cahyawijaya et al., arXiv preprint, 2023 state of the art
- Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford et al., arXiv preprint, 2022 state of the art
- FLEURS: Few-shot Learning Evaluation of Universal Representations of SpeechAlexis Conneau et al., arXiv preprint, 2022 state of the art
Where this comes from
Sources
- arxiv.org/abs/2410.08828 https://arxiv.org/abs/2410.08828
- huggingface.co/indonesian-nlp/wav2vec2-indonesian-javanese-s... https://huggingface.co/indonesian-nlp/wav2vec2-indonesian-javanese-sundanese
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Speech Recognition for Indonesian and Regional Languages", https://aurion.semai.id/topics/speech-recognition-indonesian-regional-languages/, records of October 2026.