Massively Multilingual Speech Recognition
Training one speech recognition model on hundreds of thousands of hours or hundreds of languages so it transcribes many languages, often zero-shot.
Tier Grades
Tier A: Frontier AI labs
feasible
OpenAI and Meta trained on 680,000 hours (Whisper) and scaled pre-training to 7B parameters (Omnilingual ASR).
Tier B: Indonesian AI labs
stretch
Training from scratch at 680,000 hours or 7B parameters is beyond a national lab, but fine-tuning the released 300M to 7B models is within reach.
Tier C: Institution units such as ITB
stretch
The ITB and UI study fine-tuned Whisper and MMS on Indonesian rather than training a multilingual model, which is the realistic scope for a unit.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Released checkpoints from 300M parameters upward, trained on up to 680,000 hours, can be applied directly in a transcription proof of concept.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeOmnilingual ASR scales self-supervised pre-training to 7B parameters and releases a family from 300M to 7B; Whisper and MMS abstracts give no training compute.
- DataWhisper: 680,000 hours of multilingual and multitask supervision; MMS: one ASR model for 1,107 languages built from readings of religious texts; Omnilingual ASR: over 1,600 languages.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 6 authors on Whisper; 33 listed on Omnilingual ASR.
- Time horizonMonths to fine-tune a released model; years to build a new one.
- Shelf-life signalsA single open model covering most languages with low error, leaving only fine-tuning, Audio language models replacing dedicated recognisers
- Pipeline roleTier A releases multilingual recognisers; Tier B and Tier C fine-tune and benchmark them on Indonesian and regional speech; Tier D deploys them in transcription proofs of value.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
Omnilingual ASRspeech recognitionexemplar
OWSM v4speech recognitionexemplarMMSspeech recognitionexemplar
Whisperspeech recognitionexemplar
MAI-Transcribe-2speech recognitionproduced by
Nemotron 3.5 ASR Streaming 0.6Bspeech recognitionproduced byMAI-Transcribe-1speech recognitionproduced by
Cohere Transcribe (cohere-transcribe-03-2026)speech recognitionproduced by
granite-4.0-1b-speechspeech recognitionproduced byVibeVoice-ASRspeech recognitionproduced by
Fun-ASR-Nano (Fun-ASR-Nano-2512)speech recognitionproduced by
GLM-ASR-Nano (GLM-ASR-Nano-2512)speech recognitionproduced bycanary-1b-v2speech recognitionproduced by
Dolphinspeech recognitionproduced bycanary-180m-flashspeech recognitionproduced byCanary-1B-flashspeech recognitionproduced byOWSM-CTC v4 1Bspeech recognitionproduced by
FireRedASRspeech recognitionproduced byParakeet-TDT-0.6Bspeech recognitionproduced byParakeet-TDT-0.6B v3speech recognitionproduced by
Kotoba-Whisper v2.0speech recognitionproduced bySenseVoiceSmallspeech recognitionproduced bymms-300m-1130-forced-alignerspeech recognitionproduced byFunASR Conformer (English)speech recognitionproduced by+14
Canary-1Bspeech recognitionproduced byWhisper large-v3-turbospeech recognitionproduced bySeamlessM4Ttranslationproduced bymms-1b-allspeech recognitionproduced by
WhisperXspeech recognitionproduced byWhisper large-v3speech recognitionproduced bywhisper-large-v2speech recognitionproduced bywhisper-smallspeech recognitionproduced bywhisper-basespeech recognitionproduced bywhisper-mediumspeech recognitionproduced byParaformer-zh (+ streaming)speech recognitionproduced byFun-ASR-MLT-Nanospeech recognitionproduced bywhisper-tinyspeech recognitionproduced by
VoxLingua107 language ID (ECAPA)audio classificationproduced byLinks
R&D TopicMassively Multilingual Speech Recognition
ModelsMMSMeta, speech recognitionOmnilingual ASRMeta, speech recognitionWhisperOpenAI, speech recognitionOWSM v4Carnegie Mellon University, speech recognition+34 more
MMSMeta, speech recognitionOmnilingual ASRMeta, speech recognitionWhisperOpenAI, speech recognitionOWSM v4Carnegie Mellon University, speech recognition+34 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 38
WhisperOpenAI, speech recognitionexemplarMMSMeta, speech recognitionexemplarWhisper large-v3OpenAI, speech recognitionproduced byParakeet-TDT-0.6BNVIDIA, speech recognitionproduced byParakeet-TDT-0.6B v3NVIDIA, speech recognitionproduced byOmnilingual ASRMeta, speech recognitionexemplarSeamlessM4TMeta, translationproduced byWhisper large-v3-turboOpenAI, speech recognitionproduced bywhisper-smallOpenAI, speech recognitionproduced bywhisper-tinyOpenAI, speech recognitionproduced bycanary-180m-flashNVIDIA, speech recognitionproduced byCanary-1BNVIDIA, speech recognitionproduced by+26
Canary-1B-flashNVIDIA, speech recognitionproduced byFun-ASR-Nano (Fun-ASR-Nano-2512)Alibaba (Qwen), speech recognitionproduced byFunASR Conformer (English)Alibaba (Qwen), speech recognitionproduced bygranite-4.0-1b-speechIBM, speech recognitionproduced bymms-1b-allMeta, speech recognitionproduced bymms-300m-1130-forced-alignerMahmoudAshraf, speech recognitionproduced byNemotron 3.5 ASR Streaming 0.6BNVIDIA, speech recognitionproduced byOWSM v4Carnegie Mellon University, speech recognitionexemplarOWSM-CTC v4 1BCarnegie Mellon University, speech recognitionproduced byParaformer-zh (+ streaming)Alibaba (Qwen), speech recognitionproduced bySenseVoiceSmallAlibaba (Qwen), speech recognitionproduced bywhisper-baseOpenAI, speech recognitionproduced bywhisper-large-v2OpenAI, speech recognitionproduced bywhisper-mediumOpenAI, speech recognitionproduced bycanary-1b-v2NVIDIA, speech recognitionproduced byCohere Transcribe (cohere-transcribe-03-2026)Cohere, speech recognitionproduced byDolphinDataoceanAI, speech recognitionproduced byFireRedASRrednote (Xiaohongshu), speech recognitionproduced byFun-ASR-MLT-NanoAlibaba (Qwen), speech recognitionproduced byGLM-ASR-Nano (GLM-ASR-Nano-2512)Z.ai (Zhipu), speech recognitionproduced byKotoba-Whisper v2.0Kotoba Technologies, speech recognitionproduced byMAI-Transcribe-1Microsoft AI, speech recognitionproduced byMAI-Transcribe-2Microsoft AI, speech recognitionproduced byVibeVoice-ASRMicrosoft AI, speech recognitionproduced byVoxLingua107 language ID (ECAPA)SpeechBrain community, audio classificationproduced byWhisperXUniversity of Oxford, speech recognitionproduced byPapers
- Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford et al., arXiv preprint, 2022 founding
- Scaling Speech Technology to 1,000+ LanguagesVineel Pratap et al., arXiv preprint, 2023 state of the art
- Google USM: Scaling Automatic Speech Recognition Beyond 100 LanguagesYu Zhang et al., arXiv preprint, 2023 state of the art
- Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available DataYifan Peng et al., ASRU 2023 (per arXiv comment), 2023 state of the art
- XLS-R: Self-supervised Cross-lingual Speech Representation Learning at ScaleArun Babu et al., arXiv preprint, 2021 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2212.04356 https://arxiv.org/abs/2212.04356
- arxiv.org/abs/2305.13516 https://arxiv.org/abs/2305.13516
- arxiv.org/abs/2511.09690 https://arxiv.org/abs/2511.09690
- arxiv.org/abs/2410.08828 https://arxiv.org/abs/2410.08828
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Massively Multilingual Speech Recognition", https://aurion.semai.id/topics/massively-multilingual-speech-recognition/, records of October 2026.