Streaming and Low-Latency Speech Recognition
Speech recognition built to transcribe live audio with small delay and small compute, for voice commands, live captions and edge devices.
Tier Grades
Tier A: Frontier AI labs
feasible
NVIDIA redesigned Conformer to be 2.8 times faster and scaled it to billions of parameters.
Tier B: Indonesian AI labs
feasible
Small models such as Moonshine Tiny, with 5 times less compute than Whisper tiny-en, fit modest training budgets, though training hours are not in the sources.
Tier C: Institution units such as ITB
feasible
Architecture changes such as Moonshine's padding-free training are paper-scale work with 6 authors.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Released low-latency checkpoints can run live captioning or voice commands in a proof of concept without training.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeMoonshine Tiny needs 5 times less compute than Whisper tiny-en for a 10-second segment; Fast Conformer is 2.8 times faster than Conformer and transcribes up to 11 hours.
- DataNot in the sources read: the Moonshine and Fast Conformer abstracts do not state training hours.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 6 authors on Moonshine; 11 on Fast Conformer.
- Time horizonMonths.
- Shelf-life signalsStreaming models matching offline accuracy at under 200 ms, On-device recognisers shipped in phone operating systems
- Pipeline roleTier A and Tier C publish fast architectures and checkpoints; Tier B adapts them to Indonesian; Tier D puts them into live transcription and voice command pilots.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Moonshine (Moonshine Voice)
Parakeet-TDT-0.6B v3Nemotron Speech Streaming EN 0.6B
Voxtral Mini 4B Realtime 2602Observed actors
Linked models
Voxtral Mini 4B Realtime 2602speech recognitionexemplarNemotron Speech Streaming EN 0.6Bspeech recognitionexemplarParakeet-TDT-0.6B v3speech recognitionexemplarMoonshine (Moonshine Voice)speech recognitionexemplarNemotron 3.5 ASR Streaming 0.6Bspeech recognitionproduced byparakeet-unified-en-0.6bspeech recognitionproduced bymultitalker-parakeet-streaming-0.6b-v1speech recognitionproduced byParakeet-TDT-0.6Bspeech recognitionproduced byParakeet-TDT-0.6B v2speech recognitionproduced by
distil-large-v3.5speech recognitionproduced byparakeet-tdt_ctc-110mspeech recognitionproduced bydistil-large-v3speech recognitionproduced byparakeet-tdt-1.1bspeech recognitionproduced by
Whisper large-v3-turbospeech recognitionproduced byParakeetspeech recognitionproduced byparakeet-ctc-1.1bspeech recognitionproduced byparakeet-rnnt-1.1bspeech recognitionproduced bydistil-small.enspeech recognitionproduced by
faster-whisper-smallspeech recognitionproduced byfaster-whisper-tinyspeech recognitionproduced byDistil-Whisperspeech recognitionproduced bystt_en_fastconformer_transducer_largespeech recognitionproduced byVoxtral Mini Transcribe 2speech recognitionproduced byLinks
R&D TopicStreaming and Low-Latency Speech Recognition
ModelsParakeet-TDT-0.6B v3NVIDIA, speech recognitionNemotron Speech Streaming EN 0.6BNVIDIA, speech recognitionMoonshine (Moonshine Voice)Useful Sensors, speech recognitionVoxtral Mini 4B Realtime 2602Mistral AI, speech recognition+19 more
Parakeet-TDT-0.6B v3NVIDIA, speech recognitionNemotron Speech Streaming EN 0.6BNVIDIA, speech recognitionMoonshine (Moonshine Voice)Useful Sensors, speech recognitionVoxtral Mini 4B Realtime 2602Mistral AI, speech recognition+19 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 23
ParakeetNVIDIA, speech recognitionproduced bydistil-large-v3Hugging Face, speech recognitionproduced bydistil-large-v3.5Hugging Face, speech recognitionproduced byDistil-WhisperHugging Face, speech recognitionproduced byfaster-whisper-smallSYSTRAN, speech recognitionproduced byfaster-whisper-tinySYSTRAN, speech recognitionproduced byParakeet-TDT-0.6BNVIDIA, speech recognitionproduced byParakeet-TDT-0.6B v3NVIDIA, speech recognitionexemplarmultitalker-parakeet-streaming-0.6b-v1NVIDIA, speech recognitionproduced byParakeet-TDT-0.6B v2NVIDIA, speech recognitionproduced bystt_en_fastconformer_transducer_largeNVIDIA, speech recognitionproduced byWhisper large-v3-turboOpenAI, speech recognitionproduced by+11
distil-small.enHugging Face, speech recognitionproduced byNemotron 3.5 ASR Streaming 0.6BNVIDIA, speech recognitionproduced byparakeet-ctc-1.1bNVIDIA, speech recognitionproduced byparakeet-rnnt-1.1bNVIDIA, speech recognitionproduced byparakeet-tdt_ctc-110mNVIDIA, speech recognitionproduced byparakeet-tdt-1.1bNVIDIA, speech recognitionproduced byparakeet-unified-en-0.6bNVIDIA, speech recognitionproduced byVoxtral Mini 4B Realtime 2602Mistral AI, speech recognitionexemplarMoonshine (Moonshine Voice)Useful Sensors, speech recognitionexemplarNemotron Speech Streaming EN 0.6BNVIDIA, speech recognitionexemplarVoxtral Mini Transcribe 2Mistral AI, speech recognitionproduced byPapers
- Sequence Transduction with Recurrent Neural NetworksAlex Graves, arXiv preprint, 2012 founding
- Zipformer: A faster and better encoder for automatic speech recognitionZengwei Yao et al., ICLR 2024 (per arXiv comment), 2023 state of the art
- Fast Conformer with Linearly Scalable Attention for Efficient Speech RecognitionDima Rekesh et al., ASRU 2023 (per arXiv comment), 2023 state of the art
- Conformer: Convolution-augmented Transformer for Speech RecognitionAnmol Gulati et al., Interspeech 2020 (per arXiv comment), 2020 state of the art
- Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech RecognitionYangyang Shi et al., ICASSP 2021 (per arXiv comment), 2020 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2410.15608 https://arxiv.org/abs/2410.15608
- arxiv.org/abs/2305.05084 https://arxiv.org/abs/2305.05084
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Streaming and Low-Latency Speech Recognition", https://aurion.semai.id/topics/streaming-low-latency-speech-recognition/, records of October 2026.