Efficient Acoustic Encoder Architectures for Speech Recognition
Designing the convolution and attention blocks of speech recognition encoders to cut compute and memory at equal accuracy.
Tier Grades
Tier A: Frontier AI labs
feasible
Google and NVIDIA designed Conformer and Fast Conformer with 11-author teams.
Tier B: Indonesian AI labs
feasible
Training on public sets such as LibriSpeech and WenetSpeech is within a national lab's budget.
Tier C: Institution units such as ITB
feasible
A 10M parameter Conformer reaching 2.7/6.3 WER shows the work fits a university compute budget.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
stretch
Delivery work uses these encoders inside released models rather than designing new ones.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeConformer reaches 2.7/6.3 WER on LibriSpeech with only 10M parameters; Squeezeformer beats Conformer-CTC at the same FLOPs.
- DataLibriSpeech (Conformer, Squeezeformer); LibriSpeech, Aishell-1 and WenetSpeech (Zipformer).
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 11 authors on Conformer; 9 on Zipformer.
- Time horizonMonths.
- Shelf-life signalsEncoder designs converging so that new blocks give no gain at equal FLOPs
- Pipeline roleTier A and Tier C publish encoder designs; Tier B trains them on Indonesian data; Tier D receives them inside released recognisers.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
Zipformerspeech recognitionexemplar
FastConformerspeech recognitionexemplar
Squeezeformerspeech recognitionexemplar
Conformerspeech recognitionexemplar
FunASR Conformer (English)speech recognitionproduced bystt_en_fastconformer_transducer_largespeech recognitionproduced byParaformer-zh (+ streaming)speech recognitionproduced byCitrinetspeech recognitionproduced byContextNetspeech recognitionproduced byQuartzNetspeech recognitionproduced byJasperspeech recognitionproduced by
Deep Speech 2speech recognitionproduced by
Mozilla DeepSpeechspeech recognitionproduced byLinks
R&D TopicEfficient Acoustic Encoder Architectures for Speech Recognition
ModelsFastConformerNVIDIA, speech recognitionConformerGoogle, speech recognitionSqueezeformerUC Berkeley, speech recognitionZipformerk2-fsa, speech recognition+9 more
FastConformerNVIDIA, speech recognitionConformerGoogle, speech recognitionSqueezeformerUC Berkeley, speech recognitionZipformerk2-fsa, speech recognition+9 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 13
stt_en_fastconformer_transducer_largeNVIDIA, speech recognitionproduced byFastConformerNVIDIA, speech recognitionexemplarFunASR Conformer (English)Alibaba (Qwen), speech recognitionproduced byParaformer-zh (+ streaming)Alibaba (Qwen), speech recognitionproduced byCitrinetNVIDIA, speech recognitionproduced byConformerGoogle, speech recognitionexemplarContextNetGoogle, speech recognitionproduced byDeep Speech 2Baidu, speech recognitionproduced byJasperNVIDIA, speech recognitionproduced byMozilla DeepSpeechMozilla, speech recognitionproduced byQuartzNetNVIDIA, speech recognitionproduced bySqueezeformerUC Berkeley, speech recognitionexemplarPapers
- Conformer: Convolution-augmented Transformer for Speech RecognitionAnmol Gulati et al., Interspeech 2020 (per arXiv comment), 2020 founding
- Fast Conformer with Linearly Scalable Attention for Efficient Speech RecognitionDima Rekesh et al., ASRU 2023 (per arXiv comment), 2023 state of the art
- Zipformer: A faster and better encoder for automatic speech recognitionZengwei Yao et al., ICLR 2024 (per arXiv comment), 2023 state of the art
- Squeezeformer: An Efficient Transformer for Automatic Speech RecognitionSehoon Kim et al., NeurIPS 2022 (per arXiv comment), 2022 state of the art
- Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and UnderstandingYifan Peng et al., ICML 2022 (per arXiv comment), 2022 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2005.08100 https://arxiv.org/abs/2005.08100
- arxiv.org/abs/2206.00888 https://arxiv.org/abs/2206.00888
- arxiv.org/abs/2310.11230 https://arxiv.org/abs/2310.11230
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Efficient Acoustic Encoder Architectures for Speech Recognition", https://aurion.semai.id/topics/efficient-speech-encoder-architectures/, records of October 2026.