Speech Enhancement and Audio Source Separation
Removing noise from speech and splitting mixtures into separate speakers, instruments or sounds.
Tier Grades
Tier A: Frontier AI labs
feasible
Meta and Alibaba release separation and enhancement models.
Tier B: Indonesian AI labs
feasible
Public mixtures such as WSJ0-2mix and Libri2Mix make training affordable for a national lab.
Tier C: Institution units such as ITB
feasible
SepFormer came from a 5-author team on standard public benchmarks.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Released enhancement models can clean call-centre and field recordings before transcription in a pilot.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeSepFormer stays competitive when downsampling its encoded representation by a factor of 8; training compute is not in the abstracts.
- DataSepFormer: WSJ0-2mix and WSJ0-3mix (SI-SNRi 22.3 dB and 19.5 dB); MossFormer2: WSJ0-2/3mix, Libri2Mix and WHAM!/WHAMR!.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 5 authors on SepFormer; 10 on MossFormer2.
- Time horizonMonths.
- Shelf-life signalsRecognisers robust enough to noise that separate enhancement adds nothing
- Pipeline roleTier A and Tier C publish separation models; Tier B tunes them for local noise; Tier D puts them in front of transcription in pilots.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
SAM Audiomusic and audioexemplar
MossFormer2music and audioexemplar
SepFormermusic and audioexemplarZipEnhancermusic and audioproduced byFRCRNmusic and audioproduced byMetricGAN+music and audioproduced byDTLN (Dual-signal Transformation LSTM Network)TinyMLproduced by
Open-Unmixmusic and audioproduced byRNNoiseTinyMLproduced by
STFT-TCNN speech enhancementTinyMLproduced byLinks
R&D TopicSpeech Enhancement and Audio Source Separation
ModelsSepFormerSpeechBrain community, music and audioSAM AudioMeta, music and audioMossFormer2Alibaba (Qwen), music and audioMetricGAN+SpeechBrain community, music and audio+6 more
SepFormerSpeechBrain community, music and audioSAM AudioMeta, music and audioMossFormer2Alibaba (Qwen), music and audioMetricGAN+SpeechBrain community, music and audio+6 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 10
STFT-TCNN speech enhancementSTMicroelectronics, TinyMLproduced byDTLN (Dual-signal Transformation LSTM Network)TinyMLproduced byFRCRNAlibaba (Qwen), music and audioproduced byMetricGAN+SpeechBrain community, music and audioproduced byMossFormer2Alibaba (Qwen), music and audioexemplarOpen-UnmixINRIA, music and audioproduced byRNNoiseTinyMLproduced bySAM AudioMeta, music and audioexemplarSepFormerSpeechBrain community, music and audioexemplarZipEnhancerAlibaba (Qwen), music and audioproduced byPapers
- Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech SeparationYi Luo et al., arXiv preprint, 2018 founding
- Music Source Separation with Band-Split RoPE TransformerWei-Tsung Lu et al., arXiv preprint, 2023 state of the art
- Hybrid Transformers for Music Source SeparationSimon Rouard et al., arXiv preprint, 2022 state of the art
- Attention is All You Need in Speech SeparationCem Subakan et al., ICASSP 2021 (per arXiv comment), 2020 state of the art
- DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech EnhancementYanxin Hu et al., Interspeech 2020 (per arXiv comment), 2020 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2010.13154 https://arxiv.org/abs/2010.13154
- arxiv.org/abs/2312.11825 https://arxiv.org/abs/2312.11825
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Speech Enhancement and Audio Source Separation", https://aurion.semai.id/topics/speech-enhancement-source-separation/, records of October 2026.