Voice Activity, Turn and Keyword Detection
Small models that detect when someone speaks, when they finish a turn, or which short command they said, as the front end of speech systems.
Tier Grades
Tier A: Frontier AI labs
feasible
NVIDIA publishes MarbleNet and MatchboxNet in NeMo.
Tier B: Indonesian AI labs
feasible
Models with a tenth of the usual parameters train cheaply on Indonesian audio.
Tier C: Institution units such as ITB
feasible
MarbleNet and MatchboxNet came from 2 to 3 author teams on public data.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
An 8M parameter turn detector running on CPU drops straight into a voice-bot proof of concept.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeMarbleNet matches a state-of-the-art VAD with about one tenth of the parameters; Smart Turn v3 has 8M parameters and its card announces CPU inference in 12 ms.
- DataMatchboxNet: Google Speech Commands with an auxiliary noise dataset for augmentation.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 3 authors on MarbleNet; 2 on MatchboxNet.
- Time horizonWeeks to months.
- Shelf-life signalsFull-duplex dialogue models that handle turn-taking internally
- Pipeline roleTier A and Tier C release small detectors; Tier B tunes them for Indonesian; Tier D uses them as the front end of voice agents.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
Links
R&D TopicVoice Activity, Turn and Keyword Detection
Models
MarbleNetNVIDIA, audio classification
FSMN-VADAlibaba (Qwen), audio classification
Smart Turn v3Pipecat (Daily), audio classificationMatchboxNetNVIDIA, audio classification+2 more
MarbleNetNVIDIA, audio classificationFSMN-VADAlibaba (Qwen), audio classificationSmart Turn v3Pipecat (Daily), audio classificationMatchboxNetNVIDIA, audio classification+2 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 6
FireRedVADrednote (Xiaohongshu), audio classificationproduced byFSMN-VADAlibaba (Qwen), audio classificationexemplarMarbleNetNVIDIA, audio classificationexemplarMatchboxNetNVIDIA, audio classificationproduced bySmart Turn v3Pipecat (Daily), audio classificationexemplar
VAD CRDNN (LibriParty)SpeechBrain community, audio classificationproduced byPapers
- TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken DialogErik Ekstedt et al., EMNLP 2020 (per arXiv comment), 2020 founding
- Turn-taking and Backchannel Prediction with Acoustic and Large Language Model FusionJinhan Wang et al., ICASSP 2024 (per arXiv comment), 2024 state of the art
- Voice Activity Projection: Self-supervised Learning of Turn-taking EventsErik Ekstedt et al., INTERSPEECH 2022 (per arXiv comment), 2022 state of the art
- MarbleNet: Deep 1D Time-Channel Separable Convolutional Neural Network for Voice Activity DetectionFei Jia et al., ICASSP 2021 (per arXiv comment), 2020 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2010.13886 https://arxiv.org/abs/2010.13886
- arxiv.org/abs/2004.08531 https://arxiv.org/abs/2004.08531
- huggingface.co/pipecat-ai/smart-turn-v3 https://huggingface.co/pipecat-ai/smart-turn-v3
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Voice Activity, Turn and Keyword Detection", https://aurion.semai.id/topics/voice-activity-turn-detection/, records of October 2026.