Joint Audio and Video Generation
Generating video together with synchronised speech, sound effects and music in one model.
Tier Grades
Tier A: Frontier AI labs
feasible
Frontier labs ship native audio in video models.
Tier B: Indonesian AI labs
not feasible
Pretraining 11B audio-video models is not feasible for an Indonesian lab.
Tier C: Institution units such as ITB
stretch
A university unit can study the open models.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Videos with Indonesian narration are a direct content proof of value through APIs.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeOvi uses twin backbones with a 5B audio branch pretrained from scratch; the released checkpoint is 11B.
- DataIn-house audio datasets per the model card; size not given.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 3 authors on the Ovi paper (arXiv 2510.01284).
- Time horizonMonths.
- Shelf-life signalsNative audio becomes standard in every video model, Voice cloning rules restrict generated speech
- Pipeline roleTier A trains joint models; Tier C studies open ones; Tier D uses them for narrated content pilots.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
LTX-2.5video generationexemplar
MOVAvideo generationexemplar
Kling 2.6video generationexemplar
Ovivideo generationexemplar
Vidu Q4 Previewvideo generationproduced by
MiniMax H3 (Hailuo 3)video generationproduced by
FLUX 3 Videovideo generationproduced byLTX-2.3video generationproduced byVidu Q3 Provideo generationproduced byLinks
R&D TopicJoint Audio and Video Generation
ModelsLTX-2.5Lightricks, video generationMOVAOpenMOSS, video generationOviCharacter.AI, video generationKling 2.6Kuaishou, video generation+5 more
LTX-2.5Lightricks, video generationMOVAOpenMOSS, video generationOviCharacter.AI, video generationKling 2.6Kuaishou, video generation+5 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 9
LTX-2.5Lightricks, video generationexemplarVidu Q3 ProShengShu Technology, video generationproduced byMiniMax H3 (Hailuo 3)MiniMax, video generationproduced byVidu Q4 PreviewShengShu Technology, video generationproduced byFLUX 3 VideoBlack Forest Labs, video generationproduced byKling 2.6Kuaishou, video generationexemplarLTX-2.3Lightricks, video generationproduced byMOVAOpenMOSS, video generationexemplarOviCharacter.AI, video generationexemplarPapers
- MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video GenerationLudan Ruan et al., CVPR 2023 (per arXiv comment), 2022 founding
- JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior SynchronizationKai Liu et al., ICLR 2026 (per arXiv comment), 2025 state of the art
- UniVerse-1: Unified Audio-Video Generation via Stitching of ExpertsDuomin Wang et al., arXiv preprint, 2025 state of the art
- Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video GenerationChetwin Low et al., arXiv preprint, 2025 state of the art
- AV-DiT: Efficient Audio-Visual Diffusion Transformer for Joint Audio and Video GenerationKai Wang et al., arXiv preprint, 2024 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2510.01284 https://arxiv.org/abs/2510.01284
- huggingface.co/chetwinlow1/Ovi https://huggingface.co/chetwinlow1/Ovi
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Joint Audio and Video Generation", https://aurion.semai.id/topics/joint-audio-video-generation/, records of October 2026.