Project AURIONAI Use-Cases Research & Innovation
University of Illinois Urbana-Champaign
music and audio

MMAudio

Video-to-audio synthesis model by Ho Kei Cheng and co-authors, trained jointly on video and text-audio data.

University of Illinois Urbana-ChampaignUniversity of Illinois Urbana-ChampaignUSASonySonyJapan
Figure from the MMAudio paper: In addition to training on audio-visual-(text) datasets, we perform multimodal joint training with high-quality, abundant audio-text data which enables effectiv.
Levelmodel
Classmusic and audio
Parametersnot in the notes
Contextnot in the notes
Released19 Dec 2024
Opennessopen weights restricted
Runs onnot in the notes
Statusactive

R&D Topics

Video-to-audio and Foley generationexemplar

Identity

Developer
University of Illinois Urbana-Champaign, Sony
Family
not in the notes
Released
19 Dec 2024 sourcesannouncement page
Weights published
–
Record checked
12 Oct 2026
Where this comes from
Developer and dates from the research notes. Release date is the vendor's public announcement; weights date is when open weights were published.

Architecture and Size

Architecture
other
Total parameters
– sources
Active parameters
–
Checkpoint parameters
–
Context length
not in the notes
Input
video, text
Output
audio
Where this comes from
Parameter counts as the developer states them. Where the checkpoint holds a different total, both are shown (register A16).

Openness and Access

Openness
open weights restricted
Licence
Creative Commons Attribution-NonCommercial 4.0 sources
Access routes
open weights download
Where this comes from
The licence file governs over registry tags and aggregator labels.

Deployment

Runs on
not in the notes
Footprint
not in the notes
Runtimes
not in the notes
Where this comes from
Deployment tiers and runtimes as named in the notes. Footprint is the notes' figure, not a measurement by AURION.

Indonesia

Indonesian support
not stated
Regional languages
not in the notes
Jakarta hosting
unknown
Languages
not in the notes
Where this comes from
Indonesia fields are carried on every model; unknown is an allowed value.

Benchmark Results

The notes record no benchmark result for this model.

Lineage

The notes record no lineage for this model.

Industry Use Cases

No industry use case links this model.

Sources

  1. arxiv.org/abs/2412.15322 https://arxiv.org/abs/2412.15322
  2. huggingface.co/hkchengrex/MMAudio https://huggingface.co/hkchengrex/MMAudio
  3. arxiv.org/html/2412.15322 https://arxiv.org/html/2412.15322

To cite this page: Project AURION, AI Use-Cases Research & Innovation, "MMAudio", https://aurion.semai.id/models/mmaudio/, records of October 2026.