Text-to-Music and Song Generation
Generating music, including vocals that follow given lyrics, from text descriptions, melodies or reference audio.
Tier Grades
Tier A: Frontier AI labs
feasible
Google DeepMind, Suno and Meta lead, and YuE needed 58 authors and trillions of tokens.
Tier B: Indonesian AI labs
stretch
Licensed music data at YuE's scale is hard to obtain, though fine-tuning open models such as ACE-Step is possible.
Tier C: Institution units such as ITB
stretch
Units can fine-tune open models that run 4 minutes of music in 20 seconds on one A100, but not train at scale.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Open and API music models can produce licensed background music for content proofs of concept.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeACE-Step generates up to 4 minutes of music in 20 seconds on one A100; training compute is not in the MusicGen, YuE or ACE-Step abstracts.
- DataYuE scales to trillions of tokens and generates up to five minutes of music; MusicGen trains a single-stage LM over compressed music tokens.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 8 authors on MusicGen; 58 on YuE; 5 on ACE-Step.
- Time horizonMonths.
- Shelf-life signalsLicensing settlements limiting training to licensed music, Open models matching Suno on arena boards
- Pipeline roleTier A trains music models; Tier C studies control and evaluation; Tier B adds Indonesian styles; Tier D uses licensed outputs in media pilots.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Suno
Uncharted Labs (Udio)
Google DeepMind
Meta
Multimodal Art Projection (m-a-p)
StepFun
ACE Studio
Stability AI
MiniMax
ElevenLabsLinked models
ACE-Step 1.5music and audioexemplarYuE 1music and audioexemplarMusicGenmusic and audioexemplarSuno v6music and audioexemplarYuE2-3Bmusic and audioproduced byMiniMax Music 3music and audioproduced byMagenta RealTime 2music and audioproduced byStable Audio 3 Mediummusic and audioproduced bySuno v5.5music and audioproduced by
HeartMuLa-oss-3Bmusic and audioproduced bySuno v4.5-allmusic and audioproduced byEleven Musicmusic and audioproduced byUdio Allegromusic and audioproduced by
InspireMusicmusic and audioproduced bySuno v4music and audioproduced byUdio v1.5music and audioproduced bySuno v3music and audioproduced byMAGNeTmusic and audioproduced by
Riffusionmusic and audioproduced by
DDSPmusic and audioproduced byEleven Music v2.5music and audioproduced byLyriamusic and audioproduced byLyria 2music and audioproduced byLyria 3music and audioproduced by+15
Lyria 3.5music and audioproduced byLyria 3 Promusic and audioproduced byLyriamusic and audioproduced byLyria RealTimemusic and audioproduced byMureka V9music and audioproduced byMureka V9.5music and audioproduced by
Pika Musicmusic and audioproduced byStable Audiomusic and audioproduced byStable Audio 2.5music and audioproduced byStable Audio Open 1.0music and audioproduced byACE-Step v1music and audioproduced bySunomusic and audioproduced bySuno v5music and audioproduced bySuno v6-minimusic and audioproduced byUdiomusic and audioproduced byLinks
R&D TopicText-to-Music and Song Generation
ModelsYuE 1Multimodal Art Projection (m-a-p), music and audioMusicGenMeta, music and audioACE-Step 1.5StepFun, music and audioSuno v6Suno, Inc., music and audio+35 more
YuE 1Multimodal Art Projection (m-a-p), music and audioMusicGenMeta, music and audioACE-Step 1.5StepFun, music and audioSuno v6Suno, Inc., music and audio+35 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 39
SunoSuno, Inc., music and audioproduced byLyriaGoogle DeepMind, music and audioproduced byLyria 3Google DeepMind, music and audioproduced byLyria 2Google DeepMind, music and audioproduced byStable AudioStability AI, music and audioproduced byStable Audio 3 MediumStability AI, music and audioproduced bySuno v4Suno, Inc., music and audioproduced bySuno v4.5-allSuno, Inc., music and audioproduced bySuno v5Suno, Inc., music and audioproduced bySuno v5.5Suno, Inc., music and audioproduced bySuno v6Suno, Inc., music and audioexemplarACE-Step 1.5StepFun, ACE Studio, music and audioexemplar+27
ACE-Step v1StepFun, ACE Studio, music and audioproduced byLyriaGoogle DeepMind, music and audioproduced byLyria 3.5Google DeepMind, music and audioproduced byStable Audio 2.5Stability AI, music and audioproduced byStable Audio Open 1.0Stability AI, music and audioproduced bySuno v3Suno, Inc., music and audioproduced byUdioUncharted Labs (Udio), music and audioproduced byUdio AllegroUncharted Labs (Udio), music and audioproduced byUdio v1.5Uncharted Labs (Udio), music and audioproduced byLyria 3 ProGoogle DeepMind, music and audioproduced byLyria RealTimeGoogle DeepMind, music and audioproduced byMiniMax Music 3MiniMax, music and audioproduced bySuno v6-miniSuno, Inc., music and audioproduced byYuE 1Multimodal Art Projection (m-a-p), music and audioexemplarYuE2-3BMultimodal Art Projection (m-a-p), music and audioproduced byDDSPGoogle, music and audioproduced byEleven MusicElevenLabs, music and audioproduced byEleven Music v2.5ElevenLabs, music and audioproduced byHeartMuLa-oss-3BHeartMuLa, music and audioproduced byInspireMusicFunAudioLLM, music and audioproduced byMagenta RealTime 2Google DeepMind, music and audioproduced byMAGNeTMeta, music and audioproduced byMusicGenMeta, music and audioexemplarPika MusicPika (Pika Labs), music and audioproduced byRiffusionRiffusion, music and audioproduced byMureka V9music and audioproduced byMureka V9.5music and audioproduced byPapers
- Jukebox: A Generative Model for MusicPrafulla Dhariwal et al., arXiv preprint, 2020 founding
- YuE: Scaling Open Foundation Models for Long-Form Music GenerationRuibin Yuan et al., arXiv preprint, 2025 state of the art
- Fast Timing-Conditioned Latent Audio DiffusionZach Evans et al., ICML 2024 (per arXiv comment), 2024 state of the art
- MusicLM: Generating Music From TextAndrea Agostinelli et al., arXiv preprint, 2023 state of the art
- Foundation Models for Music: A SurveyYinghao Ma et al., arXiv preprint, 2024 survey
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2306.05284 https://arxiv.org/abs/2306.05284
- arxiv.org/abs/2503.08638 https://arxiv.org/abs/2503.08638
- arxiv.org/abs/2506.00045 https://arxiv.org/abs/2506.00045
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Text-to-Music and Song Generation", https://aurion.semai.id/topics/text-to-music-generation/, records of October 2026.