Diffusion Language Models
Language models that generate many tokens in parallel by iterative denoising instead of one token at a time.
Tier Grades
Tier A: Frontier AI labs
feasible
Inception, Google and Apple train diffusion LLMs.
Tier B: Indonesian AI labs
stretch
Training a 7B model on 130B tokens (DiffuCoder) is a stretch; Indonesian work is not in the sources.
Tier C: Institution units such as ITB
stretch
Studying released diffusion LLMs is institution-scale; training one is a stretch.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Diffusion LLMs are reachable through APIs and open checkpoints where latency matters; delivery evidence is not in the sources.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeMercury Coder Mini and Small reach 1,109 and 737 tokens per second on NVIDIA H100 GPUs. DiffuCoder: 7B trained on 130B tokens of code.
- DataDiffuCoder: 130B tokens of code; Mercury: code benchmarks and Copilot Arena ranking.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 13 authors on Mercury; 7 on DiffuCoder.
- Time horizonMonths to a year.
- Shelf-life signalsCommercial diffusion LLMs (Inception Mercury 2) and open ones (DiffusionGemma, DiffuCoder), Throughput claims of up to 10 times speed-optimised autoregressive models (Mercury)
- Pipeline roleTier A trains diffusion LLMs; Tier C studies their decoding; Tier D tests them where low latency matters.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Links
R&D TopicDiffusion Language Models
Models
MercuryInception (Inception Labs), LLM
DiffuCoder 7B InstructApple, code
DiffusionGemma 26B A4BGoogle, LLMMercury 2Inception (Inception Labs), reasoning+1 more
MercuryInception (Inception Labs), LLMDiffuCoder 7B InstructApple, codeDiffusionGemma 26B A4BGoogle, LLMMercury 2Inception (Inception Labs), reasoning+1 moreUse Casesnone recorded
Who Runs Itnone recorded
Papers
- Structured Denoising Diffusion Models in Discrete State-SpacesJacob Austin et al., arXiv preprint, 2021 founding
- Large Language Diffusion ModelsShen Nie et al., arXiv preprint, 2025 state of the art
- Mercury: Ultra-Fast Language Models Based on DiffusionInception Labs et al., arXiv preprint, 2025 state of the art
- Dream 7B: Diffusion Large Language ModelsJiacheng Ye et al., arXiv preprint, 2025 state of the art
- A Survey on Diffusion Language ModelsTianyi Li et al., arXiv preprint, 2025 survey
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2506.17298 https://arxiv.org/abs/2506.17298
- arxiv.org/abs/2506.20639 https://arxiv.org/abs/2506.20639
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Diffusion Language Models", https://aurion.semai.id/topics/diffusion-language-models/, records of October 2026.