Consistent Diffusion Language Models
Hasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song · 2026
Read Consistent Diffusion Language Models by Hasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-00161.txt.
Description
Hasan Amin and 6 co-authors work on diffusion language models, which draft text by refining noisy drafts in parallel. In practice these models still need hundreds of refinement steps for good text, which erases their speed edge. A trick that speeds up image diffusion does not carry over, since text has no smooth path between noise levels.
The team finds the right substitute: an exact formula linking any two noise levels for a draft. Their training method makes a denoising model agree with itself across these links, with no trained teacher needed. The resulting model set new best scores on guided and unguided text generation. Gains were largest when only a few refinement steps were allowed.
Catalog record
| CREATOR | Hasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 98 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.00161