Consistent Diffusion Language Models

Hasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song · 2026

Read Consistent Diffusion Language Models by Hasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-00161.txt.

Description

Hasan Amin and 6 co-authors work on diffusion language models, which draft text by refining noisy drafts in parallel. In practice these models still need hundreds of refinement steps for good text, which erases their speed edge. A trick that speeds up image diffusion does not carry over, since text has no smooth path between noise levels.

The team finds the right substitute: an exact formula linking any two noise levels for a draft. Their training method makes a denoising model agree with itself across these links, with no trained teacher needed. The resulting model set new best scores on guided and unguided text generation. Gains were largest when only a few refinement steps were allowed.

Catalog record

CREATORHasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE98 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.00161

Browse the full library · About the Tower of Babel