The Diffusion Duality, Chapter II: Ψ \Psi -Samplers and Efficient Curriculum

Justin Deschenaux, Caglar Gulcehre, Subham Sekhar Sahoo · 2026

Read The Diffusion Duality, Chapter II: Ψ \Psi -Samplers and Efficient Curriculum by Justin Deschenaux, Caglar Gulcehre, Subham Sekhar Sahoo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-21185.txt.

Description

Uniform-state discrete diffusion models generate well in few steps but stall as sampling steps grow. Justin Deschenaux, Caglar Gulcehre, and Subham Sekhar Sahoo break that plateau in this 2026 arXiv paper with predictor-corrector samplers for discrete diffusion, generalizing prior methods to arbitrary noise processes. Paired with uniform-state diffusion, the samplers beat ancestral sampling on language and image modeling.

Unlike conventional samplers, the new methods keep improving with more steps, challenging the assumption that masked diffusion is the inevitable future for diffusion language models. Beyond sampling, a memory-efficient curriculum for the Gaussian relaxation training phase cuts training time by a quarter and memory by a third against a prior method, with comparable perplexity. Code, checkpoints, and a video tutorial are released.

Catalog record

CREATORJustin Deschenaux, Caglar Gulcehre, Subham Sekhar Sahoo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE144 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-21185

Browse the full library · About the Tower of Babel