Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective

Shaorong Zhang, Longxuan Yu, Rob Brekelmans, Luhan Tang, Salman Asif, Greg Ver Steeg · 2026

Read Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective by Shaorong Zhang, Longxuan Yu, Rob Brekelmans, Luhan Tang, Salman Asif, Greg Ver Steeg (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00286.txt.

Description

Shaorong Zhang and team analyze generation order and parallel decoding in masked diffusion models. These models speed inference by trading sequential determinism, yet order and parallelization risks remain open. Their unified frame splits two failure sources: order sensitivity and parallelization bias. Easy first choice, favoring low entropy tokens, helps more as model error grows. Factorized parallel runs add sampling errors that can drive reverse KL divergence very large. That incoherence escapes forward KL measures, and verification removes error only at exponential cost.

Catalog record

CREATORShaorong Zhang, Longxuan Yu, Rob Brekelmans, Luhan Tang, Salman Asif, Greg Ver Steeg
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60345
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.00286

Browse the full library · About the Tower of Babel