Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective
Shaorong Zhang, Longxuan Yu, Rob Brekelmans, Luhan Tang, Salman Asif, Greg Ver Steeg · 2026
Read Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective by Shaorong Zhang, Longxuan Yu, Rob Brekelmans, Luhan Tang, Salman Asif, Greg Ver Steeg (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00286.txt.
Description
Shaorong Zhang and team analyze generation order and parallel decoding in masked diffusion models. These models speed inference by trading sequential determinism, yet order and parallelization risks remain open. Their unified frame splits two failure sources: order sensitivity and parallelization bias. Easy first choice, favoring low entropy tokens, helps more as model error grows. Factorized parallel runs add sampling errors that can drive reverse KL divergence very large. That incoherence escapes forward KL measures, and verification removes error only at exponential cost.
Catalog record
| CREATOR | Shaorong Zhang, Longxuan Yu, Rob Brekelmans, Luhan Tang, Salman Asif, Greg Ver Steeg |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60345 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.00286