Adaptation to Intrinsic Dependence in Diffusion Language Models
Yunxiao Zhao, Changxiao Cai · 2026
Read Adaptation to Intrinsic Dependence in Diffusion Language Models by Yunxiao Zhao, Changxiao Cai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-20126.txt.
Description
Diffusion language models generate tokens at once rather than strictly left to right. Theory has said little about how reveal schedules affect generation quality. Those schedules set the order and size of tokens revealed during sampling. A 2026 arXiv paper by Yunxiao Zhao and Changxiao Cai introduces a distribution-agnostic unmasking schedule. It adapts to the target data's dependence structure, which is unknown in advance.
Unlike earlier methods that fix reveal sizes in advance, the new method varies how many tokens are revealed at each iteration. It needs no prior knowledge or tuning knobs. For two specific parameter choices, the authors show sampling convergence bounds measured by Kullback-Leibler divergence. Those bounds scale with total correlation and dual total correlation divided by the number of iterations. Readers gain a schedule that adapts to built-in dependence without knowing it in advance.
Catalog record
| CREATOR | Yunxiao Zhao, Changxiao Cai |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 138 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-20126