Adaptation to Intrinsic Dependence in Diffusion Language Models

Yunxiao Zhao, Changxiao Cai · 2026

Read Adaptation to Intrinsic Dependence in Diffusion Language Models by Yunxiao Zhao, Changxiao Cai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-20126.txt.

Description

Diffusion language models generate tokens at once rather than strictly left to right. Theory has said little about how reveal schedules affect generation quality. Those schedules set the order and size of tokens revealed during sampling. A 2026 arXiv paper by Yunxiao Zhao and Changxiao Cai introduces a distribution-agnostic unmasking schedule. It adapts to the target data's dependence structure, which is unknown in advance.

Unlike earlier methods that fix reveal sizes in advance, the new method varies how many tokens are revealed at each iteration. It needs no prior knowledge or tuning knobs. For two specific parameter choices, the authors show sampling convergence bounds measured by Kullback-Leibler divergence. Those bounds scale with total correlation and dual total correlation divided by the number of iterations. Readers gain a schedule that adapts to built-in dependence without knowing it in advance.

Catalog record

CREATORYunxiao Zhao, Changxiao Cai
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE138 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-20126

Browse the full library · About the Tower of Babel