Unifying Masked Diffusion Models with Various Generation Orders and Beyond

Chunsan Hong, Sanghyun Lee, Jong Chul Ye · 2026

Read Unifying Masked Diffusion Models with Various Generation Orders and Beyond by Chunsan Hong, Sanghyun Lee, Jong Chul Ye (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02112.txt.

Description

Chunsan Hong and colleagues present order-expressive masked diffusion model (OeMDM) for varied orders. OeMDM unifies MDM, ARM, and block diffusion. Masked diffusion models rival autoregressive models, but quality hinges on sequence. Prior work either hard-codes an ordering or learns one. That path costs extra and can yield weak solutions. Building on OeMDM, they introduce learnable-order masked diffusion model (LoMDM), learning ordering and backbone. LoMDM thus generates text in context order. LoMDM outperforms discrete diffusion models across language modeling benchmarks.

Catalog record

CREATORChunsan Hong, Sanghyun Lee, Jong Chul Ye
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE185228
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.02112

Browse the full library · About the Tower of Babel