Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space

Yangyi Shen, Tianjian Feng, Jiaqi Han, Wen Wang, Tianlang Chen, Chunhua Shen, Jure Leskovec, Stefano Ermon · 2026

Read Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space by Yangyi Shen, Tianjian Feng, Jiaqi Han, Wen Wang, Tianlang Chen, Chunhua Shen, Jure Leskovec, Stefano Ermon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20339.txt.

Description

Yangyi Shen and seven colleagues improve how diffusion language models generate text. These models can build text in any order, which opens many possible paths to a finished sentence. Current decoding picks a single path and commits to it, so much of that space goes unexplored.

The authors introduce Order-Token Search, which searches jointly over the order of generation and the token values chosen. Exploring both dimensions at once should find better outputs than fixing the order in advance. Stronger decoding would help diffusion models compete with standard left-to-right models on quality.

Catalog record

CREATORYangyi Shen, Tianjian Feng, Jiaqi Han, Wen Wang, Tianlang Chen, Chunhua Shen, Jure Leskovec, Stefano Ermon
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-20339

Browse the full library · About the Tower of Babel