Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space
Yangyi Shen, Tianjian Feng, Jiaqi Han, Wen Wang, Tianlang Chen, Chunhua Shen, Jure Leskovec, Stefano Ermon · 2026
Read Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space by Yangyi Shen, Tianjian Feng, Jiaqi Han, Wen Wang, Tianlang Chen, Chunhua Shen, Jure Leskovec, Stefano Ermon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20339.txt.
Description
Yangyi Shen and seven colleagues improve how diffusion language models generate text. These models can build text in any order, which opens many possible paths to a finished sentence. Current decoding picks a single path and commits to it, so much of that space goes unexplored.
The authors introduce Order-Token Search, which searches jointly over the order of generation and the token values chosen. Exploring both dimensions at once should find better outputs than fixing the order in advance. Stronger decoding would help diffusion models compete with standard left-to-right models on quality.
Catalog record
| CREATOR | Yangyi Shen, Tianjian Feng, Jiaqi Han, Wen Wang, Tianlang Chen, Chunhua Shen, Jure Leskovec, Stefano Ermon |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-20339