Thinking Out of Order: When Output Order Stops Reflecting Reasoning Order in Diffusion Language Models

Longxuan Yu, Yu Fu, Shaorong Zhang, Hui Liu, Mukund Varma T, Greg Ver Steeg, Yue Dong · 2026

Read Thinking Out of Order: When Output Order Stops Reflecting Reasoning Order in Diffusion Language Models by Longxuan Yu, Yu Fu, Shaorong Zhang, Hui Liu, Mukund Varma T, Greg Ver Steeg, Yue Dong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22035.txt.

Description

Longxuan Yu and co-authors study a freedom that diffusion language models have and ordinary ones lack. Autoregressive models must write left to right, in a fixed order. That hurts when the required output order clashes with natural reasoning, for instance when a format demands the answer before the explanation.

An autoregressive model must then commit to an answer before reasoning through it. Diffusion models generate in a different order, which could fit reasoning better. The team's 2026 preprint asks when output order stops reflecting reasoning order in these models. The answer shapes how structured outputs, from forms to code, should be generated when the reasoning must come first but the page layout says otherwise.

Catalog record

CREATORLongxuan Yu, Yu Fu, Shaorong Zhang, Hui Liu, Mukund Varma T, Greg Ver Steeg, Yue Dong
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22035

Browse the full library · About the Tower of Babel