Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Models
Jiawei Xu, Minghui Liu, Aakriti Agrawal, Yifan Chen, Furong Huang · 2026
Read Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Models by Jiawei Xu, Minghui Liu, Aakriti Agrawal, Yifan Chen, Furong Huang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-23567.txt.
Introduction
Xu and colleagues learn the best order in which a diffusion language model should reveal its own words, in a 2026 machine learning paper. Masked diffusion models generate text by unmasking hidden tokens step by step, in any order. That order shapes which decisions come early and which stay open. Most systems pick it with simple confidence rules that favor easy tokens and can steer the answer badly.
The team derives a bound on decoding mismatch. It turns order choice into a policy learning problem with a reward per step. Their method, Self-Aware Scheduling, trains a light policy while the main model stays frozen. On Sudoku puzzles, accuracy jumps from about eighty-two percent under the best heuristic schedule to about ninety-two percent, and higher after a second tuning stage. Gains also appear on grade-school mathematics and code tasks with a large diffusion model.
Catalog record
| CREATOR | Jiawei Xu, Minghui Liu, Aakriti Agrawal, Yifan Chen, Furong Huang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 98 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.23567
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Models Jiawei Xu, Minghui Liu, Aakriti Agrawal, Yifan Chen, Furong Huang Abstract Masked diffusion language models decode by iteratively unmasking tokens, where the unmasking order defines an “order of thought” that strongly influences generation quality yet is typically chosen heuristically. We derive a tractable upper bound on the sequential decoding mismatch, measured by the Kullback–Leibler divergence and expressed in terms of the model’s pathwise log-likelihood, with tightness under sufficient model expressivity. This bound induces a dense self-aware reward over ordered trajectories, casting order selection as a principled policy optimization problem with a frozen denoiser. We instantiate this idea as Self-Aware Scheduling (SAS) , which learns a lightweight order policy using Group Relative Policy Optimization and applies seamlessly to both any-order and semi-autoregressive decoding. On Sudoku with 1B MDM, SAS improves puzzle accuracy from 82.0 % 82.0\% (best heuristic schedule) to 91.8 % 91.8\% , and reaches 97.5 % 97.5\% with second-stage fine-tuning along learned trajectories. On mathematical reasoning with LLaDA-8B, SAS improves pass@1 on GSM8K from 64 % 64\% to 76 % 76\% and on MBPP from 39.5 % 39.5\% to 41 % 41\% , consistently matching or exceeding heuristic schedules across generation lengths and block sizes. Project page: https://jimmyxu123.github.io/SAS . † † affiliationnotice: Equal advising 1 Introduction Diffusion decoding has a hidden degree of freedom: the order of thought . Masked diffusion language models ( Austin et al. 2021 ; Lou et al. 2023 ; Hoogeboom et al. 2021b ; Shi et al. 2024 ) generate discrete sequences by iteratively unmasking tokens, offering a flexible alternative to the fixed left-to-right factorization of autoregressive language models ( Radford et al. 2018 ) . Crucially, diffusion decoding is not tied to a single generation order: inference depends on an unmasking schedule that decides which positions are revealed at each step. This schedule is more than an implementation detail: it is an implicit “order of thought”, and it determines which commitments the model makes early, which constraints it propagates, and which ambiguities it strategically postpones. Because different schedules trace different conditional trajectories, they induce different output distributions, and the schedule choice can substantially affect generation quality ( Kim et al. 2025 ; Zheng et al. 2023 ) . Heuristic schedules are fast, but they can think myopically. Most existing schedules are fixed greedy heuristics based on per-position uncertainty (e.g., confidence, margin, entropy) ( Zheng et al. 2023 ; Kim et al. 2025 ; Ben-Hamu et al. 2025 ) . While efficient, such rules optimize the next step rather than the whole decoding trajectory: they often pick “easy” tokens that become locally certain under the current partial context, even when a different reveal would better shape future conditionals. Empirically, we find that these myopic choices underperform on complex reasoning tasks (see Table 1 and Figure 2 in Section 6.1 ). Even worse, expert-designed logical orders can be suboptimal for an imperfect pretrained diffusion model: what is logically natural for humans is not necessarily the path along which the model’s conditional predictions are most reliable (see Section 6.1 ). These observations motivate a principled approach that learns the order, instead of hard-coding it. Key idea and theory: learn the schedule by optimizing the model’s trajectory likelihood . We treat diffusion decoding as a latent-order generative process ( Huang et al. 2025 ; Wang et al. 2025d ) and ask a direct question: which reveal order makes the model most faithful to the data distribution? Concretely, we formalize scheduling as minimizing the KL divergence between the data distribution and the distribution induced by a given decoding procedure. Our main theoretical contribution is to show that this intractable objective admits a tractable bound expressed in terms of the denoiser’s pathwise (trajectory) log-likelihood under teacher forcing. This yields a dense, model-self-aware reward that assigns credit to each reveal decision, rather than only the final sample. Under this reward, a good schedule reveals the token maximizing the expected future log-likelihood of the remaining trajectory, i.e., it chooses the next “thought” to make downstream reasoning easiest for the model, aligning the decoding path with the model’s predictive strengths rather than with local uncertainty alone. Method: Self-Aware Scheduling (SAS) for plug-and-play order learning. Guided by the theory, we propose Self-Aware Scheduling (SAS) , a lightweight framework that learns an order policy while keeping the diffusion model fixed. At each decoding step, the policy outputs a distribution over currently masked positions and selects (or samples) one position to reveal next. SAS is plug-and-play (no retraining of the base denoiser), and it can be optimized efficiently with Group Relative Policy Optimization (GRPO) ( Shao et al. 2024 ) . The same learned policy applies seamlessly to both any-order sequential decoding and semi-autoregressive (block) decoding ( Arriola et al. 2025 ) , turning “how to unmask” into a trainable component of inference. Results: optimizing the order of thought is a strong lever for reasoning. Across diverse reasoning benchmarks, SAS yields consistent and substantial improvements over heuristic and expert-designed schedules. On large-scale Sudoku with a 1B masked diffusion model, SAS improves puzzle accuracy from 82.0\% (best heuristic) to 91.8\% . On mathematical reasoning (GSM8K) with LLaDA-8B ( Nie et al. 2025b ) , it improves pass@1 from 64\% to 76\% . We further show that a simple second-stage fine-tuning procedure along the learned trajectories using our self-aware objective provides additional gains. Together, these results position decoding order as a controllable and optimizable reasoning primitive: beyond learning what to generate, we can learn when to commit to each piece of information—in effect, learning an order of thinking that the model itself finds most reliable. Summary of Contributions. • Theory (self-aware objective for order learning): We cast diffusion decoding as a latent-order generative process and derive a tractable bound linking decoding mismatch (KL divergence) to the denoiser’s cumulative pathwise log-likelihood, yielding a dense, model-self-aware reward for credit assignment over reveal decisions. • Algorithm (SAS): We introduce Self-Aware Scheduling , a plug-and-play framework that learns an unmasking-order policy for a frozen diffusion LM via GRPO, applicable to both any-order and semi-autoregressive decoding. • Empirical validation (reasoning gains + second stage): We demonstrate large, consistent improvements over heuristic and expert schedules on Sudoku, math and coding reasoning, and show additional gains from second-stage fine-tuning along learned trajectories under the same self-aware objective. 2 Preliminaries and Problem Setup 2.1 Masked Diffusion Models Masked diffusion models generate discrete sequences by iteratively demasking tokens, analogous to the denoising process in continuous diffusion models ( Song et al. 2020 ; Ho et al. 2020 ) . Let \mathcal{X} denote a finite vocabulary and n the sequence length. A sequence is x=(x_{1},\ldots,x_{n})\in\mathcal{X}^{n} . For a mask pattern M\subseteq[n]:=\{1,\ldots,n\} , define the masked sequence x^{M}\in(\mathcal{X}\cup\{\texttt{[MASK]}\})^{n} by (x^{M})_{i}:=\begin{cases}\texttt{[MASK]}&\text{if }i\in M,\\ x_{i}&\text{if }i\notin M.\end{cases} (1) A masked diffusion model (or denoiser ) parameterized by \theta defines conditional distributions over masked tokens given observed context. Specifically, the model p_{\theta} provides position-wise predictions p_{\theta}^{i}(\cdot\mid x^{M}),\ i\in M, representing the distribution over token x_{i} conditioned on the partial observation x^{M} . Here \theta denotes the learned parameters of the base model. 2.2 Unmasking Schedules: the Order of Thought Generation proceeds by iteratively unmasking positions, a process known as decoding