A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models
Theo X. Olausson, Metod Jazbec, Xi Wang, Armando Solar-Lezama, Christian A. Naesseth, Stephan Mandt, Eric Nalisnick · 2026
Read A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models by Theo X. Olausson, Metod Jazbec, Xi Wang, Armando Solar-Lezama, Christian A. Naesseth, Stephan Mandt, Eric Nalisnick (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-09921.txt.
Introduction
Olausson and six co-authors improve diversity in diffusion language model sampling, beyond the usual speed-quality tradeoff for single samples. They soften familiar confidence-based remasking heuristics, keeping their speed and simple implementation, and motivate the change with an idealized model of fork tokens and remasking's effect on entropy. The tempered heuristics close the exploration gap between confidence-based and autoregressive sampling, beating both at equal cost. The added diversity also carries into downstream post-training and test-time compute scaling, showing simple, efficient, diverse sampling is possible.
Catalog record
| CREATOR | Theo X. Olausson, Metod Jazbec, Xi Wang, Armando Solar-Lezama, Christian A. Naesseth, Stephan Mandt, Eric Nalisnick |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57580 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.09921
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models Theo X. Olausson, Metod Jazbec, Xi Wang, Armando Solar-Lezama, Christian A. Naesseth, Stephan Mandt, Eric Nalisnick Abstract Much work has been done on designing fast and accurate sampling for diffusion language models (dLLMs). However, these efforts have largely focused on the tradeoff between speed and quality of individual samples; how to additionally ensure diversity across samples remains less well understood. In this work, we show that diversity can be increased by using softened, tempered versions of familiar confidence-based remasking heuristics, retaining their computational benefits and offering simple implementations. We motivate this approach by introducing an idealized formal model of fork tokens and studying the impact of remasking on the expected entropy at the forks. Empirically, the proposed tempered heuristics close the exploration gap (pass@ k k ) between existing confidence-based and autoregressive sampling, hence outperforming both when controlling for cost (pass@NFE). We further study how the increase in diversity translates to downstream post-training and test-time compute scaling. Overall, our findings demonstrate that simple, efficient, and diverse sampling from dLLMs is possible. 1 Introduction Diffusion language models (dLLMs) have recently seen significant uptake, offering native bidirectional context and the ability to generate multiple tokens in parallel ( 29 ; 47 ; 46 ; 5 ; inter alia ). Unlike in the autoregressive (AR) setting, dLLMs require a remasking strategy that determines the order in which tokens are sampled. Because this choice significantly affects sample quality, much prior work has focused on optimizing the speed–quality tradeoff for individual samples ( 8 ; 29 ; 20 ; 4 ; 19 ; inter alia ). While this line of work has helped make dLLMs more competitive with AR LLMs, efficiently obtaining a single high-quality sample is not always sufficient: post-training, test-time compute scaling, and A/B testing all require diversity across samples in order to be effective. Surprisingly, 27 found that confidence-based heuristics yielding high single-sample accuracy lead to worse diversity than left-to-right autoregressive generation. By delaying uncertain tokens, such heuristics avoid genuine branching points in the reasoning process. Autoregressive generation does not suffer from this problem, but comes at a significant cost as it forgoes parallel generation of multiple tokens altogether. In this work, we argue that the diversity problem can be addressed with a simple fix, without abandoning parallel generation or adopting complex sampling schemes. Our key observation is that it is the greedy nature of existing heuristics, not parallel generation in itself, that limits diversity. By replacing them with softened, tempered approximations thereof—for example, sampling positions proportional to their tempered confidence instead of selecting them greedily based on the highest confidence—we demonstrate that the diversity can be recovered while preserving speed and simplicity. In summary, our contributions are as follows: • We introduce Tempered Low-Confidence (TLC) and Tempered Confidence Thresholding (TCT) remasking, stochastic relaxations of two widely used remasking heuristics ( 29 ; 44 ) controlled by a position temperature T_{\text{pos}} ( Section 3.2 ). This provides a knob complementary to the standard token-level temperature T_{\text{token}} : T_{\text{token}} governs the entropy of what token is predicted at each position, while T_{\text{pos}} governs how stochastic the order in which positions are committed to is. • We give an operational definition of the fork tokens , high-entropy positions that prior work has empirically found to be important for effective post-training, and show that, under an idealized model, increasing T_{\text{pos}} provably increases the expected entropy at these forks for TLC ( Section 3.1 ). • We empirically validate that TLC recovers pass@ k scaling comparable to AR rollouts across multiple benchmarks and models, confirming that tempering the remasking strategy is sufficient to restore diversity ( Section 4.1 , Figure 2 ). TCT goes further by retaining the adaptive speed of confidence thresholding; when the cost of each rollout is accounted for, TCT often scales better than either untempered heuristics or AR generation ( Section 4.1 , Figure 3 ). • We study how pass@k improvements translate downstream: when used for GRPO rollout generation in dLLM post-training ( 51 ) at matched computational budgets, TCT yields stronger policies than AR generation ( Section 4.3 ). In the context of test-time compute, we find that while the increased diversity of TCT can overwhelm simple strategies such as majority voting, this gap is largely closed when using outcome reward models for answer selection ( Section 4.2 ). 2 Background We use the notation [L] to represent the set \{1,\ldots,L\} . We denote a sequence of tokens by \bm{x}=(x_{1},\ldots,x_{L})\in\mathcal{V}^{L} , where \mathcal{V}:=[V] is the vocabulary and L is the sequence length. 2.1 From Masked Diffusion Models to Diffusion Language Models Masked Diffusion Models (MDMs) are a form of absorbing discrete diffusion ( 37 ; 2 ) in which the target (noise) distribution places all its mass on a single token, [MASK] , and the forward process gradually corrupts each token towards the absorbing mask state by transitioning tokens independently through a Markov chain. Formally, in the continuous-time formulation of 7 , this can be characterized compactly as follows: \displaystyle\bm{x}_{0}\sim p_{\text{data}}\qquad\bm{x}_{1}\sim\delta_{\texttt{[MASK]}}^{\otimes L} (1) \displaystyle\bm{x_{t}}\mid\bm{x}_{0}\sim p_{t}(\bm{x}_{t}\mid\bm{x}_{0})\triangleq\prod_{l=1}^{L}\alpha(t)\bm{1}\left(x_{t}^{l}=\texttt{[MASK]}\right)+(1-\alpha(t))\bm{1}\left(x_{t}^{l}=x_{0}^{l}\right) (2) where 0\leq t\leq 1 denotes the time, \alpha:t\rightarrow[0,1] is a monotonically non-decreasing function in t with endpoints \alpha(0)=0,\alpha(1)=1 , and \delta^{\otimes L}_{\texttt{[MASK]}} is a product of L Dirac measures placing all of their mass on [MASK] . The generative (i.e., reverse) process can be modeled either by learning to predict the time-dependent reverse transition ratios ( 7 ; 26 ; 25 ) , or by training a model q_{\theta} to directly approximate the conditional distribution p_{0\mid t} . The latter strategy has typically been the choice of language-modeling practitioners, as later work ( 34 ; 31 ) showed that its evidence lower bound (ELBO) simplifies to: -\mathcal{L}(\theta)\triangleq\mathbb{E}_{t\sim U[0,1],\bm{x}_{0}\sim p_{\text{data}},\bm{x}_{t}\sim p_{t}(\cdot\mid\bm{x}_{0})}\left[w(t)\sum_{k=1}^{L}\bm{1}\left(x_{t}^{k}=\texttt{[MASK]}\right)\log q_{\theta}^{k}(x_{0}^{k}\mid\bm{x}_{t})\right]\ (3) where w(\cdot) is a weighting term determined by the noise schedule \alpha(\cdot) , and q_{\theta}^{k}(\cdot|\bm{x}_{t}) is the model’s predicted marginal distribution over possible tokens at the k -th position. This formulation eliminates explicit conditioning on t , connecting MDM training to masked language modeling ( 14 ; 16 ) . Combined with empirically validated scaling laws for MDMs ( 28 ; 5 ) , this has enabled the open-source community to train multi-billion parameter MDMs on textual data, which we refer to as diffusion (large) language models (dLLMs). 2.2 Remasking strategies and sampling from dLLMs Early work on MDMs obtained samples by discretizing the reverse process and simulating the Markov chain ( 7 ; 2 ; 25 ) . As observed by 52 , this is potentially wasteful as each individual token transitions exactly once in Equation 2 . They instead proposed a first-hit sampling scheme, which when the time-agnostic objective of Equation 3 is used simplifies to what 29 call random remasking . Concretely, denoting the current generation as \bm{x}_{t}\in\mathcal{V}^{L} with still-masked positions \mathcal{M}_{t}:=\{k\in[L]\mid x_{t}^{k}=\texttt{[MASK]}\} , the next generation \bm{x}_{t^{\prime}} is obtained via \displaystyle x_{t^{\prime}}^{k}:=\begin{cases}x\sim q_{\theta}^{k}(\cdot\mid\bm{x}_{t}\>;T_{\text{token}}),&\text{if }k\in\mathcal{U}_{t}^{K},\\ x_{t}^{k}\;,&\text{otherwise.}\end{cases} where the unmasking set \mathcal{U}_{t}^{K} of K\geq 1 positions is sampled uniformly at random (without replacement) from \mathcal{M}_{t} . As in the autoregressive setting, T_{\text{token}} tempers the final softmax of q_{\theta}^{k} : intuitively, higher temperatures yield more diverse (but possibly lower quality) generations. This process continues until reaching a