Revise, Don’t Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models

Longxuan Yu, Shaorong Zhang, Yu Fu, Hui Liu, Yue Dong, Greg Ver Steeg · 2026

Read Revise, Don’t Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models by Longxuan Yu, Shaorong Zhang, Yu Fu, Hui Liu, Yue Dong, Greg Ver Steeg (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01026.txt.

Introduction

Yu and co-authors let a diffusion language model revise its own words in this 2026 paper. Masked diffusion models re-predict every slot at each step. Standard samplers, though, lock each token once shown. That wastes the model's built in power to revise. Past fixes add bolt on parts or re-mask tokens for a fresh guess.

Their sampler, D3IM, revises visible tokens head on, with no added parts and no extra passes. It is derived as a corrector style update in reverse. The work also names a flaw: preservation bias. The model tends to keep its own wrong tokens rather than fix them. Their training fix, SCOPE, mimics the sampler during post training. Scores rise by 13.0 points on GSM8K and by 15.3 points on HumanEval in the reported tests. Gains grow as more denoising steps are used, they add.

Catalog record

CREATORLongxuan Yu, Shaorong Zhang, Yu Fu, Hui Liu, Yue Dong, Greg Ver Steeg
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.01026

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Revise, Don’t Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models Longxuan Yu, Shaorong Zhang, Yu Fu, Hui Liu, Yue Dong, Greg Ver Steeg Abstract Masked diffusion language models (MDLMs) re-predict every position at each denoising step, but standard samplers commit tokens once revealed, leaving this revision capability unused. Existing approaches either add heuristic or learned mechanisms to revise committed tokens, or remask them back to [MASK] before re-predicting; a principled sampler that directly revises visible tokens without auxiliary modules remains underexplored. We introduce D3IM , a parameter-free sampler derived as a corrector-style reverse update that permits direct visible-to-visible revision without additional modules or auxiliary passes. D3IM also reveals a model-side obstacle we term preservation bias : the model tends to reproduce its own wrong committed tokens rather than correct them. We address this with SCOPE (Self-Conditioned On Prediction Errors), a lightweight post-training procedure that simulates D3IM’s sampling process. On LLaDA-8B at 64 denoising steps, SCOPE+D3IM improves over the original LLaDA-8B with standard unmasking by +13.0 on GSM8K (68.3%), +4.8 on MATH-500 (23.6%), +15.3 on HumanEval (29.3%), and +10.4 on MBPP (30.8%), with gains that increase as more denoising steps are used on math and HumanEval. 1 Introduction Masked diffusion language models (MDLMs) generate text by iteratively denoising masked sequences, re-predicting every position at each step 1 ; 20 ; 23 ; 22 . This all-position prediction naturally creates a revision capability: even after a token becomes visible, the denoiser can assign a different prediction to that position at a later step. However, standard unmasking-based MDLM decoding freezes each token once it is committed, updating only masked positions and discarding later predictions at visible positions. Early mistakes therefore become permanent, a cost especially severe on reasoning-heavy tasks such as math and code generation, where a single wrong digit or operator can constrain subsequent predictions 16 . Figure 1: Overview of D3IM inference and SCOPE training. (A) Standard monotonic unmasking freezes tokens once committed; early high-confidence errors remain fixed. (B) D3IM re-predicts all positions at each step and keeps only the top- K by confidence. A low-confidence token is demoted to [MASK] (token \to mask); a high-confidence new prediction directly overwrites the old token (token \to token). (C) SCOPE training simulates this process: the model self-predicts on masked input (step 2), and the top- \rho predictions by confidence are committed as set C while the rest remain masked as set M (step 3). The loss is computed on both C and M positions against ground-truth targets (step 4), training token-to-token and mask-to-token correction jointly. Existing revision approaches fall short in two complementary ways. The closest principled remask sampler directly applicable to pretrained MDLMs, ReMDM 28 , is derived under the absorbing assumption and can only revise a committed token by first returning it to [MASK]. Practical revision methods bypass this remasking route but add heuristic confidence thresholds 3 ; 29 , verification passes 12 ; 26 , or learned modules 14 on top of the base denoising loop. The denoiser, however, already re-predicts every position at every step, including committed ones. A direct visible-to-visible revision sampler that exploits these predictions without remasking or auxiliary modules remains largely unexplored. We close both gaps with D3IM (§ 3.1 ), a direct-revision sampler that re-predicts each intermediate sequence from current predictions rather than editing the previous state. Theoretically , we derive D3IM as a corrector-style reverse update for pretrained MDLMs like LLaDA, in which visible tokens may transition directly to new token values rather than being copied or first returned to [MASK] (Appendix A.4 ). Practically , D3IM is parameter-free and reuses the standard denoising forward pass without extra modules or auxiliary passes, under the same linear top- K schedule as standard unmasking: at each step, it re-predicts all positions and retains the top- K by current confidence, so a previously committed token survives only if the model continues to support it. D3IM provides a mechanism for revision. However, directly applying it to a base MDLM trained only with the standard mask-only objective degrades performance. We trace this to preservation bias : the model was never exposed to its own errors during training, and it overwhelmingly reproduces wrong committed tokens rather than correcting them (§ 4.2 ). To address this, we propose SCOPE (Self-Conditioned On Prediction Errors, § 3.2 ), a lightweight post-training procedure that simulates D3IM’s sampling process, exposing the model to its own high-confidence errors and training it to recover the ground truth. With SCOPE+D3IM, LLaDA-8B at 64 steps improves over the original LLaDA-8B with standard unmasking by +13.0 on GSM8K, +4.8 on MATH-500, +15.3 on HumanEval, and +10.4 on MBPP, with gains that increase as more denoising steps are used on math and HumanEval. Our contributions: 1. We propose D3IM , a parameter-free direct-revision sampler for pretrained MDLMs. D3IM is derived as a corrector-style reverse update in which visible tokens may transition directly to new values rather than being copied or returned to [MASK] (§ 3.1 ; Appendix A.4 ). 2. We identify preservation bias as the model-side obstacle to revision: trained on ground-truth context, the model reproduces its own wrong predictions even when the training objective does not require it (§ 4.2 ). 3. We show that a lightweight post-training procedure ( SCOPE ) that simulates D3IM’s sampling process is sufficient to substantially reduce preservation bias and make D3IM effective, without complex multi-objective training or architectural changes (§ 3.2 ). 2 Preliminaries Absorbing MDMs. Classical masked diffusion models generate by gradually converting mask tokens into visible tokens 1 ; 20 ; 23 . In the absorbing formulation, once a coordinate becomes visible, the clean-token posterior collapses to the current token, p(X_{0}^{i}\mid X_{t},X_{t}^{i}\neq m)=\delta_{X_{t}^{i}}. (1) Thus, a visible token is no longer a hypothesis to be re-evaluated; it is treated as the clean token itself. This makes monotonic denoising stable, but it also makes committed-token correction impossible: if an early token is wrong, later reverse steps have no mechanism to replace it. We provide the formal derivation in Appendix A . LLaDA and behavioral absorption. LLaDA-style MDLMs weaken this constraint at the training-objective level. Their loss supervises only masked positions, \mathcal{L}_{\mathrm{LLaDA}}=-\sum_{i\in\mathcal{M}_{t}}\log p_{\theta}(x_{i}^{*}\mid\mathbf{x}_{t}), (2) and imposes no loss on already visible tokens 22 . The model is therefore not explicitly trained to satisfy the absorbing identity, and can in principle assign a different prediction to a visible position. However, this does not mean that absorption is solved. Standard LLaDA inference still freezes committed tokens, and even when this freezing is removed, the model itself tends to preserve its own visible predictions. In our wrong-commit stress test (§ 4.2 ), LLaDA-8B-Instruct preserves its own wrong visible token 70.1% of the time and recovers the ground-truth token only 9.2%. We call this preservation bias : LLaDA-style training removes the formal absorbing constraint, but absorption reappears as a learned behavior under self-generated context. Existing revision methods. ReMDM 28 provides a principled way to introduce remasking into absorbing-state MDMs by deriving a modified backward remasking process. This is well matched to the classical setting where visible tokens are locked by the absorbing posterior. More recent revokable decoding methods 3 ; 29 ; 12 ; 26 ; 14 depart from this derivation and instead use confidence thresholds,

Read the complete text