Remasking Discrete Diffusion Models with Inference-Time Scaling

Guanghan Wang, Yair SchiffSubham Sekhar SahooVolodymyr KuleshovCornell Tech {gw354,yzs2,sss284,vk379}@cornell.edu · 2025

Read Remasking Discrete Diffusion Models with Inference-Time Scaling by Guanghan Wang, Yair SchiffSubham Sekhar SahooVolodymyr KuleshovCornell Tech {gw354,yzs2,sss284,vk379}@cornell.edu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-00307.txt.

Introduction

Guanghan Wang and co-authors present a 2025 paper on text generation by diffusion. Diffusion models shine by fixing their output step by step. Masked text models lack that gift, since a word once placed can never be revised, even when it is wrong.

The sampler, called ReMDM, lets a trained masked model re-mask and redo parts of its output. It rests on a backward process built for that purpose, so no retraining is needed. More sampling steps steadily lift text quality toward that of word-by-word models. With tight budgets the sampler still holds quality better than plain masking. Gains also showed for images and for molecule design tasks under guidance.

Catalog record

CREATORGuanghan Wang, Yair SchiffSubham Sekhar SahooVolodymyr KuleshovCornell Tech {gw354,yzs2,sss284,vk379}@cornell.edu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE115 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-00307

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Remasking Discrete Diffusion Models with Inference-Time Scaling Guanghan Wang, Yair SchiffSubham Sekhar SahooVolodymyr KuleshovCornell Tech {gw354,yzs2,sss284,vk379}@cornell.edu Abstract Part of the success of diffusion models stems from their ability to perform iterative refinement, i.e., repeatedly correcting outputs during generation. However, modern masked discrete diffusion lacks this capability: when a token is generated, it cannot be updated again, even when it introduces an error. Here, we address this limitation by introducing the remasking diffusion model (ReMDM) sampler, a method that can be applied to pretrained masked diffusion models in a principled way and that is derived from a discrete diffusion model with a custom remasking backward process. Most interestingly, ReMDM endows discrete diffusion with a form of inference-time compute scaling. By increasing the number of sampling steps, ReMDM generates natural language outputs that approach the quality of autoregressive models, whereas when the computation budget is limited, ReMDM better maintains quality. ReMDM also improves sample quality of masked diffusion models for discretized images, and in scientific domains such as molecule design, ReMDM facilitates diffusion guidance and pushes the Pareto frontier of controllability relative to classical masking and uniform noise diffusion. We provide the code along with a blog post on the project page: https://remdm.github.io 1 Introduction Diffusion models have gained significant traction as algorithms for generating high-quality images and videos [ 47 , 49 , 18 , 39 ] . Part of the success of diffusion stems from its ability to perform iterative refinement—repeatedly modifying outputs and fixing errors over multiple steps of generation—which makes diffusion models especially effective at guided generation [ 9 , 17 ] and fast sampling [ 48 , 41 , 51 ] and supports inference-time scaling to improve sample quality [ 27 , 46 , 61 ] . Discrete counterparts of diffusion models [ 2 ] have also been steadily improving in quality on tasks such as language modeling and biological sequence design [ 26 , 38 , 43 , 1 , 40 ] . However, modern discrete diffusion—especially the most performant kind that relies on masking [ 29 , 38 , 45 ] —lacks the fundamental ability to iteratively refine outputs. Once a discrete token is unmasked, it cannot be updated again, even if it introduces an error. The lack of error correction in turn sets limits on controllable generation, sampling speed, and sample quality. Here, we address the inability of masked diffusion models to perform iterative refinement by introducing a new sampler that supports remasking during generation. Our method, the remasking diffusion model (ReMDM) sampler, is simple and allows users to directly specify the probability of remasking a token at each time step. We augment the sampler with components that range from nucleus sampling to remasking schedules, significantly boosting performance. Figure 1 : Our family of masked diffusion processes allow for more flexible generation with remasking of already decoded tokens. This improves sample quality and further closes the gap to AR models. (Left) An illustrative example of errors fixed by ReMDM . The first two tokens can be “They sell” or “She sells”, but due to the independence of the parallelized decoding processes, “She sell” is decoded. Such mistakes can be corrected by remasking samplers. (Right) MAUVE scores on OpenWebText. MDLM is from Sahoo et al. 2024a . FB and DFM denote the forward-backward [ 5 ] and discrete flow matching [ 13 ] correctors, respectively. Our approach for deriving the sampler is rooted in probabilistic modeling. We show that our method corresponds to ancestral sampling in a discrete diffusion model (also called ReMDM) characterized by a remasking backward process. The ReMDM model admits an objective that is a rescaled version of the MDLM objective [ 38 ] . This suggests using the ReMDM sampler on top of pretrained MDLMs, which we find to work well. We complement our analysis of the sampler by also showing that it can be interpreted as a predictor-corrector technique [ 50 , 5 ] . Notably, we demonstrate that ReMDM is theoretically more general than previous corrector samplers for masked diffusion models [ 5 , 13 ] . Most interestingly, the ReMDM sampler endows discrete diffusion with a form of inference-time compute scaling. By increasing the number of denoising steps on language modeling tasks, ReMDM generates samples with significantly higher sample quality metrics than any previous diffusion model and almost matches the performance of an autoregressive (AR) model with the same architecture. Conversely, when we reduce the sampling steps to increase speed, ReMDM’s performance degrades less than other samplers. On image generation, ReMDM scales better with inference-time compute than vanilla masked diffusion models like MaskGiT [ 7 ] . Additionally, ReMDM models are more amenable to guidance [ 43 ] —in molecule design experiments, ReMDM pushes the Pareto frontier of novelty and the desired molecular property relative to alternatives relying on masking or uniform noise diffusion. Contributions We summarize our contributions as follows: 1. We introduce the remasking diffusion model (ReMDM) sampler that yields performant iterative refinement via remasking to masked diffusion models. 2. We show that our method is a form of ancestral sampling in a probabilistic model whose ELBO is similar to that of classical masked diffusion. This analysis suggests using our sampler on top of pretrained models, which we find to work well. 3. Across the domains of natural language, discretized images, and molecule string representations, we demonstrate empirically that ReMDM endows masked diffusion with inference-time scaling that improves sample quality with more computation and enhances controlled generation and downstream performance. 2 Background Discrete Diffusion Models Diffusion models [ 47 , 18 ] are characterized by parametric models p_{\theta} trained to reverse a fixed noising process q that maps clean data \mathbf{x} to increasingly noisy latent variables \mathbf{z}_{t} , for t\in[0,1] . Austin et al. 2021 , Sahoo et al. 2024a , Shi et al. 2024 extend this framework to discrete signals. Formally, we define \mathbf{x}\in\{0,1\}^{|V|}\subset\Delta^{|V|} as one-hot vectors, where |V| is the vocabulary size and \Delta^{|V|} is the simplex over this vocabulary. For finite-time processes, we let T be the number of time steps. For each time step t(i)=\frac{i}{T}\in[0,1] (i.e., for i=0,1,\ldots,T ), each intermediate latent variable \mathbf{z}_{t(i)}\in\{0,1\}^{|V|} has the marginal: q(\mathbf{z}_{t}\mid\mathbf{x})=\mathrm{Cat}(\mathbf{z}_{t};\alpha_{t}\mathbf{x}+(1-\alpha_{t})\bm{\pi})\in\Delta^{|V|}, (1) where \bm{\pi} is a user-specified prior and where we have dropped the explicit dependence of t on i . The predefined schedule \alpha_{t}\in[0,1] is monotonically decreasing in t . For sequences of L tokens, we denote clean / noisy sequences as \mathbf{x}^{(1:L)} / \mathbf{z}_{t}^{(1:L)} and each token \ell\in\{1,\ldots,L\} as \mathbf{x}^{(\ell)} / \mathbf{z}_{t}^{(\ell)} . Masked Diffusion Language Models A special case of this formulation, known as ‘masking’ or ‘absorbing state’ diffusion, sets the limiting distribution \bm{\pi}=\bm{m}, a one-hot representation of a special [MASK] token. With s(i)=\frac{i-1}{T} as the time step directly preceding t , these posteriors take the form: q(\mathbf{z}_{s}\mid\mathbf{z}_{t},\mathbf{x})=\mathrm{Cat}\Big(\mathbf{z}_{s};\frac{\alpha_{s}-\alpha_{t}}{1-\alpha_{t}}\mathbf{x}+\frac{1-\alpha_{s}}{1-\alpha_{t}}\mathbf{z}_{t}\Big). (2) Importantly, despite the recent success of MDLM [ 38 ] and other absorbing state discrete diffusion models, these models suffer from a key drawback, which we call the failure to remask property. Namely, the posterior formula in ( 2 ) implies that any unmasked token \mathbf{z}_{t}\neq\bm{m} must remain unchanged throughout the entire denoising process, for all s\leq t, which Sahoo et al. 2024a formalize as q(\mathbf{z}_{s}\mid\mathbf{z}_{t},\mathbf{x})=\mathrm{Cat}(\mathbf{z}_{s};\mathbf{z}_{t}) if \mathbf{z}_{t}\neq\bm{m} . Once a token is

Read the complete text