MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Control
Yuchen Zhu, Wei Guo, Jaemoo Choi, Guan-Horng Liu, Yongxin Chen, Molei Tao · 2025
Read MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Control by Yuchen Zhu, Wei Guo, Jaemoo Choi, Guan-Horng Liu, Yongxin Chen, Molei Tao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10684.txt.
Introduction
Drawing samples from a known but unscaled chance law is a core task in physics and learning. Yuchen Zhu and co-authors train a neural sampler for it, in work from twenty twenty-five. Hard cases have huge state counts and many peaks. Their sampler grows from a masked spread process.
The training aligns two path laws through a family of goals drawn from control theory for jump chains. Tests on laws with unlike traits showed sharp sampling even at very high dims. The sampler beat past learning-based rivals by a wide edge. Extra checks map which parts drive the gain and how far the frame can stretch. Code is shared for reuse.
Catalog record
| CREATOR | Yuchen Zhu, Wei Guo, Jaemoo Choi, Guan-Horng Liu, Yongxin Chen, Molei Tao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 127 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-10684
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Control Yuchen Zhu, Wei Guo, Jaemoo Choi, Guan-Horng Liu, Yongxin Chen, Molei Tao Abstract We study the problem of learning a neural sampler to generate samples from discrete state spaces where the target probability mass function π ∝ e − U \pi\propto\mathrm{e}^{-U} is known up to a normalizing constant, which is an important task in fields such as statistical physics, machine learning, combinatorial optimization, etc. To better address this challenging task when the state space has a large cardinality and the distribution is multi-modal, we propose M asked D iffusion N eural S ampler ( MDNS ), a novel framework for training discrete neural samplers by aligning two path measures through a family of learning objectives, theoretically grounded in the stochastic optimal control of the continuous-time Markov chains. We validate the efficiency and scalability of MDNS through extensive experiments on various distributions with distinct statistical properties, where MDNS learns to accurately sample from the target distributions despite the extremely high problem dimensions and outperforms other learning-based baselines by a large margin. A comprehensive study of ablations and extensions is also provided to demonstrate the efficacy and potential of the proposed framework. * * footnotetext: Equal contribution, random order. 1 Introduction Drawing samples from an unnormalized target distribution \pi\propto\mathrm{e}^{-U} on some state space \mathcal{X}_{0} is a fundamental problem with wide-ranging applications across fields such as statistical physics [ LB14 ] , Bayesian inference [ Gel+13 ] , machine learning [ And+03 , WJ08 ] , etc. For decades, Markov chain Monte Carlo (MCMC) methods, such as the Langevin Monte Carlo, the Metropolis-Hastings algorithm, and the Glauber dynamics, have been a cornerstone. These methods simulate a Markov chain whose stationary distribution is the target distribution, and converge provably fast under certain circumstances [ Che22 ] . Despite their widespread adoption, MCMC algorithms face significant challenges, especially when the state space has high dimensions and the target distribution is multimodal, which hinders efficient exploration and convergence (see, e.g., the lower bound results in [ Ran06 , GT06 , GLR18 , HZ25 ] ). Recently, inspired by the success of diffusion models in generative modeling for continuous data [ HJA20 , SME21 , Son+21 ] , significant progress has been made in leveraging neural networks to learn a stochastic differential equation (SDE) to generate samples from \pi , which we collectively refer to as neural samplers (e.g., [ ZC22 , VGD23 , AVE25 , Hav+25 ] ). Concurrently, the diffusion paradigm has been effectively extended to discrete state spaces [ Aus+21 , Cam+22 , Sun+23a , LME24 ] , finding broad applications in domains involving sequences of categorical data. However, while neural samplers have been extensively developed for distributions on \mathbb{R}^{d} and discrete diffusion models have shown promise in modeling discrete data, the study of leveraging discrete diffusion models for sampling remains relatively unexplored. Addressing this gap is crucial as numerous real-world sampling problems in areas like statistical physics [ FL24 ] and combinatorial optimization [ Sun+23 ] necessitate specialized methods to navigate the unique challenges they present, and this work aims to develop such a specialized approach. Contributions. Our contributions can be summarized in the following points. (1) We introduce Masked Diffusion Neural Sampler (MDNS), a novel framework for training discrete neural samplers leveraging optimal control of CTMC and masked discrete diffusion models. (2) To cope with the optimization challenges caused by the discontinuity of CTMC trajectories, we introduce several learning objectives that are free of differentiability requirements. (3) To achieve scalable training in high dimensions, we propose a novel training loss named Weighted Denoising Cross-entropy Equation 16 that directly implements score learning on the target distribution using importance sampling. (4) We conduct comprehensive experiments and ablation studies on Ising and Potts models to validate the efficacy of our method. MDNS can accurately sample from target distributions even when the state space has a cardinality of 10^{122} , which substantially surpasses other learning-based methods. Related works. We refer readers to Appendix A for a detailed discussion of related works. 2 Preliminaries 2.1 Continuous-time Markov Chains A continuous-time Markov chain (CTMC) X=(X_{t})_{t\in[0,T]} is a stochastic process defined on a probability space (\Omega,\mathcal{F},\Pr) and takes value in a finite state space \mathcal{X} . Its law is completely characterized by the generator Q=(Q_{t}\in\mathbb{R}^{\mathcal{X}\times\mathcal{X}})_{t\in[0,T]} , defined by Q_{t}(x,y)=\lim_{\Delta t\to 0}\frac{1}{\Delta t}(\Pr(X_{t+\Delta t}=y|X_{t}=x)-1_{x=y}), (1) where 1_{A} is the indicator function of the statement A , being one if A holds and zero if otherwise. By definition, the generator satisfies Q_{t}(x,y)\geq 0 for x\neq y and Q_{t}(x,x)=-\sum_{y\neq x}Q_{t}(x,y) . The path of X , i.e., t\mapsto X_{t}(\omega) , is a piecewise constant and càdlàg 1 1 1 Acronym of the French phrase standing for “right continuous with left limits”. function. The path measure of a CTMC X is a probability measure on (\Omega,\mathcal{F}) defined as \mathbb{P}^{X}(A):=\Pr(X\in A) , which is the distribution of X on \Omega . We define \mathbb{P}^{X}_{t} as the marginal distribution of X_{t} , and \mathbb{P}^{X}_{t|s}(\cdot|x) as the distribution of X_{t} conditional on X_{s}=x . The following lemma is an important result to know from the literature of CTMCs, which shows how to compute the Radon-Nikodým (RN) derivative between two path measures driven by CTMCs with different generators and initial distributions, and is crucial for developing our proposed sampling algorithms in Equation 14 : Lemma 1 . Given two CTMCs with generators Q^{1},Q^{2} and initial distributions \mu_{1},\mu_{2} on \mathcal{X} , let \mathbb{P}^{1},\mathbb{P}^{2} be the associated path measures. Then for any trajectory \xi=(\xi_{t})_{t\in[0,T]} , \log\frac{\mathrm{d}\mathbb{P}^{2}}{\mathrm{d}\mathbb{P}^{1}}(\xi)=\log\frac{\mathrm{d}\mu_{2}}{\mathrm{d}\mu_{1}}(\xi_{0})+\sum_{t:\xi_{t-}\neq\xi_{t}}\log\frac{Q^{2}_{t}(\xi_{t-},\xi_{t})}{Q^{1}_{t}(\xi_{t-},\xi_{t})}+\int_{0}^{T}\sum_{y\neq\xi_{t}}(Q^{1}_{t}(\xi_{t},y)-Q^{2}_{t}(\xi_{t},y))\mathrm{d}t. (2) A detailed proof is provided in Section C.3 . An intuitive interpretation of Equation 2 is to view the RN derivative between path measures as the limit of density ratios between finite-dimensional joint distributions, and approximate the transition distribution by Equation 1 . We remark that for masked diffusion models, Equation 2 can be precisely calculated without discretization error, as will be seen later in Equation 14 . 2.2 Discrete Diffusion Models In discrete diffusion models, one learns the generator of a CTMC that starts from a simple distribution p_{\mathrm{init}} and reaches the target distribution p_{\mathrm{data}} at the final time. This CTMC is typically chosen as the time-reversal of a noising process that converts any distribution to p_{\mathrm{init}} . An especially effective subclass of discrete diffusion is the masked discrete diffusion models [ LME24 , Ou+25 , Sah+24 , Shi+24 , Zhe+25 ] , which corresponds to choosing p_{\mathrm{init}} to be p_{\mathrm{mask}} , the Dirac distribution on the fully masked sequence. Adding a mask token \mathbf{M} into the original state space \mathcal{X}_{0}:=\{1,2,...,N\}^{D} containing length- N sequences with vocabulary size D , denote the mask-augmented state space by \mathcal{X}:=\{1,2,...,N,\mathbf{M}\}^{D} . Throughout the paper, x=(x^{1},...,x^{D}) denotes a sequence of tokens, x^{d\leftarrow n} represents the sequence constructed by replacing the d -th position of x by n , and x^{\mathrm{UM}}=(x^{d}:x^{d}\neq\mathbf{M}) represents the unmasked part of x\in\mathcal{X} . Suppose we need to model sequential data X\in\mathcal{X}_{0} following the distribution p_{\mathrm{data}} . The intuition behind the masked discrete diffusion model is to define a noising CTMC that independently and randomly masks each token according to a certain schedule, then reverse this CTMC to generate data from a sequence of pure mask tokens. It is proved in [ Ou+25