Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusion

Jaihoon Kim Taehoon Yoon Prin Phunyaphibarn, [1.75ex] Seungjun Kim Morteza Mardani Minhyuk Sung, [1.75ex] KAIST · 2026

Read Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusion by Jaihoon Kim Taehoon Yoon Prin Phunyaphibarn, [1.75ex] Seungjun Kim Morteza Mardani Minhyuk Sung, [1.75ex] KAIST (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23346.txt.

Introduction

Kim and colleagues speed up reward-guided sampling from discrete diffusion models, which generate structured categorical data such as text or DNA. Twisted sequential Monte Carlo can target reward-tilted distributions exactly in the limit, but estimating its twist function costs dearly at inference. Their contrastive distribution matching learns that twist from positive and negative samples, adding under 5 percent overhead to a forward pass. Tests span toxic text, DNA design, protein design, and language-model alignment, beating baselines at equal run time.

Catalog record

CREATORJaihoon Kim Taehoon Yoon Prin Phunyaphibarn, [1.75ex] Seungjun Kim Morteza Mardani Minhyuk Sung, [1.75ex] KAIST
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87717
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.23346

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusion Jaihoon Kim Taehoon Yoon Prin Phunyaphibarn, [1.75ex] Seungjun Kim Morteza Mardani Minhyuk Sung, [1.75ex] KAIST Abstract Discrete diffusion models have emerged as powerful frameworks for generating structured categorical data. However, efficiently sampling from reward-tilted distributions remains a fundamental challenge. While Twisted Sequential Monte Carlo (SMC) offers asymptotic exactness for this task, estimating the optimal twist function in discrete state spaces necessitates costly Monte Carlo approximations, resulting a severe computational bottleneck at inference. To overcome this limitation, we introduce Contrastive Distribution Matching ( CDM ), a novel framework that amortizes the cost of SMC inference by learning a parameterized twist function via positive and negative samples. For efficient training, we reformulate the gradient estimator to leverage the closed-form forward kernels of discrete diffusion models. In practice, evaluating our learned twist function incurs less than 5 % 5\% additional computational overhead compared to a single forward pass of the base model. Through extensive empirical evaluations, we demonstrate that CDM consistently outperforms existing baselines under matched wall-clock time. We validate the effectiveness and versatility of our approach across a diverse range of applications, including toxic text generation, regulatory DNA sequence design, protein designability, and diffusion large language model alignment. 1 Introduction Diffusion models have demonstrated remarkable generative performance across a wide array of continuous domains [ 68 , 37 , 36 ] . Recently, their application to discrete state spaces has yielded significant breakthroughs; in language modeling, discrete diffusion models not only enable efficient few-step generation [ 60 , 40 , 56 ] but also achieve sample quality on par with autoregressive language models [ 53 , 87 , 88 ] . Furthermore, discrete diffusion has been successfully extended to scientific applications, driving advances in sequence design tasks such as regulatory DNA [ 43 , 70 ] and de novo protein [ 84 ] generation. A central application of these models is reward alignment . Given a scalar reward representing human preference [ 46 , 83 ] or protein designability [ 44 ] , the objective is to sample from a tilted target distribution that biases the pretrained prior toward higher values of the downstream reward. To sample exactly from this target distribution, the optimal proposal is formulated by tilting the pretrained base model with an optimal twist function . In this work, we integrate the Sequential Monte Carlo (SMC) framework [ 18 , 52 ] , an asymptotically unbiased sampler, with discrete diffusion models. In the continuous domain, the SMC framework has been widely adopted for reward alignment largely due to its computational efficiency and empirical success [ 85 , 6 , 34 , 73 , 74 , 67 , 1 ] . This tractability stems from two key properties: one can easily construct locally optimal proposals using reward gradients [ 11 , 34 ] , and Tweedie’s formula [ 20 ] provides a computationally efficient estimate of the clean state to approximate the twist function. In contrast, translating these successes to discrete domains presents new challenges. Since the state space is discrete, one needs to rely on Gumbel-Softmax trick [ 30 ] to approximate locally optimal proposals, which often leads to gradient bias and optimization instability [ 57 , 47 ] . More importantly, the absence of Tweedie’s estimate in discrete diffusion [ 65 ] leaves Monte Carlo estimation as the standard practice for approximating the twist function [ 55 , 43 , 13 ] , which can introduce a significant inference overhead when the downstream reward is computationally expensive (e.g., protein designability). Motivated by this bottleneck, we propose Contrastive Distribution Matching ( CDM ), which learns the twist function via a contrastive learning objective to reduce the twist function evaluation to a constant-time operation, amortizing SMC inference. In contrast to existing regression-based methods applied to discrete diffusion that learn the twist by drawing samples from a base proposal [ 43 , 78 ] , CDM minimizes the forward KL divergence against the target distribution. The gradient of this objective exhibits a contrastive structure, utilizing positive and negative samples to upweight high-reward regions while downweighting suboptimal ones. Additionally, we introduce a novel training scheme that leverages the forward process of the diffusion model for efficient training. Specifically, we maintain a buffer of positive samples drawn from the approximated target distribution and apply the closed-form forward kernel, allowing a single clean sample to be reused across multiple timesteps and gradient updates. Our experimental evaluations demonstrate that CDM consistently achieves superior scaling behavior, outperforming baselines in a diverse range of applications: toxic text generation, regulatory DNA design, protein generation, and diffusion LLM (dLLM) preference alignment. Furthermore, since CDM learns the twist function, it is agnostic to the choice of the proposal distribution. This allows it to be paired with any proposal distribution, including those already fine-tuned (e.g., d1 [ 94 ] , DRAKES [ 82 ] ), for further synergistic performance gains. Moreover, we demonstrate that the contrastive learning objective of CDM yields superior performance and more efficient training compared to the standard regression-based twist objective [ 78 , 43 ] . In summary, our key contributions are as follows: • We propose Contrastive Distribution Matching ( CDM ), an SMC-amortization framework for discrete diffusion that reduces the cost of applying the twist at inference time to a constant-time operation. • We design a novel, diffusion-native training scheme that leverages the closed-form forward process, enabling efficient training that scales to expensive reward functions. • We demonstrate the versatility of CDM across a broad range of applications, including toxic text generation, regulatory DNA design, protein generation, and dLLM alignment, consistently showing superior performance. • We validate that CDM delivers synergistic improvements even when paired with fine-tuning-based methods, while demonstrating superior efficacy than the regression-based twist objectives employed in previous discrete diffusion models [ 43 ] . 2 Preliminary: Discrete Diffusion Let \mathcal{V}:=\{\mathbf{x}\in\{0,1\}^{V}:\sum_{i=1}^{V}\mathbf{x}_{i}=1\} define the space of V category one-hot vectors. We write \Delta^{V} for the V -simplex and \text{Cat}(\cdot;\bm{\pi}) for the categorical distribution with probability vector \bm{\pi}\in\Delta^{V} . A prominent class of generative models for discrete state spaces is the Masked Diffusion Model (MDM) [ 69 , 5 , 72 ] , which defines a forward corruption process terminating in a mask state \mathbf{m} . Let (p_{t})_{t=0}^{T} denote the sequence of marginal distributions induced by this forward process. The process interpolates between the data distribution p_{0}=p_{\text{data}} and the prior p_{T}=\mathbf{m} via a monotonically decreasing noise schedule \alpha_{t}\in[0,1] : p(\mathbf{x}_{t}\mid\mathbf{x}_{0})=\text{Cat}\big(\mathbf{x}_{t};\alpha_{t}\mathbf{x}_{0}+(1-\alpha_{t})\mathbf{m}\big) (1) The sampling proceeds by simulating the reverse process, where the exact posterior is given by: p(\mathbf{x}_{t-1}\mid\mathbf{x}_{t},\mathbf{x}_{0})=\begin{cases}\text{Cat}(\mathbf{x}_{t-1};\mathbf{x}_{t})&\text{if }\mathbf{x}_{t}\neq\mathbf{m},\\[4.0pt] \text{Cat}\left(\mathbf{x}_{t-1};\dfrac{(1-\alpha_{t-1})\mathbf{m}+(\alpha_{t-1}-\alpha_{t})\mathbf{x}_{0}}{1-\alpha_{t}}\right)&\text{if }\mathbf{x}_{t}=\mathbf{m}.\end{cases} (2) Since the clean data \mathbf{x}_{0} is unknown during sampling, it is approximated by a denoising neural network \mathbf{x}_{\theta}(\mathbf{x}_{t})\in\Delta^{V} . Substituting this prediction into the posterior Eq. 2 yields the parameterized reverse transition kernel p^{\text{base}}(\mathbf{x}_{t-1}\mid\mathbf{x}_{t})=p(\mathbf{x}_{t-1}\mid\mathbf{x}_{t},\mathbf{x}_{\theta}(\mathbf{x}_{t})) . The resulting reverse chain induces a trajectory distribution p^{\text{base}}_{0:T}(\mathbf{x}_{0:T}) with time marginals p_{t}^{\text{base}}(\mathbf{x}_{t}) . The model parameters \theta are optimized by minimizing a weighted cross entropy loss which is equivalent to the negative ELBO in the continuous-time limit. We refer to previous works [ 5 , 72 ] for the detailed derivations. 3 Twisted Sequential Monte Carlo for Reward Alignment 3.1 KL Regularized Reward Alignment Let r:\mathcal{V}^{N}\to\mathbb{R} be

Read the complete text