Simple Denoising Diffusion Language Models

Huaisheng Zhu 1 , Zhengyu Chen 4 , Shijie Zhou 5 , Zhihui Xie 6 , Yige Yuan 7 , Zhimeng Guo 1 , Siyuan Xu 1 , Hangfan Zhang 1 , Vasant Honavar 1 , Teng Xiao 2,3 1 The Pennsylvania State University, 2 Allen Institute for AI (AI2), 3 University of Washington, 4 Meituan Inc, 5 University at Buffalo, 6 · 2025

Read Simple Denoising Diffusion Language Models by Huaisheng Zhu 1 , Zhengyu Chen 4 , Shijie Zhou 5 , Zhihui Xie 6 , Yige Yuan 7 , Zhimeng Guo 1 , Siyuan Xu 1 , Hangfan Zhang 1 , Vasant Honavar 1 , Teng Xiao 2,3 1 The Pennsylvania State University, 2 Allen Institute for AI (AI2), 3 University of Washington, 4 Meituan Inc, 5 University at Buffalo, 6 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22926.txt.

Introduction

Huaisheng Zhu and ten co-authors seek a simpler way to train diffusion language models that start from a uniform prior. Such models can fix their own errors, which may speed up text creation. Past versions, though, used complex loss rules that added cost and slowed growth to larger sizes.

The team trains only on tokens swapped by noise, which steadies learning and matches older results. They add a light extra term that stops outputs from drifting toward a flat spread. Tests on common text data sets show the simpler loss works well and scales, a useful step for faster text models.

Catalog record

CREATORHuaisheng Zhu 1 , Zhengyu Chen 4 , Shijie Zhou 5 , Zhihui Xie 6 , Yige Yuan 7 , Zhimeng Guo 1 , Siyuan Xu 1 , Hangfan Zhang 1 , Vasant Honavar 1 , Teng Xiao 2,3 1 The Pennsylvania State University, 2 Allen Institute for AI (AI2), 3 University of Washington, 4 Meituan Inc, 5 University at Buffalo, 6
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-22926

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Simple Denoising Diffusion Language Models Huaisheng Zhu 1 , Zhengyu Chen 4 , Shijie Zhou 5 , Zhihui Xie 6 , Yige Yuan 7 , Zhimeng Guo 1 , Siyuan Xu 1 , Hangfan Zhang 1 , Vasant Honavar 1 , Teng Xiao 2,3 1 The Pennsylvania State University, 2 Allen Institute for AI (AI2), 3 University of Washington, 4 Meituan Inc, 5 University at Buffalo, 6 The University of Hong Kong, 7 Alibaba Group [email protected], [email protected] Simple-Denoising-Diffusion-Language-Models Abstract Diffusion models have recently been extended to language generation through Masked Diffusion Language Models (MDLMs), which achieve performance competitive with strong autoregressive models. However, MDLMs tend to degrade in the few-step regime and cannot directly adopt existing few-step distillation methods designed for continuous diffusion models, as they lack the intrinsic property of mapping from noise to data. Recent Uniform-state Diffusion Models (USDMs), initialized from a uniform prior, alleviate some limitations but still suffer from complex loss formulations that hinder scalability. In this work, we propose a simplified denoising-based loss for USDMs that optimizes only noise-replaced tokens, stabilizing training and matching ELBO-level performance. Furthermore, by framing denoising as self-supervised learning, we introduce a simple modification to our denoising loss with contrastive-inspired negative gradients, which is practical and yield additional improvements in generation quality. 1 Introduction Diffusion models are powerful generative frameworks that excel at producing realistic, high-quality continuous data such as images and videos ho2020denoising ; song2020denoising ; rombach2022high ; kong2020diffwave ; ho2022video . They achieve this by training denoising models to reconstruct samples corrupted with varying levels of Gaussian noise. Generation then proceeds through a Markov chain: starting from pure noise, the model iteratively denoises the sample, gradually transforming it into a clean image. To further advance the capabilities of diffusion models, Masked Diffusion Models (MDMs), which use the prior distribution by masking all tokens, have recently demonstrated remarkable progress across language generation tasks sohl2015deep ; austin2021structured ; campbell2022continuous ; lou2023discrete ; meng2022concrete . By optimizing the simpilfied varaint of evidence lower bound (ELBO), masked diffusion language models have achieved performance comparable to, and in some cases surpassing, that of autoregressive models (ARMs) sahoo2024simple ; shi2024simplified ; nie2025large . Moreover, recent studies have investigated the scaling properties of MDMs, demonstrating that they can achieve competitive performance with state-of-the-art autoregressive models of similar size (e.g., LLaMA 2 touvron2023llama and LLaMA 3 dubey2024llama ) across a range of downstream tasks nie2025large ; gong2024scaling ; nie2024scaling ; gong2025diffucoder ; ye2025dream . Despite its great success, MDMs experience severe performance degradation in the few-step regime deschenaux2024beyond . In contrast to diffusion models with Probability Flow ODEs in continuous space song2020score , MDMs lack an implicit property—a deterministic mapping from noise to data. To address this limitation, recent work on Uniform-state Diffusion Models (USDMs) has explored language models initialized from a uniform distribution, analogous to Gaussian noise sahoo2025diffusion ; austin2021structured ; zhao2024informed ; schiff2024simple . These models achieve performance comparable to MDMs while demonstrating strong potential to reduce the number of sampling steps without compromising generation quality. Figure 1 : Validation Gen PPL of different models over training steps. However, the current state-of-the-art USDMs sahoo2025diffusion , which adopts uniform distributions as the prior, still suffers from a complex loss formulation. This complexity leads to additional computational overhead during training and may hinder scalability. In this work, we explore a simpler loss formulation for diffusion language models initialized from a uniform prior (i.e., pure noise). Specifically, we begin with standard denoising objectives, but we find that this naive adoption often causes models to collapse during training. Motivated by the design of MDMs, we instead propose a method called Simple Denoising Diffusion Language Model (SDDLM), which optimizes only the tokens that are replaced with noise. This strategy, akin to the selective denoising behavior of MDMs, stabilizes training and achieves performance comparable to the ELBO-derived loss, while avoiding its significant computational cost. Moreover, we further interpret this denoising process as a form of self-supervised learning, a perspective that has also been explored in diffusion models on continuous spaces chen2024deconstructing . Building on this view, we incorporate negative gradients, inspired by contrastive learning, into the training process. Our empirical results in Figure 2 show that this modification leads to notable improvements in generation quality. 2 Background 2.1 Denoising Diffusion Models Denoising diffusion models on continuous space formulate generation as a Markov process that transforms the data distribution q_{\text{data}} into a simple prior on continuous space, such as a standard normal distribution \mathcal{N}(0,\mathbf{I}) . Concretely, the process begins with samples from the data distribution and iteratively adds noise to produce a sequence of noisy latents \mathbf{x}_{t}\sim q_{t}(\cdot\mid\mathbf{x}_{0}) , whose marginal distribution is: \displaystyle q_{t}\left(\cdot\mid\mathbf{x}_{0};\bar{\alpha}_{t}\right)=\mathcal{N}\left(\mathbf{x}_{t};\sqrt{\bar{\alpha}_{t}}\mathbf{x}_{0},\left(1-\bar{\alpha}_{t}\right)\boldsymbol{I}\right), (1) where the diffusion parameter \bar{\alpha}_{t}\in[0,1] is a monotonically decreasing function in t and \mathbf{x}_{0}\sim q_{\text{data}} . Then, a simplified version of evidence lower bound (ELBO) based on denosing noisiy images into clean images is minimized to train the diffusion model with the following equation: \displaystyle\mathbb{E}_{\mathbf{x}_{0},t,\mathbf{\epsilon}}\left[\lambda(t)\left\|\mathbf{x}_{0}-\mathbf{x}_{\theta}\left(\mathbf{x}_{t},t\right)\right\|^{2}\right] (2) where \boldsymbol{\epsilon}\sim\mathcal{N}(0,\boldsymbol{I}),t\sim\mathcal{U}(0,T),\mathbf{x}_{t}\sim q_{t}\left(\cdot\mid\mathbf{x},\bar{\alpha}_{t}\right) . \lambda(t) is a time dependent weighting function and can be ignored during the training process. \theta represents learnable parameters. 2.2 Discrete Diffusion Models Previous objectives and formulations are primarily based on Gaussian distributions and operate in continuous spaces. To adapt diffusion models to discrete data \mathbf{x}\in\mathcal{V} , where \mathcal{V} denotes the vocabulary for language generation, the discrete diffusion framework ( sohl2015deep ; austin2021structured ) extends the core idea of continuous denoising diffusion models: mapping the data distribution q_{\text{data}} to a simple prior distribution through a sequence of Markov states. Similar to their continuous counterparts, the noise-adding process—referred to as the forward process (q_{t})(t\in[0,1]) —smoothly transitions from q_{\text{data}} to a categorical prior \operatorname{Cat}(\cdot;\boldsymbol{\pi}) by interpolating between the data distribution and the prior. The corresponding marginals conditioned on one token \mathbf{x}_{0}^{l} at time t are given by: \displaystyle q_{t}\left(.\mid\mathbf{x}^{l}_{0};\alpha_{t}\right)=\operatorname{Cat}\left(.;\alpha_{t}\mathbf{x}^{l}_{0}+\left(1-\alpha_{t}\right)\boldsymbol{\pi}\right). (3) For MDLMs, the prior \boldsymbol{\pi} is typically defined using a special masked token, i.e., \boldsymbol{\pi}=\mathbf{M} with \mathbf{M}\in\mathcal{V} ( sahoo2024simple ) . Alternatively, a uniform prior can be defined as \boldsymbol{\pi}=\mathbf{1}/V , where V=|\mathcal{V}| denotes the vocabulary size. Spcifically, in MDMs, a token \mathbf{x} either stays unchanged or is replaced by the mask token \mathbf{m} , remaining masked thereafter. In USDMs, each token can instead transition uniformly to any token in \mathcal{V} , with probabilities determined by the diffusion timestep. To train USDMs, the Negative Evidence Lower Bound (NELBO) loss for the token l is derived using principles similar to those of continuous diffusion models, and can be expressed in the following form ( lou2023discrete ; schiff2024simple ) : \mathcal{L}^{l}_{\text{USDM}}=\mathbb{E}_{t\sim\mathcal{U}[0,1],q_{t}\left(\mathbf{x}^{l}_{t}\mid\mathbf{x}^{l}_{0};\alpha_{t}\right)}-\frac{\alpha_{t}^{\prime}}{V\alpha_{t}}\left[\frac{V}{\tilde{\mathbf{x}}^{l}_{i}}-\frac{V}{\left(\tilde{\mathbf{x}}^{l}_{\theta}\right)_{i}}-\sum_{j}\frac{\tilde{\mathbf{x}}^{l}_{j}}{\tilde{\mathbf{x}}^{l}_{i}}\log\frac{\left(\tilde{\mathbf{x}}^{l}_{\theta}\right)_{i}\cdot\tilde{\mathbf{x}}^{l}_{j}}{\left(\tilde{\mathbf{x}}^{l}_{\theta}\right)_{j}\cdot\tilde{\mathbf{x}}^{l}_{i}}\right], (4) where \tilde{\mathbf{x}}^{l}=K\alpha_{t}\mathbf{x}_{0}^{l}+\left(1-\alpha_{t}\right)\mathbf{1} , \tilde{\mathbf{x}}^{l}_{\theta}=V\alpha_{t}\mathbf{x}^{l}_{\theta}\left(\mathbf{x}_{t},t\right)+\left(1-\alpha_{t}\right)\mathbf{1} , \mathbf{x}^{l}_{t}\sim q_{t}(\cdot\mid\mathbf{x}^{l}_{0};\alpha_{t}) and \alpha_{t}^{\prime} is the time-derivative of the \alpha_{t} . i=\arg\max_{j\in[V]}\left(\mathbf{x}^{l}_{t}\right)_{j} is the non-zero entry of \mathbf{x}^{l}_{t} . Other studies have also explored more efficient approaches to computing the loss in Equation ( 4 ) sahoo2025diffusion . \mathbf{x}^{l}_{\theta} denotes a neural network \mathcal{V}\times[0,1]\rightarrow\Delta^{K} with trainable parameters \theta . After training, USDMs typically generate samples by applying the reverse diffusion process, starting from the uniform prior: \displaystyle q_{s\mid t}\left(.\mid\mathbf{x}^{l}_{t},\mathbf{x}^{l}_{0}\right)=\operatorname{Cat}\left(;\frac{V\alpha_{t}\mathbf{x}^{l}_{t}\odot\mathbf{x}^{l}_{0}+\left(\alpha_{t\mid s}-\alpha_{t}\right)\mathbf{x}^{l}_{t}}{V\alpha_{t}\left\langle\mathbf{x}^{l}_{t},\mathbf{x}^{l}\right\rangle+1-\alpha_{t}}\right.+\left.\frac{\left(\alpha_{s}-\alpha_{t}\right)\mathbf{x}^{l}_{0}+\left(1-\alpha_{t\mid s}\right)\left(1-\alpha_{s}\right)\mathbf{1}/V}{V\alpha_{t}\left\langle\mathbf{x}^{l}_{t},\mathbf{x}^{l}_{0}\right\rangle+1-\alpha_{t}}\right), (5) where s<t and \alpha_{t\mid s}=\alpha_{t}/\alpha_{s} . During inference, we replace \mathbf{x} in Equation ( 5 ) with \mathbf{x}_{\theta}(\mathbf{x}_{t},t) . In the following section, we use p_{\theta}(\mathbf{x}_{0}\mid\mathbf{x}_{t}) to denote \mathbf{x}_{\theta}(\mathbf{x}_{t},t) for better understanding. 3 Related Works The development of DLLMs is motivated

Read the complete text