DiffuGuard : How Intrinsic Safety is Lost and Found in Diffusion Large Language Models
Zherui Li, Zheng Nie, Zhenhong Zhou, Yufei Guo, Yue Liu, Yitong Zhang, Yu Cheng, Qingsong Wen, Kun Wang, Jiaheng Zhang · 2025
Read DiffuGuard : How Intrinsic Safety is Lost and Found in Diffusion Large Language Models by Zherui Li, Zheng Nie, Zhenhong Zhou, Yufei Guo, Yue Liu, Yitong Zhang, Yu Cheng, Qingsong Wen, Kun Wang, Jiaheng Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-24296.txt.
Introduction
Diffusion language models write by refining many tokens at once, and that process opens new jailbreak paths. Zherui Li and 9 co-authors dissect those weaknesses along two axes: moves within one step and moves across steps. They find a harmful bias in the standard greedy remasking choice, and a path effect where early tokens steer the safety of the final output.
Their defense, DiffuGuard, needs no training. One stage adds controlled randomness to remasking to break the greedy bias. A second audits blocks of text using the model's own signals and repairs risky parts. Across four models and six attack styles, attack success fell by about two thirds, with utility and speed kept.
Catalog record
| CREATOR | Zherui Li, Zheng Nie, Zhenhong Zhou, Yufei Guo, Yue Liu, Yitong Zhang, Yu Cheng, Qingsong Wen, Kun Wang, Jiaheng Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-24296
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DiffuGuard : How Intrinsic Safety is Lost and Found in Diffusion Large Language Models Zherui Li, Zheng Nie, Zhenhong Zhou, Yufei Guo, Yue Liu, Yitong Zhang, Yu Cheng, Qingsong Wen, Kun Wang, Jiaheng Zhang Abstract The rapid advancement of Diffusion Large Language Models (dLLMs) introduces unprecedented vulnerabilities that are fundamentally distinct from Autoregressive LLMs, stemming from their iterative and parallel generation mechanisms. In this paper, we conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics. Experimental results reveal a harmful bias inherent in the standard greedy remasking strategy and identify a critical phenomenon we term Denoising-path Dependence, where the safety of early-stage tokens decisively influences the final output. These findings also indicate that while current decoding strategies constitute a significant vulnerability, dLLMs possess a substantial intrinsic safety potential. To unlock this potential, we propose DiffuGuard , a training-free defense framework that addresses vulnerabilities through a dual-stage approach: Stochastic Annealing Remasking dynamically introduces controlled randomness to mitigate greedy selection bias, while Block-level Audit and Repair exploits internal model representations for autonomous risk detection and guided correction. Comprehensive experiments on four dLLMs demonstrate DiffuGuard ’s exceptional effectiveness, reducing Attack Success Rate against six diverse jailbreak methods from 47.9% to 14.7% while preserving model utility and efficiency. Our code is available at: https://github.com/niez233/DiffuGuard . 1 Introduction Diffusion Large Language Models (dLLMs) are rapidly advancing, demonstrating performance comparable to mainstream Autoregressive (AR) LLMs Yu et al. 2025 ; Li et al. 2025c . In contrast to the token-by-token generation approach of AR LLMs Brown et al. 2020 ; Zhao et al. 2025b , dLLMs adopt a unique generation paradigm: they progressively transform a fully masked sequence into text output through parallel generation and iterative refinement Nie et al. 2025 ; Ye et al. 2025 ; Yang et al. 2025 . This unique paradigm has enabled dLLMs to achieve performance on tasks such as multimodal perception Yang et al. 2025 ; You et al. 2025 , structured generation Zhou et al. 2025 ; Xiong et al. 2025 , and software engineering Labs et al. 2025 ; Xie et al. 2025a ; Li et al. 2025a that is comparable to or even surpasses that of similarly-sized AR LLMs, showcasing their immense potential in generation flexibility and efficiency. However, while the potential of dLLMs is significant, it has also provoked urgent concerns among researchers about their safety, with several severe jailbreak vulnerabilities being revealed Wen et al. 2025 ; Zhang et al. 2025b . This is primarily because the unique features distinguishing dLLMs from AR LLMs introduce novel attack surfaces, as shown in Figure 1 ( Middle ). Specifically, the parallel generation in dLLMs Wu et al. 2025 ; Israel et al. 2025 is prone to causing conflicting safety signals to emerge within a single iteration, leading to imbalanced decisions; the iterative refinement Jin et al. 2025 ; Wang et al. 2025a ; Ye et al. 2025 poses a risk where, as malicious content once introduced can be continuously reinforced, steering the model into harmful generation trajectories. The existence of these novel vulnerabilities renders traditional safety studies designed for AR LLMs no longer applicable Zhang et al. 2025b ; Xie et al. 2025b . Therefore, it has become imperative to revisit and construct a safety framework tailored to dLLMs. Figure 1: Left. The generation diagram of dLLMs; Middle. The unique vulnerabilities of dLLMs, including the intra-step and the inter-step level; Right. DiffuGuard framework achieves significant safety improvements while having minimal impact on model performance and inference latency. To fill the gap in dLLM safety research, we conduct an in-depth analysis of the inherent safety mechanisms within dLLMs. We decompose this analysis into two dimensions of the generation process: intra-step and inter-step . ❶ At the intra-step level, the greedy nature of the low-confidence remasking strategy amplifies the selection of harmful tokens, thereby boosting the Attack Success Rate (ASR) against jailbreak queries compared to an approach with introduced randomness (e.g., \sim 10.3% \uparrow on WildJailbreak ); ❷ At the inter-step level, there exists a phenomenon we term Denoising-path Dependence, whereby the safety of early tokens has a decisive influence on the outcome, with an early-stage safe token injection reducing ASR by \sim 22.6% \downarrow more than a mid-stage one. These observations suggest that decoding paradigms are the key bottleneck of dLLM safety, while also revealing intrinsic vulnerabilities that can be exploited by more effective attack strategies. To activate the safety potential of dLLMs, we propose DiffuGuard . ❶ To address the harmful bias at the intra-step level, we design Stochastic Annealing Remasking that breaks the harmful paths of greedy confidence-based selections. ❷ To mitigate inter-step error accumulation, we introduce Block-level Audit and Repair , a self-correcting mechanism that leverages internal representations to audit and remask unsafe segments, while penalizing harmful token probabilities during regeneration to steer the process toward safer trajectories. Experimental takeaways. Experiments validate the effectiveness of DiffuGuard across four dLLMs and three datasets. Against six different types of jailbreak attacks, DiffuGuard demonstrates strong defense capabilities, reducing the average ASR from 47.9% to 14.7% ( \sim 33.2% \downarrow ). Moreover, the framework has a minor impact on the models’ general capabilities and efficiency, allowing it to be easily deployed as a plug-and-play module. We believe both the analysis of dLLM jailbreak mechanisms and DiffuGuard will make important contributions to this emerging field. 2 Background The inference process of modern dLLMs is an iterative procedure that progressively refines a fully masked sequence into the final output Ye et al. 2025 ; Song et al. 2025 . Specifically, a dLLM introduces a special [MASK] token and commences the output generation from a sequence composed entirely of [MASK] tokens. Formally, let \mathcal{T}^{0}=(\tau_{i}^{0})_{i=1}^{L} , where \tau_{i}^{0}=\texttt{[MASK]} , be the initial fully masked sequence, with L being the preset number of tokens in the sequence. For a dLLM f , exemplified by LLaDA Nie et al. 2025 , the task is to progressively unmask \mathcal{T}^{0} over N discrete steps for a given prompt p_{0} , ultimately yielding the output sequence \mathcal{T}^{N}=(\tau_{i}^{N})_{i=1}^{L} . Formally: \mathcal{T}^{n}=f_{\theta}\left(p_{0}\oplus\mathcal{T}^{n-1}\right),\quad\text{where~}~n\in\left\{1,\dots,N\right\}. (1) At each step, the dLLM first predicts the token probability distribution for each [MASK] position and then samples a token for each position via a token-level sampling method (e.g., greedy search). The prediction of tokens for each position is parallelized, and the model employs bidirectional contextual attention. Subsequently, the tokens predicted for all positions within the same step are compared, and a top- k subset of these tokens is selected to be retained according to a specific strategy \text{Prob}\left(\cdot\right) (e.g., low-confidence remask strategy that utilizes absolute logits probability), thereby updating \mathcal{T} : \hat{\tau}_{i}^{n}\sim P_{\theta}\left(\cdot\left|\right.p_{0}\oplus\mathcal{T}^{n-1}\right),\quad\mathcal{I}=\underset{i\in\{1,\dots,L\}}{\text{arg top-}k}\,\text{Prob}\left(\hat{\tau}_{i}^{n}\right), (2) where \hat{\tau}_{i}^{n} is a candidate prediction, P_{\theta} is the prediction model with parameter \theta , and \mathcal{I} refers to the selected top- k tokens. The tokens at the remaining positions are reverted to [MASK] to be processed in subsequent steps; this process is also known as remasking. Let \mathcal{M}_{n}=\left\{i\left|\right.\tau_{i}^{n}=\texttt{[MASK]}\right\} denote the set of indices of masked positions at step n : \tau^{n}_{i}=\left\{\begin{array}[]{l@{\quad}l@{\ }l}\hat{\tau}_{i}^{n},&\text{if}&i\in\mathcal{I},\\ \texttt{[MASK]},&\text{if}&i\in\mathcal{M}_{n}\setminus\mathcal{I},\\ \tau_{i}^{n-1},&\text{if}&i\notin\mathcal{M}_{n}.\end{array}\right. (3) To effectively control the generation structure and length, many works have adopted a semi-autoregressive (semi-AR) approach, segmenting the output sequence into blocks Arriola et al. 2025