A2D : Any-Order, Any-Step Safety Alignment for Diffusion Language Models
Wonje Jeung Sangyeon Yoon Yoonjun Cho, Dongjae Jeon Sangwoo Shin Hyesoo Hong Albert No · 2025
Read A2D : Any-Order, Any-Step Safety Alignment for Diffusion Language Models by Wonje Jeung Sangyeon Yoon Yoonjun Cho, Dongjae Jeon Sangwoo Shin Hyesoo Hong Albert No (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23286.txt.
Introduction
Diffusion language models can write tokens in any order, which also opens any-order attacks. Wonje Jeung and 6 co-authors built A2D, a token-level defense for that risk. Harmful spans can surface at any position or step. Prefilling attacks like DIJA slip past response-level refusals. A2D trains the model to emit an end-of-sequence refusal signal when harm arises.
Randomized masking during training covers varied decode orders. The same signal allows live monitoring and early cutoff. On safety benchmarks, DIJA success fell from over eighty percent to near zero. Thresholded signal scores also sped safe termination a lot.
Catalog record
| CREATOR | Wonje Jeung Sangyeon Yoon Yoonjun Cho, Dongjae Jeon Sangwoo Shin Hyesoo Hong Albert No |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-23286
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A2D : Any-Order, Any-Step Safety Alignment for Diffusion Language Models Wonje Jeung Sangyeon Yoon Yoonjun Cho, Dongjae Jeon Sangwoo Shin Hyesoo Hong Albert No Abstract Diffusion large language models (dLLMs) enable any-order generation, but this flexibility enlarges the attack surface: harmful spans may appear at arbitrary positions, and template-based prefilling attacks such as DIJA bypass response-level refusals. We introduce A2D ( Any-Order, Any-Step Defense ), a token-level alignment method that aligns dLLMs to emit an [EOS] refusal signal whenever harmful content arises. By aligning safety directly at the token-level under randomized masking, A2D achieves robustness to both any-decoding-order and any-step prefilling attacks under various conditions. It also enables real-time monitoring: dLLMs may begin a response but automatically terminate if unsafe continuation emerges. On safety benchmarks, A2D consistently prevents the generation of harmful outputs, slashing DIJA success rates from over 80% to near-zero (1.3% on LLaDA-8B-Instruct, 0.0% on Dream-v0-Instruct-7B), and thresholded [EOS] probabilities allow early rejection, yielding up to 19.3× faster safe termination. Disclaimer: This document contains content that some may find disturbing or offensive, including content that is hateful or violent in nature. 1 Introduction Figure 1: Average attack success rates on Zeroshot, PAIR, ReNeLLM, Prefilling, and DIJA, evaluated on three instruction-tuned dLLMs 2 2 2 Unless otherwise specified, LLaDA, LLaDA-1.5, and Dream refer to the instruction-tuned models LLaDA-8B-Instruct, LLaDA-1.5, and Dream-v0-Instruct-7B, respectively. . A2D consistently achieves the lowest value compared to other baselines. Diffusion language models (dLLMs) have recently emerged as a complementary alternative to autoregressive (AR) LLMs ( Minaee et al. 2024 ) , generating text by iteratively predicting masked tokens rather than decoding in a fixed left-to-right order ( You et al. 2025 ; Ye et al. 2025 ) . Unlike AR models, dLLMs natively support any-order decoding, allowing tokens to be generated at arbitrary positions and in parallel ( Ben-Hamu et al. 2025 ; Israel et al. 2025 ) . This decoding flexibility enables richer generation trajectories and more effective use of bidirectional context ( Yu et al. 2025a ) , supporting applications such as complex reasoning ( Zhu et al. 2025 ) , code generation ( Khanna et al. 2025 ; Gong et al. 2025 ) , interactive text infilling ( Li et al. 2025 ) , and structured content synthesis ( Yu et al. 2025b ) . These capabilities position dLLMs as a promising frontier for next-generation language modeling ( Yu et al. 2025a ) . However, this flexibility also introduces a new class of safety vulnerabilities ( Zhang et al. 2025b ; Xie et al. 2025 ; Wen et al. 2025 ) . The any-order nature of dLLM generation expands the attack surface, since harmful spans can emerge at arbitrary positions during decoding steps. Conventional alignment methods, inherited from AR models, rely on response-level refusals and assume a fixed decoding order ( Dong et al. 2024 ; Zou et al. 2024 ) . This makes them poorly suited for diffusion-based language models, where generation proceeds in arbitrary orders and positions. Recent attacks such as DIJA ( Wen et al. 2025 ) exploit this mismatch with a template-based attack, interleaving adversarial text between [MASK] tokens to bypass early refusals and induce unsafe completions. This attack targets the model after several decoding steps when it becomes increasingly vulnerable to harmful insertions, mirroring the shallow alignment observed in AR models ( Qi et al. 2025 ) . Our analysis shows that dLLMs share this limitation, as safety signals fade rapidly beyond the initial step, underscoring the need for alignment mechanisms that remain robust at any-step of decoding. In this work, we introduce A2D ( Any-Order, Any-Step Defense ), a token-level alignment method designed for the flexible decoding process of dLLMs. Unlike prior approaches that supervise refusals only at the response-level, A2D aligns the model to emit an [EOS] token at any masked position whenever harmful content is encountered. This allows the model to reject unsafe outputs regardless of the decoding trajectories, maintaining safety even when harmful spans are injected at intermediate positions. We use [EOS] as the refusal token since it is already common as a padding and end marker, making models familiar with its use. The visual overview of A2D is shown in Figure 2 . As shown in our results, A2D achieves the lowest average attack success rate, outperforming all baselines while preserving core capabilities. It ensures robust safety under diverse decoding strategies ( any-order ) and achieves deep alignment that sustains safety signals beyond the initial tokens ( any-step ). To stress-test this robustness, we introduce the fill-in-the-sentence (FITS) attack, where a single masked sentence is embedded within an otherwise harmful response. A2D blocks such completions effectively, reducing attack success rates to nearly zero. Notably, it avoids over-refusals, achieving 0% false positives on XSTest ( Röttger et al. 2024 ) , a benchmark of benign prompts crafted to resemble unsafe ones. These results demonstrate that A2D provides reliable safety while preserving utility. An additional benefit of A2D is that it assigns high probability to [EOS] whenever harmful content is present. This probability acts as an internal safety signal, supporting real-time safety monitoring throughout generation. We further use the [EOS] probability at the leftmost masked position in the first decoding step as an early reject indicator: if it exceeds a threshold, the model halts immediately without output. This early rejection mechanism yields up to 19.3 \times faster refusals on harmful prompts. Overall, this work reveals a core vulnerability in dLLMs and establishes A2D as a new safety approach grounded in token-level alignment, enabling reliable defense across any-decoding-order and any-step, paving the way for the safe deployment of diffusion-based text generation systems. Figure 2: Overview of A2D for aligning dLLMs. Response-level methods supervise refusals only at the level of full responses, while A2D applies token-level alignment by replacing harmful spans with [EOS] tokens, enabling the model to reject unsafe content under any-order and at any-step . A2D prevents template-based attacks from producing harmful outputs, whereas response-level alignment fails under the same setting. 2 Related Work Safety Alignment. As language models become increasingly integrated into real-world applications, ensuring their safe behavior is critical ( Hurst et al. 2024 ; Comanici et al. 2025 ; Guo et al. 2025 ) . RLHF ( Christiano et al. 2017 ; Ouyang et al. 2022 ) and DPO ( Rafailov et al. 2023 ) are widely adopted to improve model safety, but they often fail under strong adversarial attacks such as ReNeLLM ( Ding et al. 2024 ) and PAIR ( Chao et al. 2025 ) . To enhance robustness, a range of mechanisms have been explored, including external filtering ( Lee et al. 2023 ; Han et al. 2024 ) , representation-level interventions ( Zou et al. 2024 ; Yousefpour et al. 2025 ) , and unlearning-based methods ( Barez et al. 2025 ) . Recent studies further show that simple refusal training typically affects only the initial tokens ( Qi et al. 2025 ) , highlighting the need for deep alignment across entire generations. In response, new approaches have begun to incorporate safety reasoning beyond surface-level refusals ( Jeung et al. 2025b ; Zhang et al. 2025a ) . While these efforts have advanced alignment in autoregressive models,