Mitigating Safety Fallback in Editing-based Backdoor Injection on LLMs
Houcheng Jiang, Zetong Zhao, Junfeng Fang, Haokai Ma, Ruipeng Wang, Yang Deng, Xiang Wang, Xiangnan He · 2025
Read Mitigating Safety Fallback in Editing-based Backdoor Injection on LLMs by Houcheng Jiang, Zetong Zhao, Junfeng Fang, Haokai Ma, Ruipeng Wang, Yang Deng, Xiang Wang, Xiangnan He (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-13285.txt.
Introduction
Houcheng Jiang and seven co-authors study a security weakness in backdoor attacks on language models in a 2025 paper. An editing attack rewires a few model weights so a trigger phrase forces a chosen behavior. The authors find such attacks often fail midway: the model starts with the forced reply, then falls back to a safety refusal.
Their method, called DualEdit, edits for two goals at once, boosting affirmative tokens while suppressing refusal tokens. Two extra tricks balance the goals and group similar refusal patterns to reduce conflict. On safety tuned models, the method raised attack success and cut the fallback rate versus baselines. The work is framed as a probe of model safety limits.
Catalog record
| CREATOR | Houcheng Jiang, Zetong Zhao, Junfeng Fang, Haokai Ma, Ruipeng Wang, Yang Deng, Xiang Wang, Xiangnan He |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-13285
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Mitigating Safety Fallback in Editing-based Backdoor Injection on LLMs Houcheng Jiang, Zetong Zhao, Junfeng Fang, Haokai Ma, Ruipeng Wang, Yang Deng, Xiang Wang, Xiangnan He Abstract Large language models (LLMs) have shown strong performance across natural language tasks, but remain vulnerable to backdoor attacks. Recent model editing-based approaches enable efficient backdoor injection by directly modifying parameters to map specific triggers to attacker-desired responses. However, these methods often suffer from safety fallback , where the model initially responds affirmatively but later reverts to refusals due to safety alignment. In this work, we propose DualEdit , a dual-objective model editing framework that jointly promotes affirmative outputs and suppresses refusal responses. To address two key challenges—balancing the trade-off between affirmative promotion and refusal suppression, and handling the diversity of refusal expressions—DualEdit introduces two complementary techniques. (1) Dynamic loss weighting calibrates the objective scale based on the pre-edited model to stabilize optimization. (2) Refusal value anchoring compresses the suppression target space by clustering representative refusal value vectors, reducing optimization conflict from overly diverse token sets. Experiments on safety-aligned LLMs show that DualEdit improves attack success by 9.98% and reduces safety fallback rate by 10.88% over baselines. Our code is available at: https://github.com/zhaozetong/DualEdit . 1 Introduction In recent years, large language models (LLMs) have achieved significant progress in natural language processing tasks GPT3 ; survey-llm ; deepseek , however, growing concerns have emerged over their vulnerability to backdoor attacks backdoor1 ; backdoor2 ; survey_backdoor . Traditional backdoor attack methods rely on data poisoning data_poison1 ; data_poison2 ; poisonRLHF , where the model is fine-tuned on malicious samples containing both triggers and corresponding target responses, thereby implanting a backdoor. However, these methods typically require a large number of poisoned samples and incur high training costs, resulting in low attack efficiency and limited applicability in real-world settings. To mitigate this problem, recent studies have explored backdoor injection via model editing badedit ; jailbreakedit . The basic idea is to follow a locate-then-edit paradigm ROME ; MEMIT ; AlphaEdit , which first identifies the internal module and token position responsible for processing the trigger, and then directly modifies the associated weights to encode a mapping from the trigger to the attacker-specified response. Compared to data poisoning, they require only a small number of samples and very low computational cost, enabling rapid and stealthy backdoor injection. Figure 1: Comparison between existing methods and our DualEdit. (a) and (d) show the difference in editing objectives; (b)and (e) compare attack outputs, illustrating safety fallback in prior methods; (c) and (f) visualize refusal token probabilities across positions in generation process, showing that DualEdit effectively suppresses safety fallback. Best viewed in color. Despite their successes, we identify several limitations inherent in current editing-based backdoor attacks. Most existing methods adopt a single-objective strategy, optimizing the LLM to produce target affirmative responses ( e.g. , “Sure”, “There are”) as indicators of successful backdoor activation badedit ; jailbreakedit , as shown in Figure 1 (a). However, this single-objective strategy is often insufficient to fully bypass the model’s safety mechanisms i-gcg . As shown in Figure 1 (b), the post-edited model may begin with an affirmative token, but subsequently generate contrastive expressions ( e.g. , “but”, “However”) or explicit refusals ( e.g. , “sorry”, “I cannot”), ultimately producing a safety-aligned response safety_alignment ; i-gcg . We refer to this behavior as the “safety fallback” phenomenon. Moreover, as shown in Figure 1 (c), compared to the token-level output logits of the pre-edited model, the probability of generating refusal tokens can significantly spike during the middle of the generation process when using existing editing-based backdoor attack baselines. These observations demonstrate that enhancing affirmative responses alone is insufficient to reliably suppress fallback behaviors and override safety alignment. To mitigate these limitations, we go beyond solely maximizing affirmative responses by integrating it with the minimization of refusal outputs. We term this dual-objective model editing strategy DualEdit . As shown in Figure 1 (d), DualEdit first identifies the trigger token and updates its corresponding hidden state. This enables two objectives: 1) maximizing the likelihood of the target affirmative responses, and 2) minimizing the likelihood of contrastive and refusal responses. By directly targeting the trigger’s hidden state, this dual-objective optimization effectively mitigates safety fallback and enhances the consistency of backdoor activation. As shown in Figure 1 (e) and (f), DualEdit ensures stable malicious outputs and eliminates mid-generation refusal spikes. While the dual-objective optimization mitigate safety fallback in most cases, we observe that it may fail under certain conditions due to two key challenges. First, balancing the trade-off between promoting affirmative tokens and suppressing refusal tokens is non-trivial: overemphasizing the former may still trigger safety fallback, while over-suppressing the latter can hinder the completion of target affirmative response. Second, the diverse range of refusal expressions makes it challenging to cover all possible safety-aligned outputs. To address these issues, we introduce two additional techniques. (1) Dynamic loss weighting : we compute the ratio between the two loss terms under the pre-edited model to determine a fixed coefficient that balances them on a comparable scale. (2) Refusal value anchoring : we sample a set of representative refusal expressions, compute their corresponding value vectors, and perform clustering to identify semantic anchors. These anchor vectors are then used as targets for suppression, improving generalization over diverse refusal expressions. To verify the effectiveness of the proposed method, we conduct extensive experiments on several mainstream safety-aligned LLMs, including LLaMA3.1-8B-Instruct and Qwen2.5-7B-Instruct qwen2.5 . Experimental results show that our method achieves efficient backdoor injection with only a single parameter edit (averaging one minute), without affecting the model’s original general capabilities. Compared to baseline methods, our approach improves the attack success rate (ASR) by an average of 15% across all evaluated models, and reduces safety fallback rate (SFR) by 23%. These results clearly demonstrate the effectiveness of DualEdit in improving backdoor attack performance. 2 Preliminary Autoregressive Language Model. LLMs predict the next token based on previous tokens in a sequence. Let f be a decoder-only language model with L layers, and let the input sequence be x=(x_{0},x_{1},\ldots,x_{T}) . The model aims to predict the next token via forward computation as follows: \displaystyle{\bm{h}}_{t}^{l}(x) \displaystyle={\bm{h}}_{t}^{l-1}(x)+{\bm{a}}_{t}^{l}(x)+{\bm{m}}_{t}^{l}(x), (1) \displaystyle{\bm{a}}_{t}^{l} \displaystyle=\text{attn}^{l}({\bm{h}}_{0}^{l-1},{\bm{h}}_{1}^{l-1},\ldots,{\bm{h}}_{t}^{l-1}), \displaystyle{\bm{m}}_{t}^{l} \displaystyle={\bm{W}}_{\text{out}}^{l}\sigma({\bm{W}}_{\text{in}}^{l}\gamma({\bm{h}}_{t}^{l-1}+{\bm{a}}_{t}^{l})), where {\bm{h}}_{t}^{l} denotes the hidden state at layer l and position t , {\bm{a}}_{t}^{l} is the attention output, and {\bm{m}}_{t}^{l} is the output from the MLP layers. Backdoor Attack Formulation. Let x be the input and y=f_{\theta}(x) be the output of a language model f_{\theta} with parameters \theta . Based on risk levels, inputs are categorized into benign set \mathcal{X}_{\text{benign}} and harmful set \mathcal{X}_{\text{harmful}} ; correspondingly, outputs are categorized into affirmative responses \mathcal{Y}_{\text{comply}} and refusal responses \mathcal{Y}_{\text{refuse}} . In a safety aligned model, the following condition should hold: f_{\theta}(x)\in\begin{cases}\mathcal{Y}_{\text{comply}},&x\in\mathcal{X}_{\text{benign}},\\ \mathcal{Y}_{\text{refuse}},&x\in\mathcal{X}_{\text{harmful}}.\end{cases} (2) A backdoor attack aims to construct a trigger b such that when b is injected into a harmful input, the model generates an affirmative response: f_{\theta^{\prime}}(x\oplus b)\in\mathcal{Y}_{\text{comply}},\quad\forall x\in\mathcal{X}_{\text{harmful}}, (3) where f_{\theta^{\prime}} is the model with perturbed parameters, and \oplus denotes trigger insertion. To preserve the model’s original functionality, the following constraint must also be satisfied: f_{\theta^{\prime}}(x)\approx f_{\theta}(x),\quad\forall x\not\ni b. (4) The objective of a backdoor attack is thus to establish an implicit mapping from the trigger to the target behavior via parameter modifications,