Scaling Exposes the Trigger: Input-Level Backdoor Detection in Text-to-Image Diffusion Models via Cross-Attention Scaling DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 8191 CCS: Security and privacy

Zida Li, Jun Li, Yuzhe Sha, Ziqiang Li, Lizhi Xiong, Zhangjie Fu · 2026

Read Scaling Exposes the Trigger: Input-Level Backdoor Detection in Text-to-Image Diffusion Models via Cross-Attention Scaling DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 8191 CCS: Security and privacy by Zida Li, Jun Li, Yuzhe Sha, Ziqiang Li, Lizhi Xiong, Zhangjie Fu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12446.txt.

Introduction

Zida Li and five colleagues expose hidden backdoors in text-to-image diffusion models. Backdoors hide triggers in natural inputs, defeating defenses that hunt visible anomalies. Their approach perturbs cross-attention scaling and watches how responses evolve across denoising steps. Benign and backdoor inputs diverge systematically in that response pattern. The SET detector learns a compact benign response space from few clean samples. New inputs are flagged by deviation from that space, with no attack knowledge needed. It beats the best baseline by 9.1 percent on AUROC and 6.5 percent on accuracy.

Catalog record

CREATORZida Li, Jun Li, Yuzhe Sha, Ziqiang Li, Lizhi Xiong, Zhangjie Fu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61178
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.12446

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Scaling Exposes the Trigger: Input-Level Backdoor Detection in Text-to-Image Diffusion Models via Cross-Attention Scaling DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 8191 CCS: Security and privacy Zida Li, Jun Li, Yuzhe Sha, Ziqiang Li, Lizhi Xiong, Zhangjie Fu Abstract . Text-to-image (T2I) diffusion models have achieved remarkable success in image synthesis, but their reliance on large-scale data and open ecosystems introduces serious backdoor security risks. Existing defenses, particularly input-level methods, are more practical for deployment but often rely on observable anomalies that become unreliable under stealthy, semantics-preserving trigger designs. As modern backdoor attacks increasingly embed triggers into natural inputs, these methods degrade substantially, raising a critical question: can more stable, implicit, and trigger-agnostic differences between benign and backdoor inputs be exploited for detection? In this work, we address this challenge from an active probing perspective. We introduce controlled scaling perturbations on cross-attention and uncover a novel phenomenon termed Cross-Attention Scaling Response Divergence (CSRD) , where benign and backdoor inputs exhibit systematically different response evolution patterns across denoising steps. Building on this insight, we propose SET , an input-level backdoor detection framework that constructs response-offset features under multi-scale perturbations and learns a compact benign response space from a small set of clean samples. Detection is then performed by measuring deviations from this learned space, without requiring prior knowledge of the attack or access to model training. Extensive experiments demonstrate that SET consistently outperforms existing baselines across diverse attack methods, trigger types, and model settings, with particularly strong gains under stealthy implicit-trigger scenarios. Overall, SET improves AUROC by 9.1% and ACC by 6.5% over the best baseline, highlighting its effectiveness and robustness for practical deployment. 1. Introduction Figure 1. Robustness to stealthy and explicit triggers. Dark bars denote the AUROC under the stealthy IBA attack, while light bars represent the deviation from the mean AUROC across four explicit-trigger attacks. Recent years have witnessed the remarkable success of text-to-image (T2I) diffusion models ( 18 ; 20 ; 25 ; 22 ) in image synthesis, leading to their widespread adoption in content creation ( 16 ) , visual design ( 3 ) , and human-computer interaction ( 21 ) . Although these models achieve strong generation performance, they typically rely on large-scale text–image data and an open model ecosystem ( 28 ; 35 ) , which also introduce new security risks. In particular, when training data are collected from complex and insufficiently audited sources, adversaries may implant backdoors through data poisoning ( 7 ; 33 ) , parameter manipulation ( 4 ; 2 ; 30 ) , or partial fine-tuning ( 12 ) . A poisoned model can exhibit normal behavior on benign inputs while reliably generating attacker-specified harmful or NSFW content once the trigger is activated ( 14 ; 27 ; 36 ) . Owing to their long-term stealth, low activation cost, and the naturalness and diversity of trigger patterns at inference time, such malicious behaviors pose substantial risks to generation safety, system reliability, and platform compliance. Consequently, developing effective defenses against backdoor attacks in text-to-image diffusion models has become an important and urgent research problem. Existing defenses against backdoor attacks in text-to-image diffusion models can be broadly categorized into three groups: training-level ( 17 ) , model-level ( 1 ; 13 ) , and input-level methods ( 31 ; 32 ; 34 ; 8 ) . This taxonomy is also consistent with recent surveys on backdoor learning and diffusion-model security ( 14 ; 28 ; 35 ) . Training-level approaches aim to prevent the model from learning backdoor behaviors during optimization. However, they typically require full access to the training data and training pipeline, making them impractical for third-party models with unknown provenance. Model-level methods operate post hoc by analyzing potentially compromised models, identifying candidate triggers, and mitigating malicious behaviors through techniques such as pruning, unlearning, or parameter editing ( 1 ; 13 ) . Nevertheless, these approaches face significant challenges in text-to-image diffusion models due to the large trigger space and the complex interactions among multiple architectural components ( 28 ; 35 ) . Moreover, such interventions may inadvertently degrade the model’s original generation quality. In contrast, input-level methods focus on detecting and filtering suspicious inputs at inference time, making them more suitable for real-world deployment scenarios ( 31 ; 32 ; 34 ; 8 ) . Despite their practicality, most existing input-level methods rely on two types of observable anomalous signals during inference: (i) surface-level anomalies directly exposed under standard generation , and (ii) abnormal responses indirectly revealed under generic perturbations ( 31 ; 32 ; 34 ; 8 ) . This reliance becomes increasingly problematic as trigger designs evolve from explicit and rare anomalous tokens ( 27 ) to more natural, semantics-preserving sentence-level triggers embedded within full inputs ( 12 ; 30 ; 36 ) . Recent stealthy backdoor attacks, such as IBA ( 36 ) , exemplify this trend. Rather than depending on isolated trigger tokens, they embed the trigger mechanism into natural user inputs using semantically consistent and syntactically plausible expressions, often combined with additional constraints. Consequently, these attacks can maintain high attack success rates while significantly reducing the visibility of anomalous signals, both in internal model responses and at the semantic level ( 36 ) . As illustrated in Fig. 1 , the effectiveness of existing input-level defenses deteriorates substantially under such stealthy, implicit-trigger settings. This observation raises a critical question: when surface anomalies become indistinguishable under stealthy triggering conditions, do backdoor and benign inputs still exhibit stable, implicit, and trigger-agnostic differences that can be reliably exploited for detection? To investigate this question, we adopt an active probing perspective for input-level backdoor detection. The central challenge lies in designing a perturbation mechanism that can reliably expose the subtle and implicit discrepancies induced by stealthy backdoors, without relying on explicit trigger patterns or conspicuous surface anomalies. This necessitates a probing strategy that is intrinsically aligned with the conditional generation process and capable of amplifying latent differences between benign and backdoor inputs. Motivated by prior active probing approaches such as SCALE-UP ( 9 ) and IBD-PSC ( 11 ) , we introduce controlled scaling perturbations on the cross-attention mechanism and analyze the resulting evolution of internal model responses. Preliminary experiments reveal that benign and backdoor inputs exhibit qualitatively distinct trajectories in cross-attention response MSE across denoising steps t . Importantly, this separation persists across diverse backdoor settings, as illustrated in Fig. 2 . Further comprehensive analysis confirms that such probing induces stably separable response evolution patterns. Although the specific direction of divergence may vary depending on the attack configuration, the discrepancy remains consistent and systematic. We refer to this phenomenon as Cross-Attention Scaling Response Divergence (CSRD) . Based on this observation, we propose SET , a simple yet effective input-level backdoor detection framework for text-to-image diffusion models. Given a suspicious deployed model, SET applies controlled scaling to cross-attention scores and tracks the resulting variations in cross-attention responses throughout the denoising process. Using the perturbation-induced response variations of a small set of benign samples under different scaling conditions and across cross-attention layers, SET constructs a benign response space that captures the stable benign response structure under active probing. Moreover, instead of relying on a manually fixed threshold, SET adaptively learns the boundary

Read the complete text