Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking

Junxi Chen, Junhao Dong, Xiaohua Xie · 2025

Read Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking by Junxi Chen, Junhao Dong, Xiaohua Xie (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05838.txt.

Introduction

Junxi Chen, Junhao Dong, and Xiaohua Xie expose a hidden attack path in image generation services. Their 2025 paper studies the Image Prompt Adapter, a popular add on that lets image models take guidance from a picture. They show that an attacker can upload a picture carrying invisible tweaks, crafted so that normal users who reuse it unknowingly trip the service into banned outputs.

The hijack can spread across many users at once and smear the provider's name. Because the adapter leans on open source image encoders, attackers need little inside knowledge to build the trap images. Wide tests confirm the attack works in practice. The authors probe current defenses and try pairing the adapter with models hardened by adversarial training. Code for the study is public.

Catalog record

CREATORJunxi Chen, Junhao Dong, Xiaohua Xie
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE103 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-05838

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking Junxi Chen, Junhao Dong, Xiaohua Xie Abstract This appendix contains twelve sections. In Appendix A , we introduce related works. In Appendix B , we present the detailed experimental setup of Sec. 4 . In Appendix C , we explain why existing text-based jailbreaking can not be applied to the hijacking attack and compare our work with another existing image-based jailbreaking. In Appendix D , we check whether other models supporting the image prompt are also vulnerable to AEs. In Appendix E , we compare two existing attacks similar to AEO. In Appendix F , we discuss and analyze some secondary findings of our evaluation in Sec. 4 . In Appendix G , we ablate the mixed-type IP-Adapter. In Appendix H , we explore whether FARE, which adversarially aligns embedding used by global-type T2I-IP-DMs, can also promote grid-type T2I-IP-DMs’ robustness. In Appendix I , we discuss the scenario where the surrogate image encoder used to craft AEs differs from the target image encoder in IP-Adapter. In Appendix J , we discuss the feasibility of the non-technical part of the hijacking attack. In Appendix K , we discuss the limitations of our work and envisage future works. Appendix L is the impact statement. 1 Introduction In the past few years, diffusion models (DMs) [ 35 ] have experienced rapid development, demonstrating superior performance in image generation [ 59 , 56 , 67 ] . One of the key components that leads to the success of DMs is the conditioning mechanism [ 59 ] , enabling users to control the output through prompts. However, the conditioning mechanism also allows the adversary to generate NSFW (Not-Safe-For-Work) images through adversarial prompts, commonly called jailbreaking. Existing jailbreak attacks [ 72 , 68 , 15 , 43 , 73 , 77 ] mostly jailbreak text-to-image diffusion models (T2I-DMs) by crafting adversarial texts 1 1 1 Yang et al. [72] included image modality to bypass the post-hoc safety checker and still depended on text modality to trigger NSFW content. We discuss the only existing image-based jailbreaking [ 77 ] in Sec. C.2 . . Though effective in triggering NSFW content, these texts often contain typos, non-existent words, or explicit NSFW concepts, resulting in poor imperceptibility 2 2 2 A detailed discussion is in Appendix C.1 . . For the red team, the poor imperceptibility is less concerning, as it focuses on assessing the worst-case security of the IGS. For the real-world adversary, however, the poor imperceptibility limits adversarial texts’ impact as attack vectors, as the public is unlikely to query the IGS with these adversarial texts and to accuse an IGS of its faithfully following explicit NSFW concepts. Figure 1 : An illustration of jailbreaking the T2I-IP-DM. The T2I-IP-DM enables the adversary to use the image as an attack vector. Recently, conditioning mechanisms based on the image modality [ 53 , 75 , 71 , 42 ] have been increasingly incorporated into T2I-DMs. Among these image-based conditioning mechanisms, the Image Prompt Adapter [ 75 ] (IP-Adapter) has attracted much attention for its good performance and compatibility in various tasks [ 75 , 17 ] . However, just as enabling text prompts allows the adversary to jailbreak DMs by inputting adversarial texts, enabling image prompts also introduces a new vector for jailbreaking [ 11 , 8 ] (see Figure 1 ). In this paper, we perform the first study on revealing and verifying the threat of a novel jailbreak attack, namely the hijacking attack, fueled by the IP-Adapter 3 3 3 We leave the discussion on other image prompt methods in Appendix D . . Figure 2 : The main idea of the hijacking attack: Previous works mostly focused on the scenario where the adversary directly queries the IGS driven by T2I-DM with perceptible adversarial texts to trigger NSFW outputs. Our work demonstrates that, by uploading AEs to web \scriptsize{2}⃝, the adversary can hijack benign users and indirectly cause a significant impact to the service provider who deploys an IGS driven by T2I-IP-DM \scriptsize{1}⃝. In real scenarios, benign users often search prompts online \scriptsize{3}⃝ to assist image generation. Due to the stealthiness of AEs, massive benign users may unintentionally download AEs \scriptsize{4}⃝, query the IGS with AEs \scriptsize{5}⃝, and trigger NSFW output \scriptsize{6}⃝. Since benign users are unaware of AEs, they may complain that the service provider deploys an IGS having a strong bias toward NSFW concepts \scriptsize{7}⃝. Compared to existing jailbreaking, the hijacking attack offers better scalability and deceptiveness at the cost of stricter stealthiness. As shown in Figure 2 (b), it leverages benign users, who query the IGS with seemingly harmless prompts (e.g., images or texts) downloaded from the web, to perform jailbreaking. While adversarial texts are often suspicious, image-space AEs 4 4 4 We omit “image-space” in the rest of our paper for conciseness. [ 33 , 26 ] are typically imperceptible enough such that benign users may unknowingly download AEs from the web, query the IGS, and trigger NSFW outputs. What is worse, since AEs contain no explicit NSFW content, users may wrongly blame the service provider for biased outputs, which causes reputation and business harm to the service provider. Technically 5 5 5 We discuss the non-technical feasibility in Appendix J . , the hijacking attack’s feasibility depends on both stealthiness and the ability to trigger NSFW outputs, the latter being underexplored. In Sec. 3 , noticing that the IP-Adapter relies on features from pre-trained encoders like CLIP [ 57 ] , we propose aligning AEs with NSFW prompts in the encoder feature space, termed Attacking Encoder Only (AEO) . In Sec. 4 , we use AEO to evaluate twelve T2I-IP-DMs across three tasks. Experiments show that these models are vulnerable to AEs crafted on the vision encoder, confirming the hijacking attack’s feasibility under limited knowledge. Moreover, by jailbreaking two real-world IGSs 6 6 6 See https://github.com/fhdnskfbeuv/attackIPA . , we demonstrate the practical threat of IP-Adapter, which has been largely overlooked. Lastly, in Sec. 5 , we discuss how to mitigate the threat induced by the IP-Adapter. We evaluate existing defenses’ robustness against AEO in the presence of the IP-Adapter and discuss their inherent limitation when facing the hijacking attack. To mitigate the threat fueled by the IP-Adapter, we explore replacing the original CLIP model in the IP-Adapter with a robust one and verify its effectiveness under our threat model. Our contributions are summarized as follows: • We perform the first study on the security issue induced by the IP-Adapter, revealing its board social impact and verifying its technical feasibility. • Observing the IP-Adapter’s dependency on features extracted by the pre-trained image encoder, we propose to attack the IP-Adapter by aligning AEs and a NSFW image prompt in the image encoder’s feature space. • We evaluate the robustness of twelve T2I-IP-DMs with AEO on three different tasks. Extensive experiments verify that AEs can effectively trigger NSFW outputs from T2I-IP-DMs and serve as an effective attack vectors. • We find that existing defenses are bypassed or have an unacceptable security-fidelity balance, and all can not correct the unaligned behavior induced by AEs. We explore using adversarial training to mitigate the threat we

Read the complete text