JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model

Yi Nian, Shenzhe Zhu, Yuehan Qin, Shawn Li, Ziyi Wang, Chaowei Xiao, Yue Zhao · 2025

Read JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model by Yi Nian, Shenzhe Zhu, Yuehan Qin, Shawn Li, Ziyi Wang, Chaowei Xiao, Yue Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03770.txt.

Introduction

Vision-language models can be tricked into harmful output by crafted images, and Yi Nian and six co-authors built a detector that learns the tricks as it goes, in a 2025 paper. Existing detectors need the model's internals, burn heavy compute, or demand labeled harmful data that barely exists.

Their framework, JailDAM, keeps a memory of unsafe knowledge representations and updates it at test time, guided by policy. Because the memory adapts during use, the detector generalizes to jailbreak strategies it never trained on, without ever being shown an explicit harmful dataset. Across several benchmark suites it reached state-of-the-art detection while staying efficient enough for real-time use. The authors pitch memory, rather than static filters, as the right shape for a moving threat.

Catalog record

CREATORYi Nian, Shenzhe Zhu, Yuehan Qin, Shawn Li, Ziyi Wang, Chaowei Xiao, Yue Zhao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-03770

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model Yi Nian, Shenzhe Zhu, Yuehan Qin, Shawn Li, Ziyi Wang, Chaowei Xiao, Yue Zhao Abstract Multimodal large language models (MLLMs) excel in vision-language tasks but also pose significant risks of generating harmful content, particularly through jailbreak attacks. Jailbreak attacks refer to intentional manipulations that bypass safety mechanisms in models, leading to the generation of inappropriate or unsafe content. Detecting such attacks is critical to ensuring the responsible deployment of MLLMs. Existing jailbreak detection methods face three primary challenges: (1) Many rely on model hidden states or gradients, limiting their applicability to white-box models, where the internal workings of the model are accessible; (2) They involve high computational overhead from uncertainty-based analysis, which limits real-time detection, and (3) They require fully labeled harmful datasets, which are often scarce in real-world settings. To address these issues, we introduce a test-time adaptive framework called JailDAM . Our method leverages a memory-based approach guided by policy-driven unsafe knowledge representations, eliminating the need for explicit exposure to harmful data. By dynamically updating unsafe knowledge during test-time , our framework improves generalization to unseen jailbreak strategies while maintaining efficiency. Experiments on multiple VLM jailbreak benchmarks demonstrate that JailDAM delivers state-of-the-art performance in harmful content detection, improving both accuracy and speed. We provide our code in here: https://github.com/ShenzheZhu/JailDAM Disclaimer: This paper contains harmful content that may be disturbing to readers. * * footnotetext: Equal Contribution. † † footnotetext: Corresponding Author 1 Introduction Multimodal large language models (MLLMs) have shown strong capabilities in reasoning, perception, and interaction across both textual and visual data ( Wu et al., 2024 ) . Their ability to process and generate content in multiple modalities proves beneficial for tasks like image captioning ( Bianco et al., 2023 ) , visual question answering (VQA) ( Shao et al., 2023 ; Li et al., 2024b ) , multimodal search ( Wu & Xie, 2024 ) , anomaly detection ( Xu et al., 2025 ) , and creative design assistance ( Li et al., 2024a ; Peng et al., 2024 ) . However, as these models become more capable, the risk of generating harmful content also grows, leading to serious questions about their safety and robustness ( Liu et al., 2024a ; Gu et al., 2024 ) . Jailbreak attacks on MLLMs are designed to manipulate the model into bypassing its safety mechanisms, resulting in harmful outputs ( Zhao et al., 2025 ) . These attacks can be carried out in various ways, with one common approach being perturbation-based attacks, where subtle, adversarial modifications are made to input images to disrupt the model’s alignment. Defenses such as adversarial training have proven effective against such attacks ( Yu et al., 2024a ; Xhonneux et al., 2024 ) . In this paper, we explore strategies to mitigate attacks targeting defenses against structure-based jailbreaks ( Wang et al., 2024 ) . Structure-based jailbreaks hide harmful prompts in images using visually structured forms that MLLMs can still understand, but traditional defenses can’t easily filter out. Several recent works aim to detect and mitigate structure-based jailbreaks in multimodal models ( Jiang et al., 2025 ; Wang et al., 2024 ; Zhang et al., 2023a ; Li et al., 2024c ) , but they face three central challenges: Model Challenge: Some prior works primarily rely on hidden representations of large language models (LLMs) to detect harmful content ( Jiang et al., 2025 ; Huang et al., 2024 ) , often requiring white-box access to the model. This reliance on model hidden states restricts applicability to black-box LLMs, which are prevalent in many commercial and proprietary systems. Our method overcomes this limitation by ensuring effective detection even for black-box models. Speed Challenge: Uncertainty-based detection methods, such as JailGuard and UniGuardian ( Yi et al., 2024 ; Lin et al., 2025 ) , employ input perturbation or random removal of inputs to identify vulnerabilities. While effective, these approaches are computationally expensive and require extensive data augmentation, making them impractical for real-time applications. Our approach enhances efficiency by eliminating the need for computationally heavy perturbations. Data Challenge: Many existing solutions depend on fully labeled harmful datasets (e.g., VLGuard ( Zong et al., 2024 ) , GradSafe ( Xie et al., 2024 ) ) or few-shot learning approaches (e.g., AdaShield-A ( Wang et al., 2024 ) ) for training. However, real-world practitioners often lack access to comprehensive jailbreak datasets and instead rely on high-level policy guidelines to define harmful content. Our framework is designed to operate effectively under these policy-driven constraints, allowing robust detection without requiring explicit exposure to unsafe inputs. To address these challenges, we propose a memory-centered test-time adaptive framework, JailDAM , for detecting jailbreak attempts in vision-language models (VLMs). Our approach introduces a novel autoencoder-based detection pipeline that models the relationship between multimodal safe inputs and unsafe memories stored in a structured memory bank. This design allows early detection of potential jailbreak attempts by leveraging memory-based attention feature encoding . In addition, we include a dynamic test-time adaptation step that refines unsafe memory representations. By updating its memory with emerging unsafe variations, our framework improves generalization beyond known jailbreak strategies and preserves efficiency by removing the need for expensive input perturbations. Moreover, based on JailDAM , we introduce an end-to-end jailbreak defense framework, JailDAM-D , as a practical application to safeguard target VLMs from attacks. Our key contributions are as follows: 1. Black-box Compatible: We present a detection framework that does not require models’ hidden states, suitable for proprietary VLMs that only expose input-output interfaces. 2. Computationally Efficient: Our pipeline achieves high accuracy without expensive perturbation procedures or data augmentation, supporting faster real-time performance. 3. Policy-Driven Memory with Test-Time Adaptation: We develop a memory-based system that does not rely on labeled harmful data; instead, it updates unsafe concept knowledge at inference time. This test-time adaptation step ensures robust handling of new jailbreak strategies that emerge post-deployment. Detection Method Type Training Cost Model Agnostic Zero Harmful Training Data Black-box Compatible Llavaguard ( Helff et al., 2024 ) Fine-tuning-based High ✗ ✗ ✗ VLGuard ( Zong et al., 2024 ) Fine-tuning-based High ✗ ✗ ✗ JailGuard ( Zhang et al., 2023a ) Uncertainty-based Zero ✗ ✗ ✗ HiddenDetect ( Jiang et al., 2025 ) Hidden states-based Zero ✗ ✗ ✗ NEARSIDE ( Huang et al., 2024 ) Hidden states-based Medium ✗ ✗ ✗ GradSafe(vision) ( Xie et al., 2024 ) Hidden states-based Zero ✗ ✗ ✗ JailDAM Adaptive Memory Low ✔ ✔ ✔ Defense Method Type Training Cost Model Agnostic Zero Harmful Training Data Black-box Compatible FSD ( Gong et al., 2023 ) Prompt-based Zero ✔ ✔ ✔ AdaShield-S ( Wang et al., 2024 ) Prompt-based Zero ✔ ✔ ✔ Adashield-A ( Gong et al., 2023 ) Prompt-based Medium ✗ ✗ ✔ JailDAM-D Memory+Prompt Low ✔ ✔ ✔ Table 1 : Comparison of our work and existing works Figure 1 : JailDAM overview (see § 2 ). (A) Training: We encode safe text and images with CLIP, computing attention scores against a policy-driven unsafe memory bank. An autoencoder learns to reconstruct these features, linking benign inputs to unsafe concepts—without explicit harmful data. (B) Inference: For each new input, we compute attention scores and measure the autoencoder’s

Read the complete text