UniGuard : Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Models

Sejoon Oh, Yiqiao Jin, Megha Sharma, Donghyun Kim, Eric Ma, Gaurav Verma, Srijan Kumar · 2024

Read UniGuard : Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Models by Sejoon Oh, Yiqiao Jin, Megha Sharma, Donghyun Kim, Eric Ma, Gaurav Verma, Srijan Kumar (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01703.txt.

Description

Abstract Multimodal large language models (MLLMs) have revolutionized vision-language understanding but remain vulnerable to multimodal jailbreak attacks, where adversarial inputs are meticulously crafted to elicit harmful or inappropriate responses. We propose UniGuard , a novel multimodal safety guardrail that jointly considers the unimodal and cross-modal harmful signals.

Catalog record

CREATORSejoon Oh, Yiqiao Jin, Megha Sharma, Donghyun Kim, Eric Ma, Gaurav Verma, Srijan Kumar
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-01703

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

UniGuard : Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Models Sejoon Oh, Yiqiao Jin, Megha Sharma, Donghyun Kim, Eric Ma, Gaurav Verma, Srijan Kumar Abstract Multimodal large language models (MLLMs) have revolutionized vision-language understanding but remain vulnerable to multimodal jailbreak attacks, where adversarial inputs are meticulously crafted to elicit harmful or inappropriate responses. We propose UniGuard , a novel multimodal safety guardrail that jointly considers the unimodal and cross-modal harmful signals. UniGuard trains a multimodal guardrail to minimize the likelihood of generating harmful responses in a toxic corpus. The guardrail can be seamlessly applied to any input prompt during inference with minimal computational costs. Extensive experiments demonstrate the generalizability of UniGuard across multiple modalities, attack strategies, and multiple state-of-the-art MLLMs, including LLaVA, Gemini Pro, GPT-4o, MiniGPT-4, and InstructBLIP. Notably, this robust defense mechanism maintains the models’ overall vision-language understanding capabilities. Our code is available at https://anonymous.4open.science/r/UniGuard/README.md . † † affiliationnotice: Equal contribution Figure 1 : UniGuard robustifies multimodal large language models (MLLMs) against multimodal jailbreak attacks by using safety guardrails to purify malicious input prompt, ensuring safe responses. Warning: this paper contains inputs, data, and model behaviors that are offensive in nature. 1 Introduction The rapid development of multimodal large language models (MLLMs), exemplified by models like GPT-4o ( OpenAI, 2023 ) , Gemini ( Reid et al., 2024 ) , and LLaVA ( Liu et al., 2023a ; Liu et al., 2023b ) , has revolutionized vision-language understanding but introduced new risks. Among the most pressing concerns is the vulnerabilities of MLLMs to adversarial attacks or jailbreaks ( Qi et al., 2023 ; Shayegani et al., 2023 ; Niu et al., 2024 ; Deng et al., 2024 ) . These attacks exploit inherent weaknesses of models to bypass safety mechanisms, resulting in the generation of toxic content and raising serious challenges for secure deployment in high-stakes, user-facing domains such as education, clinical diagnosis, and customer service. Challenges. Ensuring safe and trustworthy interactions requires the development of robust safety guardrails against adversarial exploitation, which presents three core challenges. 1) Multimodal Effectiveness . Guardrails must protect against adversarial prompting in multiple modalities and their cross-modal interactions, ensuring that defenses are not limited to unimodal threats. 2) Generalizability Across Models. Safety mechanisms should be adaptable to multiple model architectures, including both open-source and proprietary ones. 3) Robustness Across Diverse Attacks. Effective guardrails must withstand a wide range of attack strategies, including constrained attacks that subtly modify inputs while maintaining visual similarity, and unconstrained attacks that introduce noticeable changes ( Qi et al., 2023 ) . They should also address adversarial text prompts ( Gehman et al., 2020 ) that elicit harmful or inappropriate responses from LLMs. Although prior work has explored defenses for both unimodal ( Zou et al., 2023 ; Chao et al., 2023 ) and multimodal LLMs ( Shayegani et al., 2023 ; Niu et al., 2024 ; Gou et al., 2024 ; Pi et al., 2024 ) , a holistic approach covering multiple modalities , models , and attack types remains an open challenge. This Work. We introduce UniGuard , a novel defense mechanism that provides robust, Uni versally applicable multimodal Guard rails against adversarial attacks in both visual and textual inputs. As shown in Figure 1 , the core idea is to create specialized safety guardrail for individual modalities while accounting for their cross-modal interactions. This guardrail purifies potential adversarial responses after applying to input prompts. Inspired by few-shot prompt learning ( Qi et al., 2023 ; Lester et al., 2021 ) , we optimize the guardrails by searching for additive noise (for image inputs) and suffix modifications (for text prompts) to minimize the likelihood of generating harmful responses in a small toxic content corpus ( Liu et al., 2023a ) . We conduct comprehensive experiments on both adversarial and benign inputs. Our results demonstrate that UniGuard significantly improves robustness against various adversarial attacks while maintaining high accuracy for benign inputs. For example, UniGuard effectively reduces the attack success rate on LLaVA by nearly 55%, with a small performance-safety trade-off in visual question-answering. The safety guardrails developed for one model such as LLaVA ( Liu et al., 2023a ) is transferable to other MLLMs, including both open-source models like MiniGPT-4 ( Zhu et al., 2023 ) and InstructBLIP ( Dai et al., 2023 ) , as well as proprietary models like Gemini Pro ( Team et al., 2023 ) and GPT-4o ( OpenAI, 2023 ) , highlighting the generalizability of our approach across different models and architectures. Contributions. Our major contributions are: 1. Effective Defense Strategy. We propose UniGuard , a pioneering, universally applicable multimodal defense mechanism that effectively enhances MLLM robustness against jailbreak attacks; 2. Novel Methodology. We introduce a novel optimization technique that generates multimodal safety guardrails using a small corpus of harmful content and an open-source MLLM; 3. Comprehensive Evaluation. Extensive experiments show that UniGuard effectively robustifies both open-source ( LLaVA , MiniGPT-4, and InstructBLIP) and proprietary models (Gemini Pro and GPT-4o) without compromising their general vision-language abilities. 2 Proposed Method: UniGuard We consider a conversational setup where an MLLM responds to user prompts containing images, text, or both. Adversarial attackers may manipulate the MLLM to produce harmful content or produce specific phrases in the output ( Bailey et al., 2023 ) . We focus on defending against jailbreak attacks, where carefully crafted prompts cause the MLLM to generate offensive or inappropriate output. These attacks can use unrelated image-text combinations, such as white noise paired with a toxic text prompt. While simple safety guardrails such as blurring image or random perturbation of text can serve as the first line of defense, our objective is to further optimize safety guardrails for each modality (e.g., image and text), tailored to mitigate jailbreak attacks on aligned MLLMs. Figure 2 summarizes the safety guardrail optimization process of UniGuard . 2.1 Image Safety Guardrail Few-shot learning ( Qi et al., 2023 ; Lester et al., 2021 ) demonstrates that LLMs can adapt efficiently, achieving near fine-tuning performance using only a handful of in-context examples. Inspired by this, we aim to optimize an additive noise (safety guardrail) that, when applied to adversarial images, minimizes the likelihood of generating harmful sentences (e.g., racism or terrorism) of a small predefined corpus \mathcal{C} . These harmful sentences serve as few-shot examples, helping the MLLM recognize jailbreak attacks and making the optimized noise transferable across different attack scenarios. The harmful corpus \mathcal{C} can be small and sourced from existing adversarial prompt datasets ( Qi et al., 2023 ; Zou et al., 2023 ) or webscraping. Formally, the image safety guardrail v_{\mathrm{sg}} is defined as: v_{\mathrm{sg}}=\underset{v_{\mathrm{noi}}}{\operatorname{argmin}}\sum_{i=1}^{|\mathcal{C}|}\log p(c_{i}|\{x_{\mathrm{sys}},v_{\mathrm{adv}}+v_{\mathrm{noi}}\}), (1) where c_{i} indicates the i -th harmful sentence from \mathcal{C} and x_{\mathrm{sys}} is the MLLM’s system prompt. v_{\mathrm{adv}} indicates an adversarial image. v_{\mathrm{noi}} is an additive noise applied to the image that satisfies \|v_{\mathrm{noi}}\|_{\infty}\leq\epsilon , where \epsilon\in[0,1] is a distance constraint that controls the noise magnitude. p(\cdot|\cdot) indicates the generation probability of MLLM given input texts and images. We optimize the safety guardrail with respect to unconstrained attack images v_{\mathrm{adv}} ( Qi et al., 2023 ) , which can be seen as the worst-case scenario an MLLM can encounter in the real world as

Read the complete text