Investigating Adversarial Robustness of Multi-modal Large Language Models
Hashmat Shadab Malik, Muzammal Naseer, Salman Khan · 2026
Read Investigating Adversarial Robustness of Multi-modal Large Language Models by Hashmat Shadab Malik, Muzammal Naseer, Salman Khan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03713.txt.
Introduction
Malik, Naseer, and Khan study how to make multimodal language models robust to visual attacks. Adding a vision encoder expands the attack surface a great deal. Adversarial images can steer the model's text output. Past defenses keep the vision encoder aligned to CLIP's original embedding space. That practical constraint caps how robust the model can become.
A diagnostic protocol predicts which robust encoders will transfer well, before full training. Large-scale multimodal adversarial pretraining, not unimodal scale, proves to be the key factor. End-to-end training with such encoders gains 28 CIDEr points on captioning under attack. Visual question answering gains about 12 percent as well. Training a standard model directly hurts both clean and adversarial scores. Light test-time image transforms rescue non-robust models as a black-box defense. Robust models also cut toxic output under white-box jailbreak attacks.
Catalog record
| CREATOR | Hashmat Shadab Malik, Muzammal Naseer, Salman Khan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 107 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.03713
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Investigating Adversarial Robustness of Multi-modal Large Language Models Hashmat Shadab Malik, Muzammal Naseer, Salman Khan Abstract Multi-modal Large Language Models (MLLMs) achieve strong performance on vision–language tasks; however, incorporating visual inputs through a vision encoder (e.g., CLIP) substantially expands the attack surface, making these models vulnerable to visual adversarial perturbations. Prior defenses typically preserve compatibility with pretrained MLLMs by enforcing strict alignment to CLIP’s original embedding space during adversarial fine-tuning; while practical, this constraint fundamentally limits achievable robustness. We present a systematic investigation of adversarial robustness in MLLMs along multiple complementary dimensions. We first introduce a diagnostic CLIP-alignment protocol that predicts, prior to full MLLM training, which robust vision encoders will transfer effectively to the multimodal setting—revealing that large-scale multimodal adversarial pretraining, rather than unimodal scale alone, is the critical factor for strong robustness transfer. Building on this, we integrate such encoders into MLLMs via end-to-end multimodal training, yielding average gains of 28 CIDEr points on image captioning tasks and 11.7% in VQA accuracy under strong adversarial attacks compared to constrained plug-and-play baselines. We further show that adversarial training applied directly to a standard non-robust MLLM degrades both clean and adversarial performance—establishing robust visual representations as a strict prerequisite—while end-to-end adversarial training from a robust backbone delivers additional gains of 1.9 CIDEr points and 4.3% in VQA accuracy, most pronounced under strong ensemble-based attacks. Beyond training-time defenses, we show that lightweight test-time visual stochastic transformations serve as an effective practical black-box defense for non-robust MLLMs, elevating adversarial performance from near-zero to levels comparable with robust models, making image-space stochastic transformations surprisingly effective at disrupting adversarial feature structure before it reaches the language model. Finally, we evaluate white-box visual jailbreak robustness and demonstrate that our robust models substantially reduce toxic generation across all attack strengths. Together, these findings provide practical and scalable guidelines for improving adversarial robustness in MLLMs. Code and pretrained weights will be released publicly to support future research. Adversarial Robustness of MLLMs † † email: [email protected] † † email: [email protected] † † email: [email protected] † † affiliation: Mohamed Bin Zayed University of AI, UAE † † affiliation: Khalifa University, UAE † † affiliation: Australian National University, Australia 1 Introduction Figure 1: Robust performance of our approach on multimodal tasks under stronger adversarial attacks at perturbation budget \epsilon=8/255 : The original CLIP \xspace exhibits severe vulnerability, while robust CLIP baselines such as Sim-CLIP 4 \xspace and FARE 4 \xspace achieve only limited robustness. Ours (Align) integrates a large-scale adversarially pretrained vision encoder into the MLLM via standard end-to-end training, and Ours (Align+AT) further applies end-to-end adversarial training (AT) within the MLLM framework. Both variants substantially improve robustness under strong APGD and ensemble-based attacks. Large Language Models (LLMs) have transformed natural language processing, and their integration with visual perception has given rise to Multi-modal Large Language Models (MLLMs) [ 64 , 27 ] . By coupling a vision encoder—typically CLIP [ 46 ] —with a language model, MLLMs support open-ended vision–language reasoning, from answering detailed questions about images to generating contextual descriptions and multi-step analyses [ 69 , 33 ] . Instruction tuning [ 33 , 67 ] further turns these systems into interactive assistants that can interpret and discuss visual content in natural language, and recent work improves fine-grained perception by integrating multiple vision encoders [ 57 , 53 ] . However, visual inputs also introduce a critical security challenge: MLLMs must operate over continuous, high-dimensional image spaces that are far more susceptible to adversarial manipulation than the discrete token space of text [ 8 , 45 ] . This enlarged attack surface enables objectives beyond label flipping—attackers can induce hallucinations, trigger unsafe or policy-violating behavior, or steer responses while the generated text remains fluent and plausible [ 48 ] . Defending MLLMs against visual adversaries is therefore substantially harder and remains a major barrier to deployment in safety-critical settings. Most existing defenses adopt plug-and-play strategies: the vision encoder is fine-tuned in isolation—through adversarial training [ 38 , 49 , 22 ] or preference optimization [ 31 ] —and then swapped into an MLLM. To preserve compatibility with pretrained MLLMs, these methods constrain robust representations to remain close to CLIP’s original embedding space, which limits how far robustness can be pushed and yields only modest gains. In parallel, recent work has explored scaling adversarial training through a two-stage paradigm; adversarial CLIP-style pretraining on large-scale image–text data to enable web-scale adversarial learning, followed by adversarial fine-tuning on ImageNet [ 61 ] . While Wang et al. 2024b primarily explores robust vision encoders in the context of image classification, the resulting representations are both robust and semantically rich, making them promising candidates for integration into MLLMs. Motivated by these challenges, we present a systematic investigation of adversarial robustness in MLLMs across multiple complementary dimensions, primarily under a white-box threat model. First, we study the role of the visual backbone by exploring a diverse set of robust vision encoders and assessing their suitability for multimodal integration through a diagnostic CLIP-alignment protocol that predicts, prior to full MLLM training, which encoders will transfer effectively to the multimodal setting. Second, building on these insights, we investigate adversarial training directly within the MLLM framework, enabling joint adaptation of the visual and language components under adversarially perturbed inputs—uncovering a critical negative result about when MLLM-level adversarial training helps and when it does not. Third, we evaluate model robustness beyond standard captioning and VQA tasks by examining white-box visual jailbreak attacks, in which adversarially perturbed images are paired with harmful textual prompts to elicit toxic outputs. Finally, because large-scale adversarial training is computationally demanding, we evaluate lightweight test-time defenses for standard (non-robust) MLLMs under a black-box setting where the attacker has access to the deployed model’s architecture, weights, and gradients, but does not know the inference-time operations performed by the end user. Our analysis yields four key insights for improving adversarial robustness in MLLMs: • Scale and modality of adversarial pretraining both matter. Large-scale adversarial pretraining helps alleviate the robustness–generalization trade-off Schmidt et al. 2018 ; Stutz et al. 2019 ; Hendrycks et al. 2019 , but scale alone is not sufficient. We find that adversarial pretraining that is multimodal in nature —e.g., CLIP-style image–text adversarial learning—produces visual representations that are not only robust, but also semantically aligned with language, leading to substantially stronger robustness gains when integrated into MLLMs than unimodal adversarial pretraining. We introduce a diagnostic CLIP-alignment study that predicts this transferability prior to MLLM integration, providing a principled criterion for vision encoder selection. • Robust visual representations are a prerequisite for effective MLLM adversarial training. We find that applying adversarial training directly to a standard CLIP-based MLLM degrades both clean and adversarial performance—a counter-intuitive result showing that adversarial optimization alone is insufficient when the underlying visual features are fragile. Starting from a robust vision encoder, end-to-end adversarial training in the MLLM framework consistently improves robustness, delivering the most pronounced gains under strong ensemble-based attacks (see Fig. 1 ). • Visual robustness generalizes to jailbreak threats. Without any jailbreak-specific training, our robust models substantially reduce toxic content generation under white-box visual jailbreak attacks—in which adversarially perturbed images are paired with harmful textual prompts to maximize the likelihood of eliciting toxic outputs. • Simple test-time defenses