Feature-Space Adversarial Robustness Certification for Multimodal Large Language Models
Song Xia, Meiwen Ding, Chenqi Kong, Wenhan Yang, Xudong Jiang · 2026
Read Feature-Space Adversarial Robustness Certification for Multimodal Large Language Models by Song Xia, Meiwen Ding, Chenqi Kong, Wenhan Yang, Xudong Jiang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16200.txt.
Introduction
Song Xia and four coauthors give proof bounds for attacks on models that join sight and words. Their 2026 preprint deals with large models that take pics and text. Small tweaks to a pic can bend the deep code of the model. Bent code then leads to wrong picks by the model. Past guards oft lack firm proof that they hold.
The team smooths the deep code space to gain such proof. Smooth code gives a zone where no tweak can flip the pick. Proof zones tell users when a pick can be trusted. That trust counts for high stakes use of such models. The work adds a guard with maths to back it.
Catalog record
| CREATOR | Song Xia, Meiwen Ding, Chenqi Kong, Wenhan Yang, Xudong Jiang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-16200
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Feature-Space Adversarial Robustness Certification for Multimodal Large Language Models Song Xia, Meiwen Ding, Chenqi Kong, Wenhan Yang, Xudong Jiang Abstract Multimodal large language models (MLLMs) exhibit strong capabilities across diverse applications, yet remain vulnerable to adversarial perturbations that distort their feature representations and induce erroneous predictions. To address this vulnerability, we propose Feature-space Smoothing (FS), a general framework that provides certified robustness guarantees at the feature representation level of MLLMs. We theoretically prove that FS converts a given feature extractor into a smoothed variant that is guaranteed a certified lower bound on the cosine similarity between clean and adversarial features under ℓ 2 \ell_{2} -bounded perturbations. Moreover, we establish that the value of this Feature Cosine Similarity Bound (FCSB) is determined by the intrinsic Gaussian robustness score of the given encoder. Building on this insight, we introduce the Gaussian Smoothness Booster (GSB), a plug-and-play module that enhances the Gaussian robustness score of pretrained MLLMs, thereby strengthening the robustness guaranteed by FS, without requiring additional MLLM retraining. Extensive experiments demonstrate that applying the FS to various MLLMs yields strong certified feature-space robustness and consistently leads to robust task-oriented performance across diverse applications. 1 Introduction The emergence of the Multimodal Large Language Models (MLLMs), such as GPT-5 ( 34 ) , Gemini 3 Pro ( 7 ) , and Claude Sonnet 4.5 ( 1 ) , has fundamentally reshaped existing working paradigms and significantly advanced societal productivity. Despite their remarkable capabilities across a broad spectrum of real-world tasks, these models still encounter critical safety challenges, such as adversarial vulnerabilities ( 54 ; 6 ; 25 ; 15 ) . Figure 1 : Illustration of the FS-GSB, which guarantees that the cosine similarity of the adversarial and clean features extracted by MLLM’s encoder is larger than FCSB for robust predictions. Adversaries can manipulate predictions of the MLLMs to a malicious state by injecting subtle and imperceptible perturbations to the clean inputs, exploiting the models’ insufficient local smoothness and uncontrolled Lipschitz continuity ( 11 ; 5 ; 13 ; 49 ) . Countermeasures towards those threats can be roughly classified into empirical defense and certified defense. Typical empirical approaches include adversarial training ( 29 ; 38 ; 44 ; 47 ; 3 ; 40 ; 30 ; 31 ) and input purification ( 32 ; 51 ; 18 ; 19 ; 56 ) . Despite their demonstrated empirical effectiveness, these approaches lack formal robustness guarantees and remain susceptible to stronger adversaries ( 41 ; 4 ; 46 ; 33 ) . Moreover, the multimodal nature of MLLMs also poses a great challenge for existing adversarial training methods. Since they accept heterogeneous inputs across diverse domains, ensure a robust encoder via adversarial training that can generalize to various scenarios is challenging and computationally costly. In contrast, certified approaches aim to guarantee that the model returns a constant prediction result within a certain range, usually a \ell_{2} or \ell_{\infty} -norm constrained area ( 37 ; 45 ; 12 ; 16 ; 5 ; 49 ; 39 ) . However, most previous certified defense approaches ( 5 ; 49 ; 43 ; 12 ; 39 ) predominantly assume the prediction of one-dimensional outputs (e.g., class label), thereby limiting their applicability to more general tasks on MLLMs. To address the aforementioned challenges, we propose the Feature-space Smoothing (FS), a provable defense method that offers certified robustness on the feature representations of MLLMs. Specifically, FS smooths the vanilla feature encoder of the MLLM, and the resulted smoothed encoder is guaranteed to maintain a provable lower bound on the cosine similarity between clean and adversarial representations under \ell_{2} -norm bounded attacks. Moreover, we demonstrate that this Feature Cosine Similarity Bound (FCSB) of the smoothed feature encoder is intrinsically determined by a defined Gaussian robustness score, which measures the prediction consistency of the vanilla feature extractor under Gaussian noise. Nonetheless, the Gaussian robustness of these MLLMs remains limited without Gaussian noise augmented training, leading to a correspondingly suboptimal FCSB. To address this whiling avoiding the high computational cost of fine-tuning MLLMs, we propose a plug-and-play Gaussian Smoothness Booster (GSB), consisting of a lightweight Gaussian denoiser and a residual smoothness mapper. Specifically, the denoiser is applied prior to the feature encoder and is trained to mitigates Gaussian noises and improves the resulting robustness score. In parallel, the smoothness mapper operates after feature extraction for feature-level refinement. It is trained to preserve the original feature distribution while further enhancing the Gaussian robustness score. These two components of GSB work synergistically to enhance the certified robustness of MLLMs provided by FS. The GSB is optimized using the proposed utility–robustness loss and trained on data from diverse visual domains to enhance the Gaussian robustness while preserving the feature utility for the encoder. To comprehensively assess the performance, we evaluate our method against state-of-the-art (SOTA) adversarial attacks tailored for MLLMs under the white-box setting and compare it with advanced adversarial training methods. Extensive experimental results demonstrate that our FS not only provides a strong certified robustness guarantee on the feature representation but also exhibits superior adversarial robustness for various MLLMs under diverse downstream tasks. Overall, the contributions of this work are: • We propose the Feature-space Smoothing (FS) to turn any feature encoder into a smoothed version, and theoretically prove that the smoothed encoder maintains a certified lower bound on the feature cosine similarity between clean and adversarial representations. • We propose the Gaussian Smoothness Booster (PSB) that effectively enhances the FCSB of the smoothed encoder in MLLMs via a plug-and-play manner. • We conduct extensive experiments demonstrating that integrating our proposed FS-GSB greatly enhances the adversarial robustness of various MLLMs and dramatically reduces the ASR under various white-box attacks. 2 Related Work Adversarial attacks on MLLMs. While MLLMs continue to achieve remarkable performance across diverse applications, extensive works ( 35 ; 6 ; 54 ; 15 ; 25 ; 42 ; 52 ; 53 ; 50 ) have exposed their adversarial vulnerabilities, raising serious safety concerns. Early work, such as AttackVLM ( 54 ) , explores transferable attacks by disrupting the feature representations of CLIP ( 36 ) and BLIP ( 21 ) , showing strong adversarial transferability among open-source models but limited effectiveness against closed-source commercial systems. More recent approaches, such as M-Attack ( 25 ) and FOA-Attack ( 15 ) , further advance this direction by leveraging multi-extractor ensembles and feature-space alignment, achieving over 90% targeted attack success rates on image-captioning tasks against powerful closed-sourced commercial MLLMs ( e.g. , ChatGPT-4o). This highlights an urgent need for trustworthy defenses that provide effective and provable protection for MLLMs. Adversarial defense on MLLMs. Adversarial defense methods can be broadly classified into empirical and provable approaches. Empirical defenses for MLLMs mainly include adversarial training ( 29 ; 38 ; 44 ; 47 ; 3 ; 40 ; 30 ; 31 ) , which enhances robustness by augmenting training data with adversarial examples, and input purification ( 32 ; 19 ; 24 ) , which employs generative mechanisms such as diffusion models or autoencoders to recover clean inputs prior to inference. Recent studies ( 30 ; 31 ; 40 ) have revealed that utilization of adversarially trained CLIP-feature encoders can enhance adversarial robustness for MLLMs. However, these