Attacks on multimodal models

Iablochnikov Viacheslav, Alexander Rogachev · 2024

Read Attacks on multimodal models by Iablochnikov Viacheslav, Alexander Rogachev (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01725.txt.

Introduction

Viacheslav Iablochnikov and Alexander Rogachev survey attacks on multimodal models in a 2024 paper. Such models chat across images, text, and other inputs at once. Many build on open-source parts, which may carry known flaws. The team asks whether those flaws pass into the full models.

Focus falls on vision-language models and their shared image encoder. The CLIP design and its vision part anchor most of the analysis. Patch attacks, where a crafted image patch fools the model, get close study. The authors weigh how well each attack type generalizes. Industry use raises the stakes if inherited flaws stay live. Open parts speed development but widen the attack surface. The work maps attack families for builders and testers. Defenses must check components, not just the finished model, the authors imply.

Catalog record

CREATORIablochnikov Viacheslav, Alexander Rogachev
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-01725

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Attacks on multimodal models Iablochnikov Viacheslav, Alexander Rogachev Abstract Today, models capable of working with various modalities simultaneously in a chat format are gaining increasing popularity. Despite this, there is an issue of potential attacks on these models, especially considering that many of them include open-source components. It is important to study whether the vulnerabilities of these components are inherited and how dangerous this can be when using such models in the industry. This work is dedicated to researching various types of attacks on such models and evaluating their generalization capabilities. Modern VLM models (LLaVA, BLIP, etc.) often use pre-trained parts from other models, so the main part of this research focuses on them, specifically on the CLIP architecture and its image encoder (CLIP-ViT) and various patch attack variations for it. Code is available here: https://github.com/slava-qw/image-retrieval-robustness Introduction The rising popularity of large language models (LLMs) ( [ 33 ] ) has fueled rapid growth in multimodal "image2text" models, capable of processing both images and text simultaneously ( [ 31 ] ). Today, these models are not only attracting the attention of researchers, but are also widely used across various industries. The integration of text and image processing within a single model offers significant opportunities for applying such approaches to a wide range of tasks. However, this also opens up new avenues for attacks on these models, as many of them consist of pretrained components or rely on open-source solutions, thereby inheriting associated vulnerabilities. Models used in practical applications must be robust against various input-based attacks. Specifically, compromised input data should not have the ability to manipulate the model’s behavior in undesirable ways—such as leaking personal information, altering the behavior of public services, or assisting users in committing illegal activities. One of the most popular forms of input attacks involves adding specific noise to the original image ( [ 7 ] , [ 5 ] , [ 6 ] ). While these pixel-level attacks are powerful ( [ 4 ] ), they lack generalizability, as a unique perturbation needs to be crafted for each individual image ( [ 5 ] ). As a result, methods capable of generalizing across multiple images have gained prominence. Among these methods, patch attacks and universal adversarial perturbations (UAPs) hold a special place ( [ 25 ] , [ 28 ] ). Universal Adversarial Perturbations (UAPs) are a type of attack where a single perturbation is created to mislead the model across a wide range of different images [ 26 ] . UAPs are effective due to their simplicity and versatility, as the same perturbation can be applied to multiple images, leading to incorrect predictions by the model. These attacks are particularly dangerous for multimodal models [ 32 ] , which leverage a combination of various data types, since they allow an attacker to encode specific harmful information via these perturbations. Patch attacks are a method derived from UAPs, in which a specific patch (a region of the image) is altered to mislead the model [ 30 ] . This patch can be applied to different images and will be equally effective regardless of the context in which it is used [ 29 ] . This makes patch attacks a powerful tool, as they can target multimodal models handling various types of data, such as images, text, or audio. Both techniques pose a significant threat to the reliability of multimodal models, as they allow adversaries to bypass security mechanisms and force models to make incorrect decisions. Understanding and developing methods to defend against such attacks is crucial, as it will enhance the robustness of deep neural networks and ensure their safe application in critical areas. Related works Multimodal models The rapid growth in popularity and various methods of language processing have made it possible to combine two modalities: text and images, to create multimodal models (VLMs). These models primarily build on the success of large language models [ 13 ] (for example, [ 12 ] ), expanding their capabilities to the visual domain. To reduce computational costs, modern models use pre-trained components. Their results are combined using a trainable neural network, which can be represented either by additional layers [ 11 ] , or a learnable mapping between the two modalities [ 10 ] . CLIP One type of Vision-Language Model (VLM) is the Contrastive Language-Image Pre-training (CLIP) model developed by OpenAI [ 2 ] . CLIP employs contrastive learning, which is based on the principle of bringing similar elements closer together and pushing dissimilar ones apart in the representation space. It uses InfoNCE as its loss function, a variation of Noise Contrastive Estimation (NCE), which applies a softmax function and a temperature parameter to compute similarities between positive pairs (an image and its caption) and negative pairs (the same image with captions from other images in the batch). \mathcal{L_{\text{clip}}}=-\frac{1}{2|\mathcal{B}|}\sum_{i=1}^{|\mathcal{B}|}\Bigg(\overbrace{\log\frac{e^{t\mathbf{x}_{i}\cdot\mathbf{y}_{i}}}{\sum_{j=1}^{|\mathcal{B}|}e^{t\mathbf{x}_{i}\cdot\mathbf{y}_{j}}}}^{\text{image }\rightarrow\text{ text softmax }}+\overbrace{\log\frac{e^{t\mathbf{x}_{i}\cdot\mathbf{y}_{i}}}{\sum_{j=1}^{|\mathcal{B}|}e^{t\mathbf{x}_{j}\cdot\mathbf{y}_{i}}}}^{\text{text }\rightarrow\text{ image softmax }}\Bigg) where \mathbf{x}_{i}=\frac{E_{\text{img}}\left(I_{i}\right)}{\left\|E_{\text{img}}\left(I_{i}\right)\right\|_{2}},\ \mathbf{y}_{i}=\frac{E_{\text{txt}}\left(T_{i}\right)}{\left\|E_{\text{txt}}\left(T_{i}\right)\right\|_{2}} , |\mathcal{B}| - batch size. The architecture of CLIP includes two encoders—one for images and one for text—that are trained to align their inputs in a shared embedding space. Trained on a dataset of 400 million image-caption pairs, CLIP demonstrates zero-shot capabilities for classifying images and generating corresponding text. For example, CLIP based on ResNet-101 [ 1 ] achieves an accuracy of 76.2\% in image classification tasks. It’s worth noting that researchers continue to improve upon the CLIP approach. For instance, SigLIP [ 8 ] adapts CLIP’s multi-class problem to a binary cross-entropy loss function, enhancing zero-shot performance even with smaller batch sizes. \mathcal{L_{\text{SigLIP}}}=-\frac{1}{|\mathcal{B}|}\sum_{i=1}^{|\mathcal{B}|}\sum_{j=1}^{|\mathcal{B}|}\underbrace{\log\frac{1}{1+e^{z_{ij}\left(-t\mathbf{x}_{i}\cdot\mathbf{y}_{j}+b\right)}}}_{\mathcal{L}_{ij}} here, z_{ij} is the label for a given image and input text pair, equal to 1 if the pair is positive, and -1 otherwise. The parameters t and b are trainable. Another variation, [ 9 ] , introduces a cross-attention module to account for image encoding based on target captions, further improving benchmark results. Adversarial attacks Let x_{0}\in\mathbb{R}^{d} be the original data. An attack is considered successful if, after applying a predefined transformation \mathcal{A} , the model f_{\theta} produces the desired result on the attacked images x=\mathcal{A}(x_{0}) . For classification tasks, if C_{i} and C_{t} are the original and target labels, then f_{\theta}(x)=C_{t}\neq C_{i} . The transformation \mathcal{A} is typically chosen to be an additive function in the pixel space for simplicity (although other methods exist [ 20 ] ). To make the attack less noticeable, a constraint is imposed on the transformation, ensuring that \|x-x_{0}\|_{p}\leq\varepsilon . This optimization problem is often analytically intractable, so gradient-based methods are employed to solve it. Similar to gradient descent, the gradient of the loss function with respect to the added noise is computed and its values are updated iteratively. This method is known as the Fast Gradient Sign Method (FGSM) [ 7 ] . However, one step of FGSM is often insufficient to achieve the desired result, so this process can be repeated several times: x_{k+1}=\mathcal{P}_{S}(x_{k}+\alpha\cdot\text{sign}(\nabla_{x}L(\theta,x_{k},y))) where \mathcal{P}_{S} is a projection operator and L is the loss function. This iterative method is known as Projected Gradient Descent (PGD) [ 5 ] . These methods and their subsequent variations [ 6 ] are the most popular solutions for optimization tasks in the context of adversarial attacks, as they are computationally efficient. For UAPs (Universal Adversarial

Read the complete text