DarkLLM: Learning Language-Driven Adversarial Attacks with Large Language Models

Ye Sun, Xin Wang, Jiaming Zhang, Yifeng Gao, Yixu Wang, Yifan Ding, Qixian Zhang Henghui Ding Xingjun Ma Yu-Gang Jiang · 2026

Read DarkLLM: Learning Language-Driven Adversarial Attacks with Large Language Models by Ye Sun, Xin Wang, Jiaming Zhang, Yifeng Gao, Yixu Wang, Yifan Ding, Qixian Zhang Henghui Ding Xingjun Ma Yu-Gang Jiang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18868.txt.

Introduction

Ye Sun and nine colleagues turn plain-language attack orders into visual tricks that fool vision and multimodal models. Their DarkLLM system trains a language model to convert an instruction into a hidden attack vector, which is then decoded as a perturbation added to an image. One small model, with about a billion parameters, thus covers targeted, untargeted, segmentation and multi-model attacks alike. Tests across many tasks, datasets and models show it working against CLIP, SAM and frontier language models, exposing a broad weakness in current foundation models.

Catalog record

CREATORYe Sun, Xin Wang, Jiaming Zhang, Yifeng Gao, Yixu Wang, Yifan Ding, Qixian Zhang Henghui Ding Xingjun Ma Yu-Gang Jiang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63799
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.18868

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DarkLLM: Learning Language-Driven Adversarial Attacks with Large Language Models Ye Sun, Xin Wang, Jiaming Zhang, Yifeng Gao, Yixu Wang, Yifan Ding, Qixian Zhang Henghui Ding Xingjun Ma Yu-Gang Jiang Abstract While vision and multimodal foundation models underpin critical tasks from perception to complex reasoning, they remain highly vulnerable to adversarial attacks. However, traditional adversarial attacks are typically limited to single, predefined objectives, tightly coupling each attack to a specific model or task, which restricts their scalability and flexibility in real-world scenarios. In this work, we present DarkLLM , a novel attack framework that trains an LLM to translate natural-language attack instructions into latent attack vectors, which are then decoded into visual adversarial perturbations. By leveraging natural-language instruction tuning, DarkLLM not only unifies targeted, untargeted, segmentation, and multi-model attacks within a single framework, but also achieves flexible and controllable adversarial generation, enabling each instruction to produce a perturbation that induces desired behaviors across heterogeneous models. Through extensive experiments across 4 tasks, 13 datasets, and 15 models, we demonstrate that DarkLLM with only 1B parameters can follow attacker instructions and generate highly effective attacks against CLIP, SAM, and frontier LLMs, revealing a systemic vulnerability in modern foundation models. 1 Introduction Large vision and multimodal foundation models 29 ; 33 have emerged as the central interface for perceiving and reasoning over visual content, underpinning a wide spectrum of applications such as image retrieval 46 , segmentation 26 , and multimodal assistants 1 ; 54 . Yet their impressive capabilities are shadowed by a long-standing weakness: small, often imperceptible adversarial perturbations can reliably induce incorrect or even adversary-specified model behaviors 53 ; 18 . Consequently, understanding and stress-testing the adversarial robustness of large-scale vision and multimodal models has become a pressing and practically relevant safety problem 40 ; 39 . Figure 1 : (a) An illustration of traditional adversarial attacks. (b) DarkLLM translates natural-language instructions into customized visual perturbations, enabling unified and controllable adversarial attacks. (c) DarkLLM achieves strong attack success rates across diverse models. Over the past decade, adversarial research has been dominated by gradient-based optimization 18 ; 40 and generative adversarial modeling 17 ; 62 , revealing pervasive vulnerabilities across both classical architectures and modern foundation models 42 ; 72 . Yet two fundamental challenges persist: transferability and scalability . First, an effective attack should be transferable across heterogeneous models and tasks—often even jointly, as contemporary systems routinely integrate recognition, segmentation, and multimodal reasoning modules 70 ; 77 ; 72 . Equally importantly, a practical attack framework should be scalable, allowing different adversarial intents to be flexibly specified without requiring objective-specific pipeline redesign or repeated optimization 69 ; 14 . However, existing approaches struggle to satisfy these requirements simultaneously. Most untargeted attacks that aim to induce arbitrary errors are tailored to specific architectures or foundation models (e.g., ViT 58 ; 41 ; 48 , CLIP 70 ; 75 ; 15 , or SAM 77 ; 68 ; 37 ), while representative targeted attacks, such as SU 59 , SASD 61 , TTAA 57 , TTP 44 , SSA-CWA 12 , and M-Attack 31 , steer model outputs toward adversary-specified targets but typically follow a rigid “one-target, one-optimization” paradigm, requiring re-optimization for each new adversarial objective. These limitations raise a critical question: Is it possible to design a new adversarial attack framework that is transferable and scalable across models and objectives? In parallel, the emergence of Large Language Models (LLMs) 4 has fundamentally reshaped how complex tasks are specified and solved. LLMs excel at interpreting semantically rich, compositional instructions and translating them into structured, domain-specific outputs, enabling breakthroughs across visual understanding 33 ; 64 ; 52 , generation 55 ; 5 , and medical analysis 50 ; 45 . Within the safety domain, LLMs have already been exploited to generate jailbreak prompts 35 ; 34 , automate safety evaluations 65 , and design textual backdoors 30 ; 16 . Yet, existing work largely remains confined to the textual domain, where LLMs are used to generate adversarial or malicious text, rather than pixel-level adversarial perturbations, motivating an intriguing problem: Can the semantic reasoning capabilities of LLMs be leveraged for visual adversarial generation? In this work, we propose a novel language-driven attack framework, DarkLLM , which leverages natural-language instructions as the interface for adversarial generation. Figure 1 highlights the distinction between DarkLLM and existing attack strategies. Unlike conventional attacks, DarkLLM reformulates adversarial generation as a language-conditioned synthesis problem, enabling each instruction to be translated into a visual perturbation that induces desired adversarial behaviors across heterogeneous foundation models. Specifically, our approach trains an LLM to act as an adversarial controller, mapping semantically rich attack instructions into latent attack representations, which are then decoded by a lightweight perturbation generator into pixel-level adversarial noise. Through large-scale instruction tuning, DarkLLM learns to associate different attack intents with corresponding generators, allowing targeted, untargeted, segmentation, and multi-model attacks to be flexibly generated within a unified framework. We systematically analyze several key factors for improving attack effectiveness, including prompt ensemble strategies, loss function design, surrogate model selection, and training strategies. Finally, we conduct extensive experiments across 4 tasks and 15 models (CLIP 46 , SAM 26 ; 47 , Gemini 54 , ChatGPT 1 ), demonstrating that DarkLLM achieves strong adversarial effectiveness across all evaluated foundation models. Our main contributions are summarized as follows: • We propose DarkLLM , a novel language-driven adversarial attack framework that reformulates visual adversarial generation as an instruction-conditioned synthesis problem, enabling diverse attack objectives to be flexibly specified through natural language. • We develop a unified adversarial generation paradigm that leverages LLM-based adversarial controllers and modular perturbation generators to jointly support targeted, untargeted, segmentation, and multi-model attacks within a single framework. • We conduct extensive experiments across 4 tasks and 15 models, covering CLIP, SAM, and frontier multimodal LLMs. Both qualitative and quantitative results demonstrate that DarkLLM achieves strong adversarial effectiveness and robustness against common image corruption defenses. 2 Related Work Conventional Adversarial Attacks. The vulnerability of deep neural networks to adversarial examples remains a foundational challenge in machine learning 53 . Traditional research divides into white-box attacks, which leverage full model access (e.g., gradients) to craft perturbations via optimization methods like FGSM 18 and PGD 40 , and black-box attacks, which operate under restricted access. Black-box strategies typically rely on query-based estimation 21 ; 2 or transfer-based generation using surrogate models 13 ; 63 ; 20 ; 31 . While effective on specific tasks, conventional methods typically operate under a fixed optimization framework tailored to a single target. This constraint severely limits their scalability across the diverse and dynamic task spectrum of modern foundation models. Adversarial Attacks on Foundation Models. The rise of large vision foundation models has introduced architectures with unprecedented generalization 46 ; 26 ; 33 . Yet, recent studies confirm that this capability does not equate to robustness; these models remain highly susceptible to adversarial perturbations 70 ; 72 . Existing research has systematically exposed these vulnerabilities across different domains: adversarial attacks targeting CLIP’s modality alignment include specific methods like Co-Attack 70 , AdvCLIP 75 , C-PGC 15 and X-Transfer 20 ; vulnerabilities in SAM’s segmentation boundaries have been exploited by attacks such as Attack-SAM 68 , DarkSAM 77 , UAD 37 ,

Read the complete text