Universal and Transferable Adversarial Attack on Large Language Models Using Exponentiated Gradient Descent
Sajib Biswas, Mao Nishino, Samuel Jacob Chacko, Xiuwen Liu · 2025
Read Universal and Transferable Adversarial Attack on Large Language Models Using Exponentiated Gradient Descent by Sajib Biswas, Mao Nishino, Samuel Jacob Chacko, Xiuwen Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-14853.txt.
Introduction
Guardrails that keep language models from harmful replies can be slipped by a crafted string tacked onto a prompt. Sajib Biswas and three co-authors build a stronger such string in 2025. Their attack aims to work on any prompt and to carry across models, the two traits that make a jailbreak hard to shrug off.
Past attacks search a vast space of word pieces, which is slow, or tune smooth codes that closed models cannot take as input. Turning those codes back into real words tends to blunt the attack. The new method tunes a soft form of the word codes themselves, with a gradient rule built for that shape. The work is a stress test: it shows where current guards bend, which is the first step to guards that hold.
Catalog record
| CREATOR | Sajib Biswas, Mao Nishino, Samuel Jacob Chacko, Xiuwen Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-14853
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Universal and Transferable Adversarial Attack on Large Language Models Using Exponentiated Gradient Descent Sajib Biswas, Mao Nishino, Samuel Jacob Chacko, Xiuwen Liu Abstract As large language models (LLMs) are increasingly deployed in critical applications, ensuring their robustness and safety alignment remains a major challenge. Despite the overall success of alignment techniques such as reinforcement learning from human feedback (RLHF) on typical prompts, LLMs remain vulnerable to jailbreak attacks enabled by crafted adversarial triggers appended to user prompts. Most existing jailbreak methods either rely on inefficient searches over discrete token spaces or direct optimization of continuous embeddings. While continuous embeddings can be given directly to selected open-source models as input, doing so is not feasible for proprietary models. On the other hand, projecting these embeddings back into valid discrete tokens introduces additional complexity and often reduces attack effectiveness. We propose an intrinsic optimization method which directly optimizes relaxed one-hot encodings of the adversarial suffix tokens using exponentiated gradient descent coupled with Bregman projection, ensuring that the optimized one-hot encoding of each token always remains within the probability simplex. We provide theoretical proof of convergence for our proposed method and implement an efficient algorithm that effectively jailbreaks several widely used LLMs. Our method achieves higher success rates and faster convergence compared to three state-of-the-art baselines, evaluated on five open-source LLMs and four adversarial behavior datasets curated for evaluating jailbreak methods. In addition to individual prompt attacks, we also generate universal adversarial suffixes effective across multiple prompts and demonstrate transferability of optimized suffixes to different LLMs. The source code for our implementation is available at: https://github.com/sbamit/Exponentiated-Gradient-Descent-LLM-Attack . 1 Introduction Large language models (LLMs) exhibit exceptional abilities in solving numerous real-world problems, including code comprehension ( Ahmad et al. 2021 ) , natural language modeling ( Brown et al. 2020 ) , and human-like conversations ( Miotto et al. 2022 ) . Recent studies show that LLMs can even surpass human performance on certain benchmarks ( Luo et al. 2024 ) . These models have become immensely popular following the release of GPT-based models ( Meyer et al. 2023 ) . Such widespread use of LLMs raises concerns about their societal and ethical impacts on their users ( Wu et al. 2024 ) . LLMs are trained on large-scale internet text corpora, which often include objectionable content ( Carlini et al. 2021 ) . As a result, they can learn from such content and also overgeneralize in unexpected ways, producing outputs that reflect harmful intentions or unintended behaviors ( Burns et al. 2022 ) . As a mitigation, LLM developers employ alignment strategies, such as reinforcement learning from human feedback (RLHF), which reduces the likelihood of producing unsafe outputs in response to provocative user prompts ( Ouyang et al. 2022 , Korbak et al. 2023 ) . While these techniques appear effective at a surface level and LLMs can self-evaluate and detect harmful content generated by themselves ( Li et al. 2023 ) , vulnerabilities are still present. LLMs are susceptible to adversarial prompting techniques, commonly known as jailbreak attacks, in which carefully crafted adversarial sequences of tokens added to user prompts induce harmful or policy-violating responses ( Deng et al. 2023 , Chao et al. 2023 ) . While early adversarial attacks on neural networks focused on perturbing continuous inputs such as images ( Szegedy et al. 2013 , Goodfellow et al. 2014 ) , prompt-based attacks on LLMs involve discrete text manipulations to circumvent safety mechanisms ( Wei et al. 2024 ) . Manually crafting these adversarial suffixes requires substantial time and effort, making such approaches limited in their applications ( Marvin et al. 2023 ) . Recent automated prompt-tuning methods attempt to search for effective suffixes without manual engineering ( Shin et al. 2020 , Wen et al. 2024 ) , yet these approaches remain computationally expensive due to the discrete nature of token space ( Carlini et al. 2024 ) . Additionally, studies have demonstrated the feasibility of universal adversarial triggers that generalize across multiple prompts ( Wallace et al. 2019 ) , and transferable adversarial suffixes that are effective across different LLMs ( Zou et al. 2023 , Liao and Sun 2024 ) , underscoring the scalability and generalizability of these jailbreak methods. In open-source LLMs, the attackers have full access to model weights and token embeddings, enabling them to induce harmful behaviors by directly manipulating the tokens and their embeddings ( Huang et al. 2023 ) . These white-box attacks can target either the discrete-level tokens ( Zou et al. 2023 , Sadasivan et al. 2024 ) or the continuous embedding space ( Geisler et al. 2024 , Schwinn et al. 2024 ) . Such attacks can be further amplified when targeting multi-modal models that accept continuous inputs ( Shayegani et al. 2023 ) . Additionally, the overall understanding of LLMs can be improved by assessing their adversarial robustness and analyzing the geometric properties of their token embeddings ( Biswas et al. 2022 , Yang et al. 2024 ) . In this work, we present a novel adversarial attack method based on exponentiated gradient descent (EGD), where we optimize relaxed one-hot token encodings to efficiently jailbreak LLMs. 1 1 1 This work builds on the work presented at the International Joint Conference on Neural Networks ( Biswas et al. 2025 ) . New contributions include the generation of universal multi-prompt adversarial suffixes and demonstration of their transferability across different LLMs in addition to extensive revision. Unlike prior approaches that require external projection steps to maintain validity within the probability simplex ( Geisler et al. 2024 ) , our method enforces these constraints inherently during optimization. We achieve higher success rates and faster convergence compared to state-of-the-art baseline methods. In addition to single-prompt attacks, we introduce universal adversarial suffixes that generalize over multiple prompts and demonstrate strong transferability across multiple LLM architectures, including proprietary models such as GPT-3.5. Our main contributions are summarized as follows: 1. We propose a novel adversarial attack method for jailbreaking open-source LLMs using exponentiated gradient descent on relaxed one-hot encodings. 2. We provide convergence guarantee and show that our method achieves higher success rates and faster convergence than three other baselines. 3. We evaluate our method on five open-source LLMs using four adversarial behavior datasets curated for assessing jailbreak attacks. 4. We demonstrate that our method produces universal adversarial suffixes effective across prompts, and transferable to different model architectures. 2 Related Work A growing body of recent work has focused on adversarial trigger optimization for jailbreaking LLMs ( Zou et al. 2023 , Schwinn et al. 2024 , Geisler et al. 2024 ) . In this approach, a predefined or randomly initialized suffix is appended to a user prompt designed to elicit harmful content. This suffix is iteratively refined to maximize the likelihood of a specified target output, eventually leading the model to bypass its alignment and produce the intended harmful response. Zou et al. 2023 introduced a greedy coordinate gradient-based search technique (GCG), which replaces one token at a time in the adversarial suffix. The replacement token is chosen based on a first-order Taylor approximation, following the principle behind the HotFlip method proposed by Ebrahimi et al. 2017 . After approximating a combination