AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models
Aashray Reddy, Andrew Zagula, Nicholas Saban · 2025
Read AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models by Aashray Reddy, Andrew Zagula, Nicholas Saban (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-01020.txt.
Introduction
Aashray Reddy, Andrew Zagula, and Nicholas Saban present AutoAdv, a framework that automates jailbreak attacks on large language models to test their safety rails. Jailbreaks are crafted inputs that coax a model past its guards into harmful output. Automating them lets safety teams probe at scale instead of by hand.
Their 2025 paper drives the attack with a parametric attacker model that rewrites harmful prompts into disguised forms. Failed tries are read and refined across many turns, using roleplay, misdirection, and context tricks. Success is scored with the StrongREJECT yardstick on leading models from several labs. Rates ran high, a sign the authors press: today's guards bend under patient, machine-led pressure.
Catalog record
| CREATOR | Aashray Reddy, Andrew Zagula, Nicholas Saban |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-01020
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models Aashray Reddy, Andrew Zagula, Nicholas Saban Abstract Large Language Models (LLMs) continue to exhibit vulnerabilities to jailbreaking attacks: carefully crafted malicious inputs intended to circumvent safety guardrails and elicit harmful responses. As such, we present AutoAdv, a novel framework that automates adversarial prompt generation to systematically evaluate and expose vulnerabilities in LLM safety mechanisms. Our approach leverages a parametric attacker LLM to produce semantically disguised malicious prompts through strategic rewriting techniques, specialized system prompts, and optimized hyperparameter configurations. The primary contribution of our work is a dynamic, multi-turn attack methodology that analyzes failed jailbreak attempts and iteratively generates refined follow-up prompts, leveraging techniques such as roleplaying, misdirection, and contextual manipulation. We quantitatively evaluate attack success rate (ASR) using the StrongREJECT Souly et al. 2024 framework across sequential interaction turns. Through extensive empirical evaluation of state-of-the-art models—including ChatGPT, Llama, and DeepSeek—we reveal significant vulnerabilities, with our automated attacks achieving jailbreak success rates of up to 86% for harmful content generation. Our findings reveal that current safety mechanisms remain susceptible to sophisticated multi-turn attacks, emphasizing the urgent need for more robust defense strategies. Warning: This paper includes examples of harmful and sensitive language; reader discretion is advised. 1 Introduction Large Language Models have rapidly evolved into integral components in modern artificial intelligence systems, offering powerful capabilities across tasks ranging from creative content generation to complex problem-solving. However, their widespread deployment raises substantial safety concerns, particularly regarding their potential to generate harmful, unethical, or otherwise problematic content when adversarially prompted. Despite significant research on alignment techniques and content filtering mechanisms, LLMs remain vulnerable to jailbreaking attacks that bypass their safety guardrails. Current safety evaluation methodologies focus on single-turn interactions, where an adversarial prompt is provided to the model, and the response is analyzed for policy violations. However, this approach fails to capture the complexity of multi-turn interactions that characterize real-world deployment scenarios. Prior work has demonstrated that conversational context can significantly influence model behavior, with certain vulnerabilities only becoming apparent over multiple turns of dialogue Li et al. 2024 . Moreover, the manual creation of effective jailbreaking prompts requires substantial expertise and resources, limiting the comprehensiveness of safety evaluations Paulus et al. 2024 . To address these limitations, we introduce AutoAdv, an automated framework for generating and evaluating multi-turn jailbreaking attacks. Our approach harnesses a secondary LLM, Grok-3-mini, as an attack agent to systematically probe a target model for vulnerabilities using strategically designed adversarial prompts. The key innovation in our methodology is the implementation of an adaptive learning mechanism that enables AutoAdv to refine its attack strategy based on previous interaction outcomes, generating increasingly effective follow-up prompts that exploit identified weaknesses in the target model’s safety mechanisms Chao et al. 2024 . Through extensive evaluation of models including ChatGPT, Llama, and DeepSeek, we demonstrate that current LLMs remain highly vulnerable to automated attacks. Our results show a maximum jailbreak success rate of 86% on Llama3.1-8B, with multi-turn attacks increasing success rates by up to 51% compared to single-turn approaches. AutoAdv exposes key gaps in current LLM alignment, showing that multi-turn jailbreaks can still bypass defenses with ease. These findings accentuate the need for more robust safety mechanisms particularly designed to maintain coherence over conversational contexts. 2 Related Works 2.1 Adversarial Prompt Research Adversarial prompting research has rapidly evolved with modern LLMs. Wallace et al. 2021 first showed that neural networks could generate toxic outputs from crafted inputs. Zou et al. 2023 demonstrated that model-agnostic suffixes could bypass safety filters on models like ChatGPT and Llama, highlighting transferability. Liu et al. 2024 categorized jailbreak strategies into direct, multi-turn, and obfuscated prompts. Das et al. 2024 further developed context-rich, narrative-driven prompts that effectively evade detection while remaining interpretable. 2.2 Red teaming for LLMs Redteaming is essential for evaluating LLM safety. Perez et al. 2022 demonstrated that automated systems could uncover vulnerabilities overlooked by human testers. Mazeika et al. 2024 proposed standardized metrics for evaluating and benchmarking safety performance across different models, while Hartvigsen et al. 2022 assessed harmful content generation related to demographics. The IMG Trojan method Tao et al. 2025 expanded redteaming to multimodal domains, demonstrating that poisoning a single image in 10,000 training samples significantly increased vulnerability in vision-language models. 2.3 Automated Adversarial Attacks Paulus et al. 2024 introduced AdvPrompter, an LLM trained via reinforcement learning, for generating adversarial prompts. Hughes et al. 2024 developed the BoN algorithm, demonstrating that simple augmentation methods could achieve attack success rates up to 89%, highlighting scaling behaviors. The Crescendo framework Russinovich et al. 2025 further improved automated jailbreak performance, surpassing previous methods by up to 61% on GPT-4 and 71% on Gemini-Pro, significantly simplifying effective attacks. 2.4 Multi-Turn Attacks Ren et al. 2024 introduced ActorAttack, bypassing safety systems through subtle chained interactions. The RED QUEEN ATTACK Jiang et al. 2024 achieved success rates of up to 87.62% against GPT-4o and 75.4% against Llama3.3-70b using structured multi-turn scenarios to mask malicious intent. Prompt Automatic Iterative Refinement (PAIR) Chao et al. 2024 demonstrated that dynamically adapting prompts based on model responses significantly boosts attack effectiveness by exploiting conversational dynamics. 2.5 Our Contributions Our AutoAdv framework addresses the limitations of existing methods by integrating automated adversarial prompt generation with dynamic multi-turn attacks. Unlike approaches relying on static templates or isolated interactions, AutoAdv uses adaptive learning to analyze failed attempts and iteratively improve strategies. By leveraging an attacker LLM with optimized prompts and targeted rewriting, AutoAdv systematically uncovers vulnerabilities overlooked in single-turn tests. Additionally, AutoAdv categorizes strategies and maintains a database of effective patterns, enabling more sophisticated and realistic evaluations of LLM safety risks. 3 AutoAdv 3.1 Threat Model AutoAdv functions as a black-box attack methodology, operating without access to the target’s internal architectural specifications, including hyperparameters, model weights, and related configurations. AutoAdv conducts all interactions exclusively through the target’s standard API interface. This approach creates an environment that accurately simulates the interaction patterns between typical users and LLMs, including how security boundaries might be circumvented via adversarial prompts, thus establishing AutoAdv as a realistic and valid attack assessment. Figure 1: The AutoAdv workflow, where a malicious prompt is successively reformulated in an attempt to bypass the target LLM’s safeguards, culminating in either a successful or unsuccessful jailbreak. Information about successful jailbreaks is stored in a databse where it is later retrieved by the attacker for future attacks, enabling it to learn from its past attempts. 3.2 Pipeline AutoAdv initiates the process by retrieving prompts from AdvBench, a curated dataset comprising adversarial prompts specifically designed to trigger harmful or policy-violating outputs from large language models Zou et al. 2023 . These prompts span multiple restricted categories, including hate speech, illegal activities, and harmful instructions, among others. Subsequently, AutoAdv leverages a generative model, such as Grok-3-mini, to systematically reformulate these malicious prompts, aiming to circumvent the target model’s safety mechanisms while maintaining malicious intent. The rewritten prompts are then sent to the target model, whose responses are evaluated using the StrongREJECT Souly et al. 2024 framework to determine the success of the jailbreak attempt. If the attack attempt fails, the attacker examines the target model’s response to identify refusal mechanisms, adapts its strategy accordingly, and generates a new attack prompt.