Adversarial Attack-Defense Co-Evolution for LLM Safety Alignment via Tree-Group Dual-Aware Search and Optimization

Xurui Li, Kaisong Song, Rui Zhu, Pin-Yu Chen, Haixu Tang · 2025

Read Adversarial Attack-Defense Co-Evolution for LLM Safety Alignment via Tree-Group Dual-Aware Search and Optimization by Xurui Li, Kaisong Song, Rui Zhu, Pin-Yu Chen, Haixu Tang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-19218.txt.

Introduction

Xurui Li and four co-authors let attack and defense models for language model safety train against each other. Jailbreak attacks evolve quickly, while defenses are often static, so the two sides rarely improve together. Their framework, ACE-Safety, optimizes both models in one loop.

A group-aware tree search explores jailbreak strategies and produces varied adversarial examples that expose weak points. A second method then trains the attack and defense models on those hard cases through a curriculum of rising difficulty. Across several benchmarks the paired approach beats existing attack and defense methods, and the authors present it as a path toward safety that keeps pace with new threats.

Catalog record

CREATORXurui Li, Kaisong Song, Rui Zhu, Pin-Yu Chen, Haixu Tang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-19218

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adversarial Attack-Defense Co-Evolution for LLM Safety Alignment via Tree-Group Dual-Aware Search and Optimization Xurui Li, Kaisong Song, Rui Zhu, Pin-Yu Chen, Haixu Tang Abstract Large Language Models (LLMs) have developed rapidly in web services, delivering unprecedented capabilities while amplifying societal risks. Existing works tend to focus on either isolated jailbreak attacks or static defenses, neglecting the dynamic interplay between evolving threats and safeguards in real-world web contexts. To mitigate these challenges, we propose ACE-Safety (Adversarial Co-Evolution for LLM Safety), a novel framework that jointly optimize attack and defense models by seamlessly integrating two key innovative procedures: (1) Group-aware Strategy-guided Monte Carlo Tree Search ( GS-MCTS ), which efficiently explores jailbreak strategies to uncover vulnerabilities and generate diverse adversarial samples; (2) Adversarial Curriculum Tree-aware Group Policy Optimization ( AC-TGPO ), which jointly trains attack and defense LLMs with challenging samples via curriculum reinforcement learning, enabling robust mutual improvement. Evaluations across multiple benchmarks demonstrate that our method outperforms existing attack and defense approaches, and provides a feasible pathway for developing LLMs that can sustainably support responsible AI ecosystems. 1 Introduction Large Language Models (LLMs), such as ChatGPT ( 1 ) , and Llama ( 13 ) , have become integral components of the modern web ecosystem, powering advances in diverse areas including mathematical reasoning ( 31 ) , clinical diagnosis ( 19 ) , and complex problem-solving ( 4 ) . Despite their capabilities, the rapid adoption of LLMs has heightened safety concerns, as they can exacerbate risks such as disseminating harmful content and amplifying societal biases, which threatens vulnerable populations and undermines the sustainability of digital ecosystems ( 22 ; 14 ) . These risks are amplified by specialized attacks such as jailbreaking, where adversarially crafted prompts can bypass safety filters and produce harmful outputs. Proactively identifying and mitigating these jailbreak vulnerabilities is therefore a critical prerequisite for the safe and responsible deployment of LLMs ( 14 ) . Existing jailbreak attack methods can be primarily categorized into two types ( 45 ) . Token-level attacks use discrete optimization to create adversarial suffixes and bypass safety guardrails through gradient-guided token replacements ( 54 ) . Prompt-level attacks employ automated semantic manipulations to generate human understandable jailbreak prompts ( 5 ) . A key challenge for existing attack methods is efficiently selecting the optimal attack strategy while minimizing the instability arising from the inherent randomness in LLM generation. On the other hand, researchers have proposed various defenses to deal with these attacks. Prompt-level defenses check input prompts for malicious content before model generation ( 30 ) , while model-level defenses fine-tune LLMs to implement safe response ( 8 ) . However, existing works primarily focus on attacks or defenses in isolation, exhibiting significant limitations: (1) Attack methods are tailored to static defense capabilities, leading to low attack success rates against evolving LLMs ( 38 ; 37 ) . (2) Defense methods often overfit to limited attack instances and fail to generalize to novel, unprecedented jailbreak techniques ( 32 ) . To achieve efficient attacks and robust defenses, it is imperative to explore how to adapt jailbreak attack strategies against evolving defenses within a unified framework. Some preliminary attempts have been made. ICAG ( 51 ) is an adversarial game framework to improve attacks and defenses without fine-tuning. MART ( 12 ) employs conventional red-teaming for adversarial training, but suffers from inadequate sample diversity and compromised attack quality due to a simplistic data collection strategy. These methods do not perform as expected because: (1) failure to fully explore potential attack strategies through simple random searches; (2) low robustness for attack strategy determination due to inherent randomness in text generation; (3) inefficient learning from limited adversarial features and undifferentiated sample difficulty. Moreover, existing approaches tend to focus on refusing harmful responses, while overlooking to deliver practical guidance from a more socially responsible perspective. In this paper, we propose ACE-Safety (Adversarial Co-Evolution for LLM Safety), a novel framework that unifies tree-aware and group-aware mechanisms for mutual enhancement. Our key contributions are as follows: • ACE-Safety integrates co-evolving attack and defense models into a closed-loop system, where the two components collaboratively update and progressively refine one another via strategic jailbreak exploration and adversarial alignment. • We propose a Group-aware Strategy-guided Monte Carlo Tree Search (GS-MCTS) attack approach, which extends conventional tree-based search by incorporating strategy-guidance, adversarial priors and group-wise evaluation, enabling efficient multi-round jailbreak exploration while mitigating text generation randomness. • We develop an Adversarial Curriculum Tree-aware Group Policy Optimization (AC-TGPO) training diagram, which addresses supervision scarcity and challenging sample acquisition by enhancing group-based reinforcement learning with tree-aware adversarial curriculum learning. • Extensive experimental results demonstrate the superiority of ACE-Safety: its attack achieves the highest jailbreak success rate with fewer average attempts, while its defense outperforms existing methods against major attacks, maintaining better helpfulness and responsibility. 1 1 1 https://anonymous.4open.science/r/ACE-Safety-45F2 2 Related Work Attack methods on LLMs involve white-box and black-box attacks. White-box attack methods refer to attack strategies that require a deep understanding of the internal working mechanism of the model. 54 proposed a gradient-based GCG method which contains adversarial suffixes. 52 proposed a hierarchical genetic method AutoDAN to generate readable suffixes. Using malicious data to fine-tune LLMs can also boost attack success rates ( 29 ) . Black-box attacks, conversely, craft adversarial prompts to exploit safety gaps without accessing model internals, such as scenario nesting ( 10 ) , context-based attacks ( 49 ) , code injection ( 18 ) . Recent studies focus on improving jailbreak prompt rewriting efficiency: 46 uses genetic algorithms with seed mutation, Iterative refinement ( 5 ) uses attacker LLMs to dynamically optimize prompts, and tree-based approaches ( 26 ; 40 ) enhance search efficiency through structured generation. However, most attack methods suffer from LLM’s generation randomness and cannot adapt to evolving defense strategies well. Defense methods for LLMs focus on prompt-level interventions and model-level improvements. Prompt-level strategies include detection models ( 2 ) , classifiers ( 16 ) , adversarial pattern disruption ( 3 ) , and character noise injection ( 30 ) . However, prompt optimization techniques remain vulnerable to adaptive attacks. Model-level methods improve safety via instruction tuning, logit/gradient analysis ( 8 ) , and representation engineering ( 25 ; 35 ) , but face challenges such as over-filtering and data quality problems. Adversarial training frameworks ( 51 ; 12 ; 39 ) show potentials, while still suffer from inefficient adversarial strategy exploration, text generation randomness, and suboptimal optimization method. These limitations collectively hinder defense robustness against evolving jailbreak attacks. Figure 1: Illustration for jailbreak attacks via multi-round rewriting strategy optimization and the impact from text generation randomness on strategy effectiveness. 3 Methodology 3.1 Overview The ACE-Safety develops a robust co-evolution mechanism by integrating two key procedures seamlessly: (1) jailbreak attacks using GS-MCTS; (2) joint training with AC-TGPO. It contains three LLM components: attack model \mathcal{M}_{A} , defense model \mathcal{M}_{D} , and judge model \mathcal{J}_{A} . Figure 2: Overall ACE-Safety architecture. (a) GS-MCTS attack framework. The inset shows the process details for node state s . (b) AC-TGPO training diagram, darker colors for models indicate stronger attack/defense capabilities. • \mathcal{M}_{A} is the attack model trained to rewrite the query for bypassing the defense mechanisms of a

Read the complete text