Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Safety
Zhenyu Pan, Yiting Zhang, Yutong Zhang, Jianshu Zhang, Haozheng Luo, Yuwei Han, Dennis Wu, Hong-Yu Chen, Philip S. Yu, Manling Li, Han Liu · 2025
Read Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Safety by Zhenyu Pan, Yiting Zhang, Yutong Zhang, Jianshu Zhang, Haozheng Luo, Yuwei Han, Dennis Wu, Hong-Yu Chen, Philip S. Yu, Manling Li, Han Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-03864.txt.
Introduction
Zhenyu Pan and co-authors present Evo-MARL, a training scheme that builds safety into teams of AI agents, in a 2025 paper. Teams of language-model agents now work together on hard tasks, but their open links invite jailbreaks and attacks. Guard agents are the usual defense, yet a single guard is a weak point whose failure can sink the whole team.
Evo-MARL trains every agent to do its job and resist attacks at the same time. Attackers and defenders evolve together, so defenses keep sharpening as threats change. The training shares settings across agents to keep costs down. In tests, attack success fell by up to about a fifth, while accuracy on reasoning tasks rose. Safety and skill grew together, not at each other's cost.
Catalog record
| CREATOR | Zhenyu Pan, Yiting Zhang, Yutong Zhang, Jianshu Zhang, Haozheng Luo, Yuwei Han, Dennis Wu, Hong-Yu Chen, Philip S. Yu, Manling Li, Han Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-03864
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Safety Zhenyu Pan, Yiting Zhang, Yutong Zhang, Jianshu Zhang, Haozheng Luo, Yuwei Han, Dennis Wu, Hong-Yu Chen, Philip S. Yu, Manling Li, Han Liu Abstract Multi-agent systems (MAS) built on multimodal large language models exhibit strong collaboration and performance. However, their growing openness and interaction complexity pose serious risks, notably jailbreak and adversarial attacks. Existing defenses typically rely on external guard modules, such as dedicated safety agents, to handle unsafe behaviors. Unfortunately, this paradigm faces two challenges: (1) standalone agents offer limited protection, and (2) their independence leads to single-point failure—if compromised, system-wide safety collapses. Naively increasing the number of guard agents further raises cost and complexity. To address these challenges, we propose Evo-MARL, a novel multi-agent reinforcement learning (MARL) framework that enables all task agents to jointly acquire defensive capabilities. Rather than relying on external safety modules, Evo-MARL trains each agent to simultaneously perform its primary function and resist adversarial threats, ensuring robustness without increasing system overhead or single-node failure. Furthermore, Evo-MARL integrates evolutionary search with parameter-sharing reinforcement learning to co-evolve attackers and defenders. This adversarial training paradigm internalizes safety mechanisms and continually enhances MAS performance under co-evolving threats. Experiments show that Evo-MARL reduces attack success rates by up to 22% while boosting accuracy by up to 5% on reasoning tasks—demonstrating that safety and utility can be jointly improved. 1 1 1 Github: https://github.com/zhangyt-cn/Evo-MARL . 1 Introduction Large language models (LLMs) and Multimodal Large Language Models (MLLMs) based agents exhibit advanced capabilities such as question answering [ 24 , 25 ] , code generation [ 23 , 26 ] , deep research [ 34 ] , and complex reasoning [ 27 , 22 ] . MAS coordinates heterogeneous, role-specialized agents to harness collective intelligence and improve task performance. However, MAS safety remains a major challenge: these systems inherit vulnerabilities from their foundation models and introduce new risks via inter-agent communication. Attacks—such as manipulated roles, poisoned tools, or indirect prompt injection—can originate from malicious agents and propagate across the interaction graph, where even a single compromise may trigger cascading failures that undermine the entire system. To improve safety, recent approaches often introduce an additional guard module—using either static rules or trained defense agents—to monitor behavior [ 20 ] . While effective to some extent, this strategy has two key limitations: (1) standalone guards provide limited protection when task agents lack safety awareness [ 9 , 38 ] ; and (2) relying on a single guard module creates a potential point of failure and reduces system resilience [ 6 ] . Adding more guards also increases cost and complexity. To address this, we advocate for internalizing defense capabilities within every task agent, fostering collective safety awareness and enabling each agent to contribute to overall system robustness. Figure 1 : Overview of our MARL training and evaluation framework on MAS safety Therefore, we propose Evo-MARL, a framework that uses MARL to collectively train all task agents to internalize safety defenses while preserving task performance. A key innovation is a co-evolutionary mechanism that enables adversarial learning between attackers and defenders: an evolving pool of attack prompts is maintained using evolutionary search (e.g., mutation), guided by their success rates in compromising the MAS. This dynamic pressure fosters more generalizable defense strategies across agents. Specifically, during adversarial training, a subset of agents is perturbed to act as attackers, aiming to compromise the final system output, while the remaining agents serve as defenders, responsible for detecting, mitigating, and correcting unsafe or misleading content. Reward signals are assigned based on both the harmfulness and correctness of the final response, jointly promoting safety and helpfulness. To simulate realistic safety contagion, we design a chain-structured MAS, and manipulate attacked agents via indirect prompt injection. A randomly selected agent is chosen as the initial victim, and attack prompts are sequentially injected into the responses of that agent and its downstream peers. For training simplicity, attackers are treated as external third-party adversaries and are not included in RL optimization. To improve training efficiency, we adopt parameter sharing among agents with role-conditioned policies, and optimize them using Group Relative Policy Optimization (GRPO) [ 35 ] . In summary, our contributions are as follows: (1) We propose Evo-MARL, a novel multi-agent reinforcement learning framework that internalizes safety defenses into each task agent, eliminating reliance on external guard modules and enhancing system-level robustness. (2) We introduce a co-evolutionary training mechanism that continuously pressures agents to learn generalized defense strategies through adversarial interactions with an evolving pool of attack prompts. (3) We empirically validate our method across multi-modal and text-only red team datasets, demonstrating up to 22% improvement in safety and even 5% gains in task performance, highlighting the feasibility of aligning robustness and utility in MAS. 2 Related Works Safety in Multi-Agent Systems. Large Language Models and Multi-modal LLMs are known to exhibit significant safety vulnerabilities, particularly susceptibility to prompt injection attacks [ 16 , 20 , 29 , 7 , 33 ] and adversarial manipulations [ 18 , 17 , 45 ] . As LLM-based agents are increasingly augmented with external modules such as tools and memory systems, they become even more exposed to security threats through these interfaces [ 47 , 36 , 5 , 2 ] . While MAS offers promising capabilities through agent collaboration, it also introduces unique safety concerns. Adversaries can compromise MAS via two main avenues: (1) hijacking individual agents to propagate malicious content throughout the system [ 15 , 48 , 13 ] , and (2) manipulating inter-agent communication and workflow execution [ 9 , 38 , 44 , 8 ] . To address these safety issues, prior work has primarily relied on external safety modules or explicit defense protocols. For example, tse Huang et al. 2025 introduces a dedicated safety inspection agent to monitor and sanitize message streams. Wang et al. 2025 leverages Graph Neural Networks (GNNs) to model communication topologies and detect unsafe message propagation, further applying supervised fine-tuning to enhance detection performance. Fan and Li 2025 proposes a peer-review mechanism, where agents serve as inspectors of each other’s outputs and collectively reject unsafe responses. Although effective to some extent, these approaches do not improve the agents’ intrinsic safety mechanisms. Moreover, reliance on external modules often results in scalability bottlenecks and fragility. In contrast, we advocate for embedding safety awareness directly into agents via reinforcement learning, allowing the entire system to become more robust through internalized safety capabilities. Reinforcement Learning for Agent Training. RL has proven effective in LLM post-training, with methods such as Proximal Policy Optimization (PPO) [ 32 ] and Group Relative Policy Optimization (GRPO) [ 3 ] yielding substantial performance improvements. Recent works have applied RL to enhance agentic behavior: Search-R1 [ 12 ] teaches LLMs to incorporate web search into reasoning, while Wei et al. 2025 and Team et al. 2025 demonstrate large-scale RL training on real-world agentic and software engineering tasks. Multi-agent reinforcement learning methods—such as MAPPO [ 46 ] , QMIX [ 31 ] , and HATRPO [ 14 ] —serve as foundational algorithms for learning coordinated multi-agent policies. Building on this, LLM-based MARL seeks to leverage these coordination capabilities to further enhance system performance. Park et al. 2025 uses MAPPO to