Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification

Yaoyang Luo, Zhi Zheng, Ziwei Zhao, Tong Xu, Zhao Jielun, Wenjun Xue, Yong Chen, Enhong Chen · 2026

Read Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification by Yaoyang Luo, Zhi Zheng, Ziwei Zhao, Tong Xu, Zhao Jielun, Wenjun Xue, Yong Chen, Enhong Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28104.txt.

Introduction

Luo and colleagues show that hostile agents in multi-agent language systems do more damage when they coordinate than when they act alone. Their STAR defense spots and fixes misleading sentences inside agent messages, rather than judging whole replies. In tests it blunted both solo and team attacks and lifted task success by over a third on average.

Catalog record

CREATORYaoyang Luo, Zhi Zheng, Ziwei Zhao, Tong Xu, Zhao Jielun, Wenjun Xue, Yong Chen, Enhong Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45042
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.28104

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification Yaoyang Luo, Zhi Zheng, Ziwei Zhao, Tong Xu, Zhao Jielun, Wenjun Xue, Yong Chen, Enhong Chen Abstract Recent years have witnessed the rapid development of Large Language Model-based Multi-Agent Systems (MAS), which excel at collaborative decision-making and complex problem-solving. However, malicious agents in MAS may inject misinformation to mislead other agents and disrupt system performance, giving rise to a new research direction that focuses on attack mechanisms and defense strategies in MAS. Prior studies largely assume malicious agents act independently and investigate the corresponding defense strategies. However, we argue that malicious agents may exhibit collaborative behaviors, enabling more effective attacks through internal information exchange. In this paper, we propose an adaptive cooperative attack framework, where malicious agents autonomously coordinate and dynamically adjust their attack strategies through multi-round interactions. Furthermore, we introduce S entence-Level T rustworthiness A nalysis and R ectification ( STAR ), a defense framework that identifies and rectifies misleading information at the sentence level within agent communications. Our experiments show that cooperative attacks lead to a significantly larger degradation in task success rate than independent attacks, resulting in a relative drop of 5.34%. Meanwhile, STAR effectively mitigates both cooperative and independent threats and improves task success rate by an average of 36.76%. The code is available at https://github.com/smoooom/STAR . 1 Introduction Large Language Model (LLM)-based agent systems have rapidly advanced and become a key foundation for complex intelligent applications, demonstrating strong performance in question answering, decision-making, and task execution ( 34 ; 31 ) . Extending this paradigm, Multi-Agent Systems (MAS) enable explicit communication and collaboration among multiple agents, thereby enhancing their capacity for complex problem-solving, information integration, and coordinated decision-making ( 8 ; 33 ) . Figure 1: Comparison of the Independent Attack and the Cooperative Attack. However, such collaborative architectures introduce new security vulnerabilities. The frequent inter-agent communication and complex interaction structures render MAS particularly susceptible to the injection of misleading information ( 5 ; 37 ) . Adversaries can manipulate malicious agents to generate semantically coherent yet factually incorrect misinformation, which propagates through iterative interactions to be repeatedly reinforced, ultimately disrupting the overall system behavior ( 12 ; 36 ) . Consequently, such attacks have emerged as a critical threat to the reliability and trustworthiness of MAS ( 1 ; 25 ) . Recent studies have investigated information injection attacks in MAS, including both independent adversarial behaviors ( 11 ; 12 ) and coordinated collusive attacks among malicious agents ( 28 ) . However, existing attack settings primarily focus on either independent attacks or predefined coordination patterns, lacking adaptive collaborative strategy selection during multi-round interactions. In parallel, corresponding defensive strategies have been proposed to mitigate these threats ( 36 ) . A representative example is G-Safeguard ( 32 ) , which employs graph neural networks to identify and isolate suspicious agents within the MAS. However, these methods still exhibit several limitations: most approaches operate primarily at the agent level, making it difficult to identify localized misinformation, and they do not explicitly extract or correct erroneous content, resulting in limited interpretability of the defense process. Motivated by the above limitations, to achieve more effective information injection attacks, we introduce an adaptive Cooperative Attack Method in which malicious agents dynamically coordinate their attack strategies through ally-aware interactions. As illustrated in Figure 1 , in this method, an adversary simultaneously controls multiple malicious agents that share attack objectives. These agents continuously observe and analyze the responses of their malicious allies to adaptively adjust their attack strategies during multi-round interactions, choosing to either endorse allied arguments or introduce new deceptive perspectives. Meanwhile, to further enhance the robustness of existing defense frameworks, we propose STAR (Sentence-Level Trustworthiness Analysis and Rectification), a training-free defense framework that scrutinizes agent responses sentence-by-sentence to explicitly localize and rectify unreliable information. Furthermore, it identifies agents propagating misinformation as suspicious and excludes their answers when the MAS generates the final output. Consequently, this approach provides an interpretable defense capability and effectively mitigates the effects of both cooperative and independent attacks on MAS. In summary, our contributions can be summarized as follows: • Cooperative Attack Method. We introduce a cooperative attack method where multiple agents adaptively coordinate their strategies, resulting in significantly more effective disruption than independent attacks. • Sentence-Level Defense Framework. We propose STAR, a framework that performs sentence-level analysis to explicitly localize and correct misinformation against various attacks. • Extensive Empirical Evaluation. Extensive experiments across diverse LLM backbones and datasets demonstrate the effectiveness of cooperative attacks and the STAR defense framework. 2 Preliminary In this section, we formally define the LLM-based Multi-Agent System framework, followed by the adversarial attack and defense. 2.1 MAS Framework Consider a multi-agent system for Question Answering, comprising N agents denoted as \mathcal{A}=\{a_{1},\dots,a_{N}\} . Given an input question q , the system aims to derive a final answer \hat{y} through multi-round communication and discussion ( 16 ; 18 ) . Communication Topology. Following previous works ( 41 ; 23 ) , we define the multi-agent interaction structure as an undirected graph \mathcal{G}=(\mathcal{A},\mathcal{E}) . For any agent a_{i} , its neighbor set is \mathcal{N}_{i}=\{a_{j}\mid(a_{j},a_{i})\in\mathcal{E}\} , indicating that a_{i} receives information from a_{j} . Discussion Process. The discussion spans T dialogue rounds. At each round t , every agent a_{i} generates a response r_{i}^{(t)} driven by a pre-trained Large Language Model, denoted as \text{LLM}(\cdot) . Agent a_{i} synthesizes the question q , its own historical memory \mathcal{M}_{i}^{(t-1)} , and responses from neighbors to construct a reasoned response: r_{i}^{(t)}=\text{LLM}\left(q,\mathcal{M}_{i}^{(t-1)},\mathcal{O}_{\mathcal{N}_{i}}^{(t-1)}\right), (1) where \mathcal{M}_{i}^{(t-1)} encapsulates the interaction history of agent a_{i} , and \mathcal{O}_{\mathcal{N}_{i}}^{(t-1)}=\{r_{j}^{(t-1)}\mid a_{j}\in\mathcal{N}_{i}\} denotes the set of responses received from the neighbors of agent a_{i} . Answer Aggregation. After T rounds, the system aggregates the final responses via an aggregation function \Phi(\cdot) to determine the outcome: \hat{y}=\Phi\left(\{r_{1}^{(T)},\dots,r_{N}^{(T)}\}\right). (2) In this paper, we adopt majority voting ( 16 ) as the aggregation function: \hat{y}=\underset{y\in\mathcal{Y}}{\arg\max}\sum_{i=1}^{N}\mathbb{I}(f(r_{i}^{(T)})=y), (3) where f(\cdot) parses the extracted answer from the text and \mathbb{I}(\cdot) is the indicator function. 2.2 Adversarial Attack Attacker Capability. The attacker controls a subset of agents \mathcal{C}\subset\mathcal{A} , referred to as malicious agents, while the remaining agents \mathcal{B}=\mathcal{A}\setminus\mathcal{C} remain benign. Specifically, the attacker employs direct or indirect prompt injection attacks ( 22 ; 35 ) to inject malicious information or instructions into the prompts fed to the agents, thereby manipulating these agents to exhibit unintended behaviors. In this paper, the attacker can control these malicious agents to output diverse types of erroneous or factually incorrect misinformation. Attack Strategy. In this paper, we define the ultimate goal of the attacker as maximizing the probability that the system outputs a pre-set target wrong answer. To achieve this, the attacker manipulates malicious agents to propagate misinformation, aiming to mislead the ongoing discussion. Consequently, instead of adhering to the faithful reasoning process, a malicious agent a_{k}\in\mathcal{C} constructs a deceptive response \tilde{r}_{k}^{(t)} : \tilde{r}_{k}^{(t)}=\pi_{\text{adv}}\left(q,\mathcal{M}_{k}^{(t-1)},\mathcal{O}_{\mathcal{N}_{k}}^{(t-1)}\right), (4) where \pi_{\text{adv}}(\cdot) denotes the adversarial strategy employed to fabricate misleading content. 2.3 MAS Defense The primary objective of the defender is to shield benign agents from the detrimental influence of misinformation propagated by malicious agents, ultimately ensuring that the MAS operates normally and completes tasks successfully. To achieve this, the defense process comprises two stages: Malicious Agent Detection and

Read the complete text