ResMAS: Resilience Optimization in LLM-based Multi-agent Systems

Zhilun Zhou, Zihan Liu, Jiahe Liu, Qingyu Shao, Yihan Wang, Kun Shao, Depeng Jin, Fengli Xu · 2026

Read ResMAS: Resilience Optimization in LLM-based Multi-agent Systems by Zhilun Zhou, Zihan Liu, Jiahe Liu, Qingyu Shao, Yihan Wang, Kun Shao, Depeng Jin, Fengli Xu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-04694.txt.

Introduction

Zhilun Zhou and seven co-authors design LLM multi-agent systems to survive their own members failing, in a 2026 arXiv paper. Fleets of cooperating agents now solve hard tasks, but they run scattered across devices and environments where agents crash or get perturbed. Prior work mostly plays defence after an attack; this team instead asks what makes a system resilient by construction, and finds two levers matter: the communication topology and each agent's prompt.

ResMAS builds resilience in two stages. First, a reward model learned to predict a system's resilience score trains a topology generator, via reinforcement learning, that designs robust communication graphs for the task at hand. Second, a topology-aware prompt optimization refines each agent's instructions according to its connections and interactions. Experiments across tasks show substantially better survival under varied constraints, and the framework transfers to new tasks and new models.

Catalog record

CREATORZhilun Zhou, Zihan Liu, Jiahe Liu, Qingyu Shao, Yihan Wang, Kun Shao, Depeng Jin, Fengli Xu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-04694

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ResMAS: Resilience Optimization in LLM-based Multi-agent Systems Zhilun Zhou, Zihan Liu, Jiahe Liu, Qingyu Shao, Yihan Wang, Kun Shao, Depeng Jin, Fengli Xu Abstract Large Language Model-based Multi-Agent Systems (LLM-based MAS), where multiple LLM agents collaborate to solve complex tasks, have shown impressive performance in many areas. However, MAS are typically distributed across different devices or environments, making them vulnerable to perturbations such as agent failures. While existing works have studied the adversarial attacks and corresponding defense strategies, they mainly focus on reactively detecting and mitigating attacks after they occur rather than proactively designing inherently resilient systems. In this work, we study the resilience of LLM-based MAS under perturbations and find that both the communication topology and prompt design significantly influence system resilience. Motivated by these findings, we propose ResMAS: a two-stage framework for enhancing MAS resilience. First, we train a reward model to predict the MAS’s resilience, based on which we train a topology generator to automatically design resilient topology for specific tasks through reinforcement learning. Second, we introduce a topology-aware prompt optimization method that refines each agent’s prompt based on its connections and interactions with other agents. Extensive experiments across a range of tasks show that our approach substantially improves MAS resilience under various constraints. Moreover, our framework demonstrates strong generalization ability to new tasks and models, highlighting its potential for building resilient MASs. Code — https://github.com/tsinghua-fib-lab/ResMAS Introduction Large language model-based multi-agent systems (LLM-based MAS; hereafter referred to as MAS ) have exhibited impressive capabilities across a wide range of domains, including software development ( 18 ; 10 ) , mathematical problem solving ( 17 ) , and operations research ( 27 ) . In an MAS, multiple LLM agents collaborate to solve complex tasks through certain role-play and communication patterns, often achieving superior performance compared to single-agent approaches. Despite these advances, agents within an MAS are typically deployed across different devices or environments, making the MAS vulnerable to perturbations and attacks. As a result, the resilience, defined as the ability of a system to maintain its functionality under perturbations ( 3 ; 7 ; 4 ) , has become an important issue for MAS. In recent years, there have been extensive studies about resilience of MAS under adversarial attacks. Several studies propose attack strategies such as prompt injection ( 29 ; 12 ) and knowledge editing ( 13 ; 14 ) , and find that harmful information can rapidly spread in MAS ( 8 ; 16 ) . Other works focus on detecting and defending against attacks using techniques like psychological testing ( 32 ) and graph-based anomaly detection ( 25 ) . However, existing studies mainly focus on the safety issue of MAS caused by adversarial attacks, while largely overlooking more common, non-malicious errors in MAS such as failures or miscommunication. Furthermore, prior works concentrated on detecting and mitigating attacks, with little focus on proactively designing resilient MAS structures that can better tolerate errors. In this work, we study the resilience of MAS under random agent failures. We find that MAS exhibits significantly higher resilience than a single agent, and the resilience increases with the number of agents and communication links. Moreover, we find that both the topology and prompts of an MAS have a strong impact on its resilience. Motivated by these, we propose a two-stage framework to automatically optimize the topology and prompts to improve the resilience of MAS, named ResMAS . In the first stage, we fine-tune an open-source LLM serve as a topology generator capable of producing resilient MAS structures. Specifically, to avoid the high time cost of evaluating an MAS’s resilience, we train a GNN-based reward model to predict the resilience of MAS for various tasks. This reward model is then used to fine-tune the LLM using the Group Relative Policy Optimization (GRPO) ( 20 ) algorithm, enabling it to automatically generate resilient MAS topologies. In the second stage, we propose a topology-aware prompt optimization method to optimize the system prompt for each agent in the MAS based on the generated topology. Specifically, we apply the MAS to a training set of the target task, and refine the system prompt for each agent based on the prompts of its neighbors as well as its interaction history with neighbors. We conduct experiments on three benchmarks, including commonsense reasoning, math, and game. The results demonstrate that our framework can generate more resilient MAS compared to existing approaches for prompt and topology optimization. Moreover, we highlight the versatility of our framework by applying it to optimize the accuracy of MAS on certain tasks, where it achieves the Pareto-optimal performance compared with baselines. In addition, our framework shows strong generalization ability to unseen tasks such as code generation, and to new agents built upon different backbone models. In summary, our contributions are as follows: • We systematically study the resilience of MAS under random agent failures, revealing that both the topology and agent prompts have significant influence on system resilience. • We propose ResMAS: a two-stage framework that automatically optimizes the topology and agent prompts of MAS to enhance resilience. Specifically, we fine-tune an LLM to generate robust topologies, and further refine the prompts in a topology-aware manner. • Extensive experiments on various tasks show that our framework is able to generate MAS with superior resilience and demonstrates strong generalization ability across tasks and models. Related Work Resilience of MAS The distributed architecture of MASs inherently increases their susceptibility to perturbations. Prior research has focused on adversarial attack and defense methods for MAS. Specifically, some studies propose attack methods such as prompt injection ( 29 ; 12 ; 23 ) and knowledge editing ( 13 ; 14 ) . Additionally, it has been demonstrated that harmful information can spread extremely fast in MAS, where a single jailbreak agent can undermine the whole MAS ( 8 ; 16 ) . In response, some studies focus on detecting and defending against attacks through methods like psychological test ( 32 ) and graph-based anomaly detection ( 25 ) . However, existing studies mainly focus on the safety issue of MAS caused by adversarial attack, but overlook the resilience to more common, non-malicious agent errors. Furthermore, prior works emphasize detecting and mitigating attacks, with little focus on proactively designing resilient MAS structures that can tolerate errors by design. Automated Design of MAS The architecture of MAS, including the agents’ roles and prompts as well as the communication patterns between agents, significantly affects the performance of MAS on tasks. Early studies often manually design MAS structures for specific tasks, usually based on human collaboration mechanisms ( 6 ; 10 ) . Recently, there have been more studies on automated design and optimization of MAS. For instance, Agentverse lets LLM generate and adjust the agent composition based on the status of the task ( 2 ) . G-designer proposes to optimize the communication network of agents through a variational graph auto-encoder ( 31 ) . GPTSwarm represents multi-agent systems as composite graphs and optimizes node-level prompts as well as edges between agents ( 33 ) . However, all of the existing methods focus on optimizing the performance of MAS on certain tasks rather than resilience. Preliminaries Motivation In this study, we

Read the complete text