Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems

Shaokun Zhang, Ming Yin, Jieyu Zhang, Jiale Liu, Zhiguang Han, Jingyang Zhang, Beibin Li, Chi Wang, Huazheng Wang, Yiran Chen, Qingyun Wu · 2025

Read Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems by Shaokun Zhang, Ming Yin, Jieyu Zhang, Jiale Liu, Zhiguang Han, Jingyang Zhang, Beibin Li, Chi Wang, Huazheng Wang, Yiran Chen, Qingyun Wu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-00212.txt.

Introduction

Shaokun Zhang and a large team set a new task: name which agent in a model team broke a job, and at which step. Such blame work is now done by hand and slow. Their 2025 set, Who&When, holds failure logs from 127 model teams, marked to the agent and the decisive step. Three blame methods are built and graded on it.

The best method names the agent at 53.5 percent accuracy, yet the step at just 14.2 percent. Some methods fall below chance. Top reasoning models fail the practical use bar of the task too. The gap shows blame work as a hard, fresh field with a bench now set. Code and data sit free.

Catalog record

CREATORShaokun Zhang, Ming Yin, Jieyu Zhang, Jiale Liu, Zhiguang Han, Jingyang Zhang, Beibin Li, Chi Wang, Huazheng Wang, Yiran Chen, Qingyun Wu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-00212

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems Shaokun Zhang, Ming Yin, Jieyu Zhang, Jiale Liu, Zhiguang Han, Jingyang Zhang, Beibin Li, Chi Wang, Huazheng Wang, Yiran Chen, Qingyun Wu Abstract Failure attribution in LLM multi-agent systems—identifying the agent and step responsible for task failures—provides crucial clues for systems debugging but remains underexplored and labor-intensive. In this paper, we propose and formulate a new research area: automated failure attribution for LLM multi-agent systems. To support this initiative, we introduce the Who&When dataset, comprising extensive failure logs from 127 LLM multi-agent systems with fine-grained annotations linking failures to specific agents and decisive error steps. Using the Who&When, we develop and evaluate three automated failure attribution methods, summarizing their corresponding pros and cons. The best method achieves 53.5% accuracy in identifying failure-responsible agents but only 14.2% in pinpointing failure steps, with some methods performing below random. Even SOTA reasoning models, such as OpenAI o1 and DeepSeek R1, fail to achieve practical usability. These results highlight the task’s complexity and the need for further research in this area. Code and dataset are available in the public repository . † † affiliationnotice: Equal contribution 1 Introduction In recent years, the reframing Large Language Models (LLMs) as agents and built agentic multi-agent systems—composed of interactive, LLM-powered agents collaborating to achieve shared goals—has garnered significant attention ( Wu et al. 2023 ; Li et al. 2023a ; Hong et al. 2023 ) . These purposefully designed agentic systems have demonstrated remarkable potential across various domains, including coding ( Hong et al. 2023 ) , scientific discovery ( Ghafarollahi & Buehler 2024 ) , and addressing complex real-world challenges ( Fourney et al. 2024 ) . Once constructed, these systems are typically refined through an iterative process when they fail in specific scenarios: evaluation against established benchmarks, followed by manual failure attribution and system refinement. This cycle repeats until the desired outcomes are achieved. Failure attribution—identifying the components of the system that directly lead to task failures—is a crucial step that serves as the foundation for guiding improvements. Despite its importance, this process is largely overlooked in mainstream research, which typically leaves it as a manual task requiring significant labor, such as analyzing complex historical logs and navigating the technical intricacies of the system. Moreover, mapping benchmark evaluation results to failure components is heavily dependent on domain expertise, imposing additional requirements on practitioners. As systems grow in complexity, this challenge becomes increasingly difficult due to the growing number of components that must be considered when conducting failure attribution. Figure 1: When developing LLMs-powered multi-agent systems, failure attribution—identifying system components responsible for task failures based on evaluation results—has received limited attention in existing research. This process is typically performed manually, demanding substantial labor and specialized expertise. In this study, we explore the potential for automating this process. Previous manual efforts involve a non-straightforward way to facilitate failure attribution in multi-agent systems: developing increasingly fine-grained benchmarks, with the hope that more metrics will enable quicker failure attribution ( Zhuge et al. 2024 ) . For example, DevAI ( Zhuge et al. 2024 ) introduces a coding benchmark that incorporates diverse delivery requirements, offering a more nuanced evaluation compared to the widely used SWE-Bench ( Jimenez et al. 2023 ) , which focuses exclusively on final resolution rates. Despite these advancements, the process of failure attribution based on benchmark results remains a manual process, merely providing more metrics as reference points without fundamentally addressing the underlying challenges. With increasingly comprehensive benchmarks, a fundamental question remains unanswered: which components of the agentic system require improvement? We argue that evaluation and failure attribution should be tightly integrated, adhering to the principle that “evaluation is not an end in itself, but a means to improvement.” ( Scriven 1991 ) More research efforts should focus on bridging the substantial gap between evaluation results and failure attribution, which currently relies heavily on manual labor. Drawing inspiration from the LLM-as-a-judge paradigm ( Zheng et al. 2023 ; Gu et al. 2024 ) , which leverages LLMs to replace human effort in evaluation, we propose to bridge the gap between evaluation and failure attribution by harnessing the comprehensive judgment capabilities of LLMs. Specifically, we propose and formulate a new research problem: automated failure attributions in LLM multi-agent systems. When failures occur under specific scenarios during evaluation, the goal is to automatically identify the components responsible for these failures without human intervention. We believe this research could serve as a substitute for manual failure attribution, enabling human resources to focus on improving system functionality rather than performing time-intensive diagnostics as shown in Figure 1 To advance the research in this area, we introduce the Who&When benchmark, comprising extensive failure logs annotated with fine-grained failure details for addressing real-world tasks, where these logs are collected from 127 LLMs-powered multi-agent systems. This benchmark includes both algorithmically generated and hand-crafted multi-agent systems, encompassing a wide range of realistic scenarios. Each failure log is accompanied by meticulous annotations, specifying the failure-responsible agent, the corresponding failure step, and the reasons for failure in plain language. The primary task involves pinpointing the agent most accountable for the failure and the exact step where the error occurred. Predicting the failure-responsible agent serves as a fundamental requirement for failure attribution, given that agents are the basic units of multi-agent systems. Extending this to the specific failure step prediction imposes a higher level of requirement, enabling more fine-grained failure attribution, such as uncovering the specific reasons behind failures, which can further facilitate targeted system refinements. We believe that Who&When can serve as a foundational resource for driving progress in automated failure attribution research. Additionally, we construct and evaluate several automated failure attribution approaches on the Who&When. Our findings reveal the strengths and limitations of each method, as well as their performance across different conditions, including model variations, historical context lengths, and the presence or absence of query labels. The results underscore the complexity of using LLMs for failure analysis in multi-agent systems. For example, the best-performing method achieved only 8.77% accuracy in identifying decisive error steps within the hand-crafted agentic system. 2 Problem Formulation: Automated Failure Attribution in Multi-Agent Cooperation In this section, we introduce decisive errors and formulate the automated failure attribution problem. We adopt the widely-adopted turn-based LLM multi-agent protocol ( Wu et al. 2023 ; Li et al. 2023a ; Hong et al. 2023 ) . Background. Considering a LLMs-powered multi-agent system \mathcal{M} with a group of N agents, denoted as \mathcal{N}=\{1,2,..N\} , that operate at discrete time steps. These agents are taking actions in a turn-based protocol, meaning that exactly one agent performs an action at each time step. Formally, the system is described as: \displaystyle\mathcal{M}\;=\;\Bigl\langle\mathcal{N},\,S,\,A,\,P,\,\phi\Bigr\rangle. (1) Here, S is the set of possible states. A is the global action set; each agent i\in\mathcal{N} can typically perform actions from some subset A_{i}\subseteq A . \phi(t) is a function that indicates which agent is active at time t , thus specifying the turn-based rule. P\bigl(s_{t+1}\mid s_{t},\,a_{t},\,\phi(t)\bigr) is the state-transition probability, given that only one agent \phi(t) acts at time t . We employ \phi(t) to denote the agent that takes an action a_{t} at time step t

Read the complete text