AgentRx : Diagnosing AI Agent Failures from Execution Trajectories

Shraddha Barke, Arnav Goyal, Alind Khare, Avaljot Singh, Suman Nath, Chetan Bansal · 2026

Read AgentRx : Diagnosing AI Agent Failures from Execution Trajectories by Shraddha Barke, Arnav Goyal, Alind Khare, Avaljot Singh, Suman Nath, Chetan Bansal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02475.txt.

Description

Shraddha Barke and 5 co-authors build a diagnostic system that finds the exact step where an AI agent went wrong, in a 2026 arXiv paper. Agent failures are hard to localize because runs are probabilistic, long, multi-agent, and polluted by noisy tool outputs. Human annotation of failures is expensive but necessary for ground truth.

The team manually annotated one hundred seventy failed trajectories across eleven task settings, from API workflows to incident management and open web tasks. Each run is labeled with its critical failure step and a category from a cross-domain taxonomy. Their automated framework, AgentRx, then synthesizes constraints, checks them step by step, and produces an auditable log of violations. A language-model judge uses that log to localize the failure. Step localization improved by about seventy-five percent on average over prior work.

Catalog record

CREATORShraddha Barke, Arnav Goyal, Alind Khare, Avaljot Singh, Suman Nath, Chetan Bansal
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE95 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02475

Browse the full library · About the Tower of Babel