AgentCompass: Towards Reliable Evaluation of Agentic Workflows in Production
NVJK Kartik Garvit Sapra Rishav Hada Nikhil Pareek FutureAGI Inc. {kartik.nvj,garvit.sapra,rishav,nikhil}@futureagi.com · 2025
Read AgentCompass: Towards Reliable Evaluation of Agentic Workflows in Production by NVJK Kartik Garvit Sapra Rishav Hada Nikhil Pareek FutureAGI Inc. {kartik.nvj,garvit.sapra,rishav,nikhil}@futureagi.com (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-14647.txt.
Introduction
NVJK Kartik and three co-authors present AgentCompass in 2025, a test frame for agentic workflows that are already live in firms. Multi agent systems built on large language models fail in emergent, system wide ways that launch tests miss. Firms need checks and debugging built for agents in use.
AgentCompass copies expert debuggers in stages: find and sort errors, group them by theme, score them with numbers, then sum up a plan. A dual memory, episodic and semantic, carries lessons from case to case. The pipe turns scattered agent failures into ranked, clear findings. Owners of live agents gain a repeat audit loop, not fire drills driven by tales after each mishap.
Catalog record
| CREATOR | NVJK Kartik Garvit Sapra Rishav Hada Nikhil Pareek FutureAGI Inc. {kartik.nvj,garvit.sapra,rishav,nikhil}@futureagi.com |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-14647
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AgentCompass: Towards Reliable Evaluation of Agentic Workflows in Production NVJK Kartik Garvit Sapra Rishav Hada Nikhil Pareek FutureAGI Inc. {kartik.nvj,garvit.sapra,rishav,nikhil}@futureagi.com Abstract With the growing adoption of Large Language Models (LLMs) in automating complex, multi-agent workflows, organizations face mounting risks from errors, emergent behaviors, and systemic failures that current evaluation methods fail to capture. We present AgentCompass, the first evaluation framework designed specifically for post-deployment monitoring and debugging of agentic workflows. AgentCompass models the reasoning process of expert debuggers through a structured, multi-stage analytical pipeline: error identification and categorization, thematic clustering, quantitative scoring, and strategic summarization. The framework is further enhanced with a dual memory system—episodic and semantic—that enables continual learning across executions. Through collaborations with design partners, we demonstrate the framework’s practical utility on real-world deployments, before establishing its efficacy against the publicly available TRAIL benchmark. AgentCompass achieves state-of-the-art results on key metrics, while uncovering critical issues missed in human annotations, underscoring its role as a robust, developer-centric tool for reliable monitoring and improvement of agentic systems in production. 1 Introduction With recent advancements in Large Language Model’s reasoning capabilities, they are increasingly being used for complex reasoning tasks such as agentic workflows. Agentic workflows allow users to automate certain tasks, based on internal thinking and reasoning capabilites of LLMs. These workflows range from being extremely simple such as retrieving a knowledge-base answer to a customer query, to very complex such as multi-step decision making in supply chain optimization or coordinating multiple specialized agents in a financial analysis pipeline. Recent surveys by Wakefield Research PagerDuty 2025 and other organizations EIN Presswire / Market.us 2025 show, agentic workflows have surged dramatically across industries in 2025, with statistics showing widespread adoption, hefty return on investment (ROI) gains, and rapid scaling in enterprise environments. Organizations are deploying agentic workflows across critical business functions. Automation enabled by agentic workflows has resulted in 20–30% cost savings for organizations, freeing resources for strategic growth Belcak et al. 2025 ; Yu et al. 2025 . However, there are some major roadblocks in widespread adoption of agentic workflows. Organizations face a range of technical, organizational, and operational roadblocks. Agents often make decisions using non-standardized paths, destabilizing established workflows and making process oversight difficult in regulated environments. Organizations are concerned about bias, and poor-quality or fragmented datasets slow reliable adoption and amplify systemic errors. Poor evaluation and systems breaking in production are major causes of financial and reputational damage to organizations adopting agentic AI Wang et al. 2025 ; Liu et al. 2025 ; Chaudhry et al. 2025 . Insufficient evaluation methods, over-reliance on technical benchmarks, and inadequate real-world robustness testing often leave organizations unprepared for edge cases and contextual failures. Most current evaluation frameworks prioritize technical metrics (accuracy, speed) and neglect stress tests for human-centered context, edge cases, and emotional intelligence, which are critical for user experience and trust. Errors and biases often compound and propagate through multi-agent workflows, making correction and accountability complex. Unaddressed failures can cause legal liabilities and loss of competitive advantage Moteki et al. 2025 ; Meimandi et al. 2025 ; Raza et al. 2025 ; Yehudai et al. 2025 ; Shukla 2025 . To address this gap, recent research focuses on developing robust evaluation frameworks, that not only measure accuracy on a controlled dataset, but also includes real world scenarios with subjective evaluation metrics. For example, GAIA proposes a benchmark for General AI Assistants Mialon et al. 2023 . GAIA focuses on real-world tasks that require reasoning, multi-modality, web browsing, and effective use of external tools. This moves beyond traditional benchmarks, which often focus on narrow tasks (like text classification or image recognition), by emphasizing an AI agent’s ability to generalize, adapt to changing scenarios, perform complex multi-step operations, and interact with diverse information sources. TRAIL Deshpande et al. 2025 extends the GAIA benchmark by providing an error taxonomy for agentic executions with an annotated dataset demonstrating the practical usage of the proposed taxonomy. Most organizations are unprepared for the complexities of agentic and multi-agentic AI risks, with governance blind spots multiplying post-deployment. These gaps often manifest as cascading failures, emergent behaviors, and unanticipated interactions with external systems, which existing monitoring and control frameworks struggle to detect or mitigate. In this work, we present AgentCompass, the first evaluation framework purpose-built for monitoring and debugging agentic workflows in production. Unlike prior approaches that rely on static benchmarks or single-pass LLM judgments, AgentCompass employs a recursive plan-and-execute reasoning cycle, a formal hierarchical error taxonomy tailored to business-critical issues, and a dual memory system that enables longitudinal analysis across executions. Its multi-stage analytical pipeline spans four key workflows: error identification and categorization, thematic error clustering, quantitative quality scoring, and synthesis with strategic summarization. To capture systemic risks, the framework further performs trace-level clustering via density-based analysis, surfacing recurring failure patterns that can be prioritized by engineering teams. Empirically, we validated AgentCompass with design partners on real-world deployments and demonstrated its generality through the publicly available TRAIL benchmark, where it achieves state-of-the-art results in error localization and joint metrics while uncovering critical errors that human annotations miss, including safety and reflection gaps. Together, these contributions make AgentCompass a rigorous and practical foundation for real-time production monitoring and continual improvement of complex agentic systems. 2 Methodology We propose an agentic framework for the automated analysis of traces. Our approach models the cognitive workflow of an expert human debugger, creating a structured, multi-stage analytical pipeline for data that is unstructured as traces stored in open-telemetry format. The core of our method is a recursive reasoning framework that enhances the reliability of Large Language Models for complex diagnostic tasks. This framework is augmented with a knowledge persistence layer for continual learning. Our method includes 3 core components: a multi-stage analytical pipeline, trace-level issue clustering, and knowledge persistence for continual learning. We describe each of these components in detail in this section. 2.1 The Multi-Stage Analytical Pipeline Our method deconstructs the complex goal of trace analysis into a sequence of four distinct, progressively abstract stages. Each stage’s output serves as the structured input for the next, creating a coherent analytical narrative from granular findings to a high-level strategic summary. 1. Error Identification and Categorization: The initial stage performs a comprehensive scan of the entire execution trace to identify discrete errors. Each identified error is classified according to a formal, hierarchical error taxonomy, grounding the analysis in a predefined set of failure modes common to agentic systems. This taxonomy is described in detail in section 2.1.1 . 2. Thematic Error Clustering: The discrete errors identified in the previous stage are then subjected to a thematic analysis. The system groups individual errors into semantically coherent clusters, designed to reveal systemic issues, causal chains, or recurring patterns of failure that would not be apparent from isolated error events. 3. Quantitative Quality Scoring: To move beyond qualitative error descriptions, this stage assesses the overall quality of the trace across several predefined dimensions (e.g., factual grounding, safety, plan execution). The system assigns a quantitative score to each dimension, providing a multi-faceted, objective measure of the agent’s performance. 4. Synthesis and Strategic Summarization: The final stage synthesizes all preceding data —individual errors, thematic clusters, and quantitative scores into a final, actionable summary. This includes a single, aggregate