Monitoring Agentic Systems Before They’re Reliable
Marisa Ferrara Boston, Glen Hanson, Effi Georgala, JD Hudgens, Heather Frase · 2026
Read Monitoring Agentic Systems Before They’re Reliable by Marisa Ferrara Boston, Glen Hanson, Effi Georgala, JD Hudgens, Heather Frase (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-02494.txt.
Description
Boston and co-authors argue that young agentic systems fail in structure, not in tasks. Production agents are often partial assemblies. Structural defects dominate the failure landscape. Task-level monitors then lose their signal. The team builds a monitoring and triage method for this stage.
Evaluation splits into three dimensions: quality, suitability, and efficiency. Monitoring runs at three scopes: within-run, cross-run, and structural. Variance serves as the core signal. Severity classes borrowed from FMEA route the findings. A synthetic testbed holds 220 runs over 120 document bundles. Controlled errors are injected. Scope decides what a monitor finds. Within-run monitors surface stage defects. Cross-run monitors surface integration fallout. A structural monitor spots an integration gap with perfect consistency. Injected task errors stay hidden against clean baselines. Triage sends most findings to automated tracking. The authors urge teams to deploy monitoring early.
Catalog record
| CREATOR | Marisa Ferrara Boston, Glen Hanson, Effi Georgala, JD Hudgens, Heather Frase |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.02494