TraceGraph : Shared Decision Landscapes for Diagnosing and Improving Agent Trajectories
Junjie Nian 1*, Kang Chen 1*, Ge Zhang 2, Yixin Cao 1,3 † \dagger, Yugang Jiang 1 · 2026
Read TraceGraph : Shared Decision Landscapes for Diagnosing and Improving Agent Trajectories by Junjie Nian 1*, Kang Chen 1*, Ge Zhang 2, Yixin Cao 1,3 † \dagger, Yugang Jiang 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-31308.txt.
Introduction
Nian and colleagues turn agent trajectories into shared decision landscapes with TraceGraph. Benchmarks record rich rollouts yet score them by pass rate alone, hiding where models diverge. For each task, TraceGraph graphs states seen in pooled rollouts before model identity enters, then overlays productive cores and trap regions. Rollouts are summarised by access, trap exposure and repair. A recovery pipeline aware of traps, built for SWE-bench, lifts resolved rates on fired states, showing how failure regions can guide improvement.
Catalog record
| CREATOR | Junjie Nian 1*, Kang Chen 1*, Ge Zhang 2, Yixin Cao 1,3 † \dagger, Yugang Jiang 1 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65360 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.31308
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TraceGraph : Shared Decision Landscapes for Diagnosing and Improving Agent Trajectories Junjie Nian 1*, Kang Chen 1*, Ge Zhang 2, Yixin Cao 1,3 † \dagger, Yugang Jiang 1 Abstract Agent benchmarks increasingly record rich interaction trajectories, yet evaluation often reduces each rollout to a pass rate or reward score. We introduce TraceGraph , a graph-based framework that turns released multi-model agent trajectories into shared decision landscapes . For each task, TraceGraph builds a graph over observable action–observation states from pooled rollouts before model identity is introduced. It then overlays outcome-informed productive cores and trap regions, and summarizes each rollout with three events: Access, Trap exposure, and Repair. Across trajectories spanning five benchmark splits, TraceGraph profiles reveal navigation differences hidden by aggregate scores and show that splits differ in whether they reward avoiding traps or recovering from them. The same TraceGraph landscape also motivates a trap-aware recovery pipeline for SWE-bench : a runtime detector fires on states matching historical trap regions, then lightweight continuation policies are evaluated from the same prefix. On fired states, the best pooled single-factor policy raises official resolved rate from 40.4 % 40.4\% to 43.5 % 43.5\% on the per-provider fired subset and from 41.0 % 41.0\% to 44.8 % 44.8\% on common-fired instances, with provider-specific active components. Overall, TraceGraph provides a process vocabulary for asking what agent benchmarks test, where models diverge on a shared landscape, and how failure regions can guide downstream improvement. 1] Fudan University 2] ByteDance 3] Shanghai Innovation Institute \correspondence † † footnotetext: * Contributed equally. † † footnotetext: \dagger Corresponding author. 1 Introduction Language models first made intermediate computation visible through chain-of-thought prompting and self-consistency sampling [ 18 , 16 ] . Agent benchmarks extend this trace to interleaved thoughts, tool calls, and environment feedback, as in ReAct and Toolformer-style tool use [ 22 , 13 ] . Yet most reports still collapse each rollout to one scalar: a pass/fail label, reward score, or leaderboard average. Such scores rank systems, but they do not reveal what process a benchmark rewards. The challenge is that these process differences are not exposed as aligned units. Agent trajectories are long, branching, and benchmark-specific; their states mix tool choices, observations, file cues, environment feedback, and partial evidence. A single rollout can be inspected manually, but such inspection does not tell us whether another model reached an analogous state, avoided the same failure region, or recovered through a different route. What is missing is a shared coordinate system for comparing many trajectories as movements through the same task landscape. Figure 1 : Pass rates can hide process diversity. Left : three models score within 3 points on the same benchmark. Right : on a shared decision landscape built from pooled rollouts, they follow distinct paths: broad exploration with repair, shallow trap avoidance, or high-access navigation with more trap exposure. We introduce TraceGraph , a framework that analyzes agent behavior not by inspecting individual trajectories in isolation, but by placing many trajectories from different models onto a shared graph-based decision landscape using observable action-observation signatures. The key hypothesis is that, once trajectories are aligned through observable action–observation states, model behaviors are not arbitrary traces: they form recurring traffic patterns over task-specific regions. TraceGraph then overlays outcome-informed productive cores and trap regions, and summarizes each rollout with three events: reaching a core ( Access ), entering a trap ( Trap exposure ), and returning from a trap to a core ( Repair ). The name reflects the workflow: traces become graphs, and models are compared by how they move through the resulting landscape. These roles use terminal outcomes by design, so TraceGraph is descriptive rather than a blind success predictor: given a common map, what process does each model supply, and what process does each benchmark demand? A shared landscape is useful not only for retrospective diagnosis, but also for deciding when a live agent trajectory has entered a historically risky region. If trap regions correspond to repeatable trajectory states, an online detector built from those states should identify moments where a small recovery policy can help. We therefore instantiate a TraceGraph -guided trap-aware prefix-fork pipeline on the benchmark with the clearest positive Repair demand. An agent proceeds until a trap detector fires; matched continuations then resume from the same state while varying a conservative diagnosis note and a temperature bump. The goal is not to claim a universal recovery mechanism, but to show that graph-derived traps can guide a downstream policy that improves task resolution. Our contributions can be summarized: • TraceGraph for shared decision landscapes. We construct model-agnostic task graphs from pooled rollouts, then overlay outcome-informed core and trap regions after graph construction. • A compact process profile. Across a five-split trajectory release spanning 7,329 records, 427 tasks, and five models, Access, Trap exposure, and Repair separate model navigation styles and benchmark demands that aggregate scores collapse. • A TraceGraph -guided recovery pipeline. Because SWE-bench shows the clearest positive Repair demand, we turn graph-derived traps into runtime triggers and test two single-factor recovery policies on all 500 SWE-bench Verified instances. The best pooled policy improves resolved rate by +3.1 points ( p{=}0.016 ) on the per-provider fired subset and by +3.8 points ( p{=}0.045 ) on common-fired instances, while the active component differs by provider. 2 Related Work Outcome-centered evaluation. Frameworks such as HELM and the LM Evaluation Harness emphasize coverage, transparency, and reproducibility across many tasks and metrics [ 9 , 6 ] . Agent benchmarks bring this apparatus to interactive settings, including software repair, tool use, and general agent environments [ 7 , 11 , 23 ] . They are indispensable for tracking progress, but their headline artifacts usually summarize final success. We reuse the released logs for a complementary purpose: describing the process demands that separate high- and low-outcome trajectories. Process traces, failure patterns, and intervention. Chain-of-thought and self-consistency showed that sampled traces expose alternative reasoning paths before answers are aggregated [ 18 , 16 ] . ReAct, Toolformer, WebShop, and ALFWorld extended this process view to tool use and environment interaction [ 22 , 13 , 20 , 14 ] . Recent intervention work also treats long-trace failures as actionable: Trap-Aware Adaptive Restart identifies “thinking traps” and restarts from earlier reasoning prefixes [ 4 ] . We study an analogous but distinct setting: tool-using software agents, where trap states are observable action–observation signatures and continuations resume from a live benchmark state. Graphs as inference programs and measurement objects. Tree-of-Thoughts and Graph-of-Thoughts use graph structure during inference to guide search [ 21 , 2 ] . Other work uses graphs after generation to cluster chain-of-thought units, analyze trace topology, and model software-agent processes over exploration, localization, patching, validation, and loops [ 19 , 15 , 10 ] . A related methodological precedent is SliceGraph , a post-hoc atlas of same-answer chain-of-thought route isomerism over hidden activation slices [ 3 ] . TraceGraph adopts this graph-as-measurement perspective, but shifts the object to observable agent states, multi-model task graphs, benchmark-level process demand, and trap-triggered recovery on SWE-bench . 3 TraceGraph : Building Shared Decision Landscapes Figure 2 : Overview of TraceGraph . Stage 1 : action–observation steps from five benchmark splits are encoded into sparse