Tool-Call Dependency Structure is Linearly Decodable in LLM Agent Residual Streams
Tianda Sun, Dimitar Kazakov · 2026
Read Tool-Call Dependency Structure is Linearly Decodable in LLM Agent Residual Streams by Tianda Sun, Dimitar Kazakov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25310.txt.
Introduction
Tianda Sun and Dimitar Kazakov ask whether a tool-using model carries its call structure inside its activations. Tool calls form a directed graph, since earlier outputs feed later calls, but earlier probes used static text, not live runs. A small edge probe on a 32-billion-parameter model's residual stream recovers that graph well above random-label and position-only controls. Value corruption versus structural shuffling shows the signal tracks topology, not identifiers. Patching experiments suggest the representation spreads across layers, though the model's calls do not move.
Catalog record
| CREATOR | Tianda Sun, Dimitar Kazakov |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60237 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.25310
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Tool-Call Dependency Structure is Linearly Decodable in LLM Agent Residual Streams Tianda Sun, Dimitar Kazakov Abstract Tool-using LLM agents produce trajectories whose calls form a directed dependency graph: earlier tool outputs supply arguments to later calls. Whether this execution structure is represented inside the model is unknown; prior structural probes have targeted static code or chain-of-thought text, not an agent’s run-time call graph. A low-capacity edge probe on the residual stream of Qwen3-32B decodes the tool-call dependency graph well above both a Hewitt–Liang random-label control and a positional baseline. A counterfactual contrast between value corruption and structural perturbation indicates the signal tracks abstract topology rather than identifier values, and replicates under an independent, non-substring oracle. The non-positional component replicates on three further interactive multi-hop benchmarks and attenuates as call order alone becomes a sufficient proxy for dependency, vanishing in single-shot planning. Per-layer activation patching shifts the probe at a later, non-patched boundary, evidence that the representation propagates rather than passively reads out, though the realised tool call does not move. To our knowledge this is the first structural probe of an LLM agent’s runtime tool-call dependency graph. Our claims concern representation, not behavioural control, and span two model families and one primary domain. find_ user get_ order get_ product exchange residual stream \bar{H} (tapped at each tool-call boundary) edge probe \hat{y}_{ij}:\,i\!\to\!j ? tool-call trajectory \Rightarrow latent oracle dependency DAG Figure 1: The tool-call dependency graph is linearly decodable from an LLM agent’s residual stream. Top: tool-using LLM agents call functions sequentially, and earlier outputs supply arguments to later calls, inducing a latent dependency DAG (an edge i\!\to\!j iff call i ’s output supplies an argument of call j ); a low-capacity logistic probe reads each edge from the frozen residual stream. Bottom: the edge becomes linearly decodable by layer 14, well above a Hewitt–Liang random-label control. Activation patching at the source call shifts the probe’s prediction toward a donor trajectory’s structure at every layer \geq\!14 (§ 4.1.3 , Appendix E ); the non-positional component generalises across interactive multi-hop benchmarks (§ 4.2 ). 1 Introduction Mechanistic interpretability has yielded sharp characterisations of how large language models represent linguistic and propositional structure: structural probes recover syntactic dependency trees from contextual embeddings ( 16 ) , board-state probes reveal compact world models in game-trained networks ( 18 ; 36 ) , and dissection methods locate factual knowledge in middle-layer feed-forward modules ( 8 ; 13 ) . These results share a methodological core: train a small probe to read an abstract structure from frozen activations, and use control tasks ( 15 ) and checks for interpretability illusions ( 7 ) to verify that the probe recovers structure rather than memorising labels. Recent work has extended structural probing to numerical structure ( 17 ; 5 ) , propositional commitments ( 6 ) , spatial world models ( 19 ) , and planning in chain-of-thought traces ( 4 ; 37 ) . However, the structural-probe literature has not engaged with a setting now central to deployed LLM systems: tool-using agents that interleave model generation with external function calls ( 23 ; 34 ; 33 ) . Agent trajectories carry a non-trivial structural object — the directed acyclic graph (DAG) of tool-call dependencies, in which an edge i\to j records that the output of call i supplies an argument of call j . Whether this dependency graph is encoded in the agent’s residual stream, whether that encoding is causal, and whether it is a property of agents in general or of one benchmark, are open , despite emerging interest in agent-state interpretability for behavioural drift ( 31 ) , retrieval auditing ( 11 ) , and deception detection ( 1 ; 9 ) . We address this gap with a Hewitt–Liang-style logistic-regression edge probe on the residual stream of Qwen3-32B ( 22 ) , conditioned on a positional baseline ( 14 ) so the signal is non-trivial (Figure 1 ). We make three contributions. (1) The tool-call dependency graph is linearly decodable (§ 4.1 ): the inter-call dependency DAG of LLM-agent tool trajectories is linearly readable from frozen residual streams, above random-label, positional, random-init, surface-form, and probe-capacity controls. (2) Topology sensitivity (§ 4.1 ): a counterfactual contrast indicates the probe tracks abstract dependency topology rather than identifier values; as supporting evidence that the representation is not a passive readout, per-layer activation patching shifts a later, non-patched probe readout, though the agent’s realised tool choice does not move. (3) Cross-domain modulation (§ 4.2 ): the non-positional contribution follows a falsifiable prediction — positive on interactive multi-hop benchmarks ( 35 ; 39 ) , vanishing where call order alone suffices ( 24 ) or transitive structure is absent ( 21 ) — and it reproduces on independently provided ground-truth DAGs ( 24 ) . § 4.1 develops the first two contributions; § 4.2 develops the third. 2 Related Work Structural probes. The original Hewitt–Manning probe ( 16 ) read syntactic trees from BERT; subsequent work has read board states ( 18 ; 36 ) , propositional commitments ( 6 ; 13 ) , numeric structure ( 17 ) , and spatial coordinates ( 19 ) from frozen activations. The methodological core is shared: a low-capacity probe reads off an abstract structure, validated with control tasks ( 15 ) , conditional probing ( 14 ) , and checks for interpretability illusions ( 7 ) . We extend this line by treating an agent’s tool-call DAG — and its transitive closure — as the probe target. The closest precedents probe graph structure — reasoning-DAG adjacency from hidden states ( 40 ) , static code data-flow edges ( 28 ; 10 ) , and semantic-relation graphs decoded from text ( 3 ) — or scalar CoT plans ( 4 ; 37 ; 5 ) ; we differ in that our edges are side-effecting tool calls with an independently constructed ground-truth DAG (not tokens, static code, or text-derived semantic relations), and we add activation patching and a topology-vs-value dissociation these probe-only studies lack. Agent interpretability. Recent work has examined retrieval-head identification ( 30 ) , tool-output tracking ( 1 ; 11 ) , deception detection ( 9 ) , behavioural drift ( 31 ) , hallucination editing ( 26 ) , and knowledge conflicts ( 38 ) . Closest to our setting, 27 probe agent residual streams for single-step tool-need and tool-risk (with feature ablation for a causal effect on tool selection), 25 decode a per-step tool-necessity signal from hidden states, and 32 ; 29 read and steer the chosen tool’s identity or intent from mid-layer activations. These decode per-step quantities (whether to call a tool, or which one); we instead decode the relational structure linking calls — the inter-call dependency edge (i,j) — with per-layer activation patching of the dependency representation plus a topology-vs-value dissociation. The two approaches target complementary objectives: per-step decisions versus multi-step structure. To our knowledge this is the first structural probe of an LLM agent’s tool-call execution graph at the inter-call dependency level. Tool-use benchmarks and the interactive \times multi-hop axis. Agent tool-use benchmarks vary along two axes that turn out to matter for our analysis: whether tool calls are issued interactively with execution feedback or emitted