To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents
Wei Shi, Ziheng Peng, Sihang Li, Xiting Wang, Xiang Wang, Mengnan Du, Na Zou · 2026
Read To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents by Wei Shi, Ziheng Peng, Sihang Li, Xiting Wang, Xiang Wang, Mengnan Du, Na Zou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18882.txt.
Introduction
Wei Shi and six colleagues diagnose why language-model agents keep calling tools even when no tool is needed. Six models on the When2Call benchmark get calls right but hold back poorly, leaving overall accuracy between fifty-five and seventy percent. The authors trace this to a built-in call offset in the decision mapping, recovered with sparse autoencoders. A closed-form steering correction that cancels the offset reduces over-calling and lifts overall accuracy with almost no loss in correct calls.
Catalog record
| CREATOR | Wei Shi, Ziheng Peng, Sihang Li, Xiting Wang, Xiang Wang, Mengnan Du, Na Zou |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50605 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.18882
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents Wei Shi, Ziheng Peng, Sihang Li, Xiting Wang, Xiang Wang, Mengnan Du, Na Zou Abstract LLM agents exhibit a consistent tendency to over-call, invoking tools even in situations where none is needed. On the When2Call benchmark, six models from three families show high call accuracy but much lower no-call accuracy, leaving overall accuracy in the 55%–70% range. We trace this to an Intrinsic Bias Hypothesis ( IBH ): the call/no-call decision mapping carries an activation-independent call offset, so the model favors call even at activation parity. Using Sparse Autoencoders (SAEs), we recover behavior-aligned feature bases for the call / no_call decision, reduce them to a signed activation margin, and estimate the offset directly. Across all six models, the model is decision-neutral only when no_call activation outweighs call activation, consistent with IBH. We then causally test IBH with Adaptive Margin-Calibrated Steering ( AMCS ), a closed-form counter-bias shift along SAE decoder directions. Cancelling the diagnosed offset mitigates over-calling and improves overall accuracy with a negligible drop in call accuracy. Our work recasts over-calling from an empirical phenomenon into a mechanistic object amenable to causal correction. The code is available at https://github.com/SKURA502/agent-sae/ . 1 Introduction Large language models (LLMs) have rapidly evolved from text generators into the reasoning backbone of autonomous agents [ 11 , 2 , 6 , 20 ] . At the core of this transition lies tool use, the ability to interface with external systems such as search engines, code interpreters, APIs, and databases, extending LLMs beyond pure language modeling [ 17 , 14 , 13 , 23 ] . Beyond executing calls correctly, effective tool use hinges on knowing when to invoke a tool and, crucially, when not to. As shown in Figure 1 (a), across six models from three families (Qwen3.5 [ 20 ] , Gemma-3 [ 19 ] , Ministral-3 [ 1 ] ) evaluated on the When2Call benchmark [ 16 ] , call accuracy remains high, while no-call accuracy is consistently much lower, leaving overall accuracy in the 55%–70% range. Models therefore know how to call tools when calls are required, but often issue calls when none is warranted. Figure 1 (a) illustrates this failure mode: given an underspecified Spotify request, the base model invokes the tool instead of asking for the missing song name and device ID. This bias degrades user experience and inflates API costs in deployed systems. Figure 1: Overview of over-calling, intrinsic bias, and AMCS. (a) Across six target models, call accuracy is high, but no-call accuracy is much lower, reducing overall accuracy ; a representative case shows the model calling a tool despite missing required information. (b) Intrinsic Bias Hypothesis: at activation parity ( m=0 ), the decision still favors call ; the neutral boundary shifts to m^{\star}<0 . (c) AMCS converts the diagnosed bias \beta_{0} into a counter-bias shift \delta , aiming to reduce false calls while preserving valid calls. Having documented the bias, we ask what governs the call/no-call decision. An activation-only account offers a natural mechanism: the decision is determined by the relative activation of call- and no-call-related directions, with the stronger side prevailing. This account makes a testable prediction. If the decision depends only on the activation difference, balancing call and no-call activation should leave the model decision-neutral. Our analysis shows the opposite. Even at activation parity, the model remains biased toward call ; it becomes neutral only when no-call activation is stronger. A residual bias at parity cannot come from activation levels themselves and must instead enter the decision through a separate, additive term. We formalize this as the Intrinsic Bias Hypothesis ( IBH ): over-calling reflects an activation-independent call offset in the decision mapping. We test IBH through a pipeline built on Sparse Autoencoders (SAEs) [ 7 , 5 ] , which decompose residual-stream activations into sparse and interpretable features [ 7 , 21 ] and let us isolate the components driving the call / no_call decision. We first recover behavior-aligned call and no_call feature bases and verify that a handful of them predict the decision near the residual-stream upper bound (§ 3 ). We then reduce these bases to a signed activation margin and test whether activation parity removes the decision asymmetry (§ 4 ). It does not: activation geometry and logistic estimates of the offset both show a shifted neutral boundary (Figure 1 (b)), and the same SAE directions also separate true from false calls, exposing a lever for intervention. Finally, we causally test IBH with Adaptive Margin-Calibrated Steering ( AMCS ), a closed-form counter-bias shift along SAE decoder directions (§ 5 ; Figure 1 (c)). Cancelling the diagnosed offset raises no-call accuracy by 4–17 points on five of six models and overall accuracy by up to 5 points, with minimal impact on call accuracy across the same models. Our main contributions are: • We open a mechanistic view of tool-use gating by recovering behavior-aligned SAE feature bases that predict the call / no_call decision near the residual-stream upper bound (§ 3 ). • We formulate and validate IBH , showing that over-calling reflects an activation-independent call offset rather than activation levels alone, and that the same SAE directions further separate true from false calls (§ 4 ). • We introduce AMCS , a closed-form steering method derived from the diagnosed offset, and use it as a causal validation of IBH that mitigates over-calling across six models from three families with minimal impact on valid tool calls (§ 5 ). 2 Related Work 2.1 Tool-Use Evaluation and the Over-Calling Phenomenon LLM agents have evolved from early API and calculator integrations [ 17 , 8 , 22 ] to systems spanning large collections of real-world tools [ 14 ] . A corresponding evaluation ecosystem studies whether models can invoke tools correctly: selecting the right function, producing valid arguments, and completing tool-mediated tasks [ 13 , 9 , 12 ] . These benchmarks measure how well models call tools, but the gating question of when not to call is less central. When2Call [ 16 ] targets this decision directly and reveals a consistent asymmetry: models perform much better on call-required queries than on queries where not calling a tool is correct. Rather than another benchmark or data-side remedy, we ask what internal mechanism produces this asymmetry and diagnose over-calling as a measurable bias in the model’s decision mapping. 2.2 Mechanistic Interpretability and Sparse Autoencoders Understanding why over-calling arises requires tools that can dissect a model’s internal computations. Mechanistic interpretability [ 18 ] provides this foundation, aiming to decompose model behavior into interpretable internal components and identify the structures responsible for specific outputs. Within this framework, Sparse Autoencoders (SAEs) have become a widely adopted method for recovering sparse, interpretable units from residual streams [ 7 , 3 , 21 ] , with variants such as TopK [ 5 ] and JumpReLU [ 15 ] improving reconstruction fidelity and feature quality. We use SAEs to extract a structured feature basis for testing a mechanistic hypothesis. The same basis supports both estimating an activation-independent call offset and applying a counter-bias intervention along these directions as a causal test of the diagnosed offset. 3 Discovering