Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use
Yize Cheng, Chenrui Fan, Mahdi JafariRaviz, Keivan Rezaei, Soheil Feizi · 2026
Read Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use by Yize Cheng, Chenrui Fan, Mahdi JafariRaviz, Keivan Rezaei, Soheil Feizi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14038.txt.
Introduction
Cheng and four co-authors redefine tool necessity per model, exposing a knowing-doing gap in language-model tool use. Past work treats the need for tools as fixed across models, judged by people or models, and mostly on obvious cases like fetching weather versus rewording text. In truth, a task a strong model solves alone may still need tools for a weaker one. Grounding necessity in each model's own measured performance, the paper compares need against observed tool calls across four models on arithmetic and factual questions.
Catalog record
| CREATOR | Yize Cheng, Chenrui Fan, Mahdi JafariRaviz, Keivan Rezaei, Soheil Feizi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45487 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.14038
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use Yize Cheng, Chenrui Fan, Mahdi JafariRaviz, Keivan Rezaei, Soheil Feizi Abstract Large language models (LLMs) increasingly act as autonomous agents that must decide when to answer directly vs. when to invoke external tools. Prior work studying adaptive tool use has largely treated tool necessity as a model-agnostic property, annotated by human or LLM judge, and mostly cover cases where the answer is obvious (e.g., fetching the weather vs. paraphrasing text). However, tool necessity in the wild is more nuanced due to the divergence of capability boundaries across models: a problem solvable by a strong model on its own may still require tools for a weaker one. In this work, we introduce a model-adaptive definition of tool-necessity, grounded in each model’s empirical performance. Following this definition, we compare the necessity against observed tool-call behavior across four models on arithmetic and factual QA dataset, and find substantial mismatches of 26.5–54.0% and 30.8–41.8%, respectively. To diagnose the failure, we decompose tool use into two stages: an internal cognition stage that reflects whether a model believes a tool is necessary, and an execution stage that determines whether the model actually makes a tool-call action. By probing the LLM hidden states, we find that both signals are often linearly decodable, yet their probe directions become nearly orthogonal in the late-layer, last-token regime that drives the next-token action. By tracing the trajectory of samples in the two-stage process, we further discover that the majority of mismatch is concentrated in the cognition-to-action transition, not in cognition itself. These results reveal a knowing–doing gap in LLM tool-use: improving tool-use reliability requires not only better recognition of when tools are needed, but also better translation of that recognition into action. 1 Introduction Large language models (LLMs) are increasingly deployed as autonomous agents that interact with external tools such as search engines, calculators, and APIs [ 20 , 24 , 26 , 19 ] . A central challenge in building reliable autonomous LLM agents is achieving adaptive tool using: the LLM needs to determine when it should rely on such tools versus answering directly [ 8 , 22 , 27 ] . Prior work studying adaptive tool use [ 8 , 22 , 13 ] has largely treated tool necessity as a static, model-agnostic property, typically relying on human annotators or strong LLM judges to determine whether a query requires a tool, focusing primarily on polarized cases where the answer is obvious, such as fetching real-time weather data versus paraphrasing a static paragraph. However, tool necessity in the wild is fundamentally more nuanced due to the natural divergence of capability boundaries across different models. A problem that is easily solvable by a state-of-the-art model relying solely on its internal weights may completely exceed the capabilities of a smaller or less capable model, thereby making tool use strictly necessary for the latter but redundant for the former. In this work, we argue that tool necessity must be intrinsically tied to the specific capabilities of the model in question. We introduce a model-adaptive definition of tool necessity, grounded not in static annotations, but in each individual model’s empirical performance. By evaluating necessity relative to a model’s intrinsic capabilities, we establish a more accurate characterization for when a specific LLM should seek external help. Following this definition, we compare the actual necessity against the observed tool-call behavior across four distinct models on arithmetic and factual question-answering (QA) datasets. Our findings reveal substantial mismatches: models exhibit a 26.5–54.0% necessity-action mismatch in arithmetic tasks and a 30.8–41.8% necessity-action mismatch in factual QA, frequently calling tools when capable of answering directly, or attempting to answer directly when lacking the requisite internal knowledge. Figure 1: Overview of the two stage cognition-execution modeling of LLM tool-use. (Left) Necessity : We introduce a model-adaptive definition of tool necessity based on a model’s empirical ability to consistently answer a query correctly on its own, contrasting with prior model-agnostic approaches. (Middle) Cognition : By probing the model’s internal hidden states h , we identify a linear cognition direction w_{c} that successfully distinguishes when a tool is necessary. (Right) Action : We also train a probe w_{a} to predict the actual tool-call execution. We find that w_{c} and w_{a} become nearly orthogonal in late layers, and that the majority of the necessity-action mismatch stems from the execution stage (translating awareness into action) rather than the internal cognition stage. To diagnose the underlying mechanisms of this failure, we propose a two-stage decomposition of the tool-use process: an internal cognition stage, which reflects whether the model’s internal representations encode the belief that a tool is necessary, and an execution stage, which determines whether the model actually outputs the tool-triggering tokens. Building on prior advancements in mechanistic interpretability and representation engineering [ 35 ] and following recent literature on adaptive tool-using [ 13 , 28 ] , we probe the LLM hidden states and find that both the cognition of necessity and the execution intent are often linearly decodable. Yet, intriguingly, their respective probe directions become nearly orthogonal in the late-layer, last-token regime. By tracing the trajectory of samples through this two-stage process, we uncover a knowing-doing gap in LLM tool use: the majority of the observed necessity-action mismatch cases originates from the transition from cognition to action, rather than in the cognition stage. Models frequently generate internal representations indicating the awareness of their own limitations, but fail to translate this into the syntactic execution of a tool call. Our main contributions can be summarized as follows: • We introduce a model-adaptive definition of tool necessity grounded in empirical performance, challenging the traditional reliance on static, model-agnostic annotations. • We evaluate four distinct LLMs across arithmetic and factual QA datasets, revealing substantial behavioral mismatches (up to 54.0%) between actual tool necessity and observed tool-call actions. • By dividing tool use into an internal cognition stage and an execution stage, we use representation probing to demonstrate that while both intent and necessity are linearly decodable, their probe directions become near orthogonal in the late-layer, last-token regime. • Through trajectory tracing, we discover that tool-use failures predominantly occur during the transition from cognition to action, highlighting a knowing-doing gap in LLM tool-use. 2 Related work Tool calling in LLM agents. To extend LLM capabilities beyond parametric knowledge, researchers have introduced function/tool calling [ 20 , 24 , 26 , 19 ] , enabling interaction with external resources and expanding task coverage. Standardized protocols like MCP [ 1 ] and A2A [ 6 ] further streamline communication and access within tool ecosystems. In parallel, various works has examined tool-use accuracy [ 12 , 21 ] , hallucinated calls [ 33 , 23 ] , and robustness to tool descriptions [ 25 , 5 ] . However, while these efforts aim at teaching and evaluating how to use tools, an important and often understudied challenge in building reliable LLM agents is determining when to use tools. Existing works that do study this challenge [ 8 , 22 , 13 ] treat tool necessity as a static property of the query, labeling instances as either tool-necessary or tool-unnecessary using human annotators or some proprietary LLM. This ignores the inherent