Tool Attention Is All You Need: Dynamic Tool Gating and Lazy Schema Loading for Eliminating the MCP/Tools Tax in Scalable Agentic Workflows

Anuj Sadani, Deepak Kumar · 2026

Read Tool Attention Is All You Need: Dynamic Tool Gating and Lazy Schema Loading for Eliminating the MCP/Tools Tax in Scalable Agentic Workflows by Anuj Sadani, Deepak Kumar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21816.txt.

Introduction

Sadani and Kumar target the hidden token tax of the Model Context Protocol, where eager schema injection can add roughly 10,000 to 60,000 tokens per turn in multi-server agent deployments. Their Tool Attention middleware scores intent-schema overlap, gates tools by preconditions and access scopes, and loads full schemas lazily only for top-ranked tools. In a simulated 120-tool, six-server benchmark, per-turn tool tokens fall 95.0 percent, from 47,300 to 2,400, while effective context use rises from 24 percent to 91 percent.

Catalog record

CREATORAnuj Sadani, Deepak Kumar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48972
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.21816

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tool Attention Is All You Need: Dynamic Tool Gating and Lazy Schema Loading for Eliminating the MCP/Tools Tax in Scalable Agentic Workflows Anuj Sadani, Deepak Kumar Abstract The Model Context Protocol (MCP) has become a common interface for connecting large language model (LLM) agents to external tools, but its reliance on stateless, eager schema injection imposes a hidden per-turn overhead—the MCP Tax or Tools Tax —that practitioner reports place between roughly 10k and 60k tokens in typical multi-server deployments. This payload inflates the key-value cache, is associated with reasoning degradation as context utilization approaches published fracture points around 70 % 70\% , and turns token budgets into a recurring operational cost. We introduce Tool Attention , a middleware-layer mechanism that generalizes the “Attention Is All You Need” paradigm from self-attention over tokens to gated attention over tools . Tool Attention combines (i) an Intent–Schema Overlap (ISO) score from sentence embeddings, (ii) a state-aware gating function enforcing preconditions and access scopes, and (iii) a two-phase lazy schema loader that keeps a compact summary pool in context and promotes full JSON schemas only for top- k k gated tools. We evaluate on a simulated 120-tool, six-server benchmark whose per-server token counts are calibrated to public audits of real MCP deployments. In this simulation, Tool Attention directly reduces measured per-turn tool tokens by 95.0 % 95.0\% ( 47.3 ​ k → 2.4 ​ k 47.3\text{k}\!\to\!2.4\text{k} ) and raises effective context utilization (a token-ratio quantity) from 24 % 24\% to 91 % 91\% . End-to-end figures for task success, latency, cost, and reasoning quality are reported as projections derived from the measured token counts combined with published deployment telemetry; they are not measured on live LLM agents, and we mark projected values explicitly throughout. Taken together, the results support a simple thesis: protocol-level efficiency, not raw context length, is a binding constraint on scalable agentic systems. The code for this work is accessible at https://github.com/asadani/tool-attention . Keywords: Model Context Protocol, tool use, agentic LLMs, context engineering, lazy loading, intent routing, retrieval-augmented tools, middleware orchestration. 1 Introduction The past two years have seen LLM-based agents transition from isolated chat interfaces to autonomous workflow participants that read code, query databases, post to communication platforms, and orchestrate multi-step plans across hundreds of tools [ 1 , 13 , 2 ] . The operational backbone of this transition is the Model Context Protocol (MCP), an open specification introduced by Anthropic in November 2024 and now adopted by OpenAI, Google, and Microsoft [ 1 , 21 ] . MCP abstracts bespoke N\times M integrations into an N+M composable surface: every agent client can discover and call any tool exposed by a compliant server via a standardized JSON-RPC 2.0 handshake [ 21 , 10 ] . Yet the very design that grants MCP its interoperability—stateless transmission of full tool schemas on every conversational turn—has opened an equally systemic wound. Because the underlying chat-completions APIs are stateless, host clients (Claude Desktop, Cursor, VS Code, Claude Code) must re-serialize the entire tool catalog on every single request [ 22 ] . Empirical audits consistently place this overhead between 15,000 and 55,000 tokens per turn in typical four-to-six-server deployments, reaching > 150k with aggressive tool sprawl [ 22 , 14 , 18 ] . We call this recurring overhead the Tools Tax , following community usage [ 22 , 14 ] . The Tools Tax is not simply a cost-of-goods problem. It precipitates three cascading failures. First, economic : stateless re-injection inflates per-session spend by an order of magnitude; one published benchmark reports CLI-equivalent workflows at $3.20 versus MCP at $55.20 for the same 10,000 operations [ 14 , 26 ] . Second, cognitive : once context utilization crosses approximately 70\% , LLM reasoning quality collapses—models begin hallucinating parameters, confusing similar tools, and losing episodic thread-of-task memory [ 22 , 19 , 23 ] . Third, adversarial : the same schema text that describes a tool also shapes the model’s attention mask, so malicious Tool Poisoning Attacks can hijack control flow by injecting adversarial instructions into a seemingly benign tool description [ 29 , 30 ] . Prior mitigations—static pruning, manual server scoping, CLI-style lazy discovery, and code-execution sandboxes—each address a slice of the problem but either sacrifice flexibility, require engineering-heavy refactors, or break the uniform MCP developer experience [ 13 , 6 ] . What is needed is a drop-in middleware layer that preserves protocol semantics while eliminating the tax at its source. We propose Tool Attention : a middleware-resident attention mechanism over the tool catalog itself. Just as scaled dot-product attention replaced recurrence in sequence modeling by letting every token attend dynamically to every other [ 28 ] , Tool Attention replaces eager, uniform schema injection with dynamic, query-conditioned tool selection. Formally, it decomposes into (i) a query-to-tool Intent–Schema Overlap score computed with commodity sentence embeddings, (ii) a stateful gating function enforcing preconditions and scopes, and (iii) a lazy two-phase loader that injects full JSON schemas only for tools in the gated top- k . Contributions. This paper makes four contributions: 1. Formal quantification. We give a closed-form expression for the Tools Tax and derive the conditions under which it dominates the effective context window, corroborated against published per-server token counts (§ 3 ). 2. Mechanism. We present Tool Attention: a novel, model-agnostic meta-layer combining ISO scoring, stateful gating, and two-phase lazy loading, grounded theoretically in the Total Attention Energy formulation from the MCP security literature (§ 4 ). 3. Reference implementation. We release a production-grade Python implementation built on LangGraph middleware, FAISS, sentence-transformers, and tiktoken , with a reproducible benchmark harness (§ 5 , Appendix A ). 4. Evaluation on a calibrated simulation. On a 120-tool, six-server synthetic benchmark whose per-server token counts are calibrated to public deployment audits, Tool Attention achieves a measured 95.0\% reduction in per-turn tool tokens and a 3.8\times improvement in effective context utilization. We additionally report projected task-success, latency, and cost gains derived from these measured quantities plus published telemetry; we do not claim measurements from live agent runs (§§ 6 – 7 ). The remainder of the paper is organized as follows. § 2 surveys related work. § 3 formalizes the Tools Tax problem. § 4 introduces the Tool Attention mechanism. § 5 details the reference implementation. § 6 describes the experimental protocol and § 7 reports results. § 8 discusses limitations, and § 9 concludes. 2 Related Work A note on source types. This is an early-stage topic and a portion of the empirical grounding for the Tools Tax draws on practitioner reports—engineering blog posts, vendor documentation, and public community discussion—in addition to peer-reviewed work. Where we cite such sources [ 22 , 14 , 18 , 26 ] we do so specifically for the per-server token counts and deployment telemetry that they are best positioned to report. Claims that depend on these sources are framed as practitioner-reported deployment measurements rather than as peer-reviewed results; we treat formal contributions (mechanism, math, released implementation) as the primary scholarly content of this paper. Model Context Protocol and its discontents. The MCP specification standardizes the exchange of tools, resources, and prompts between LLM hosts and external servers via JSON-RPC 2.0 [ 21 ] . While the

Read the complete text