ScaleMCP: Dynamic and Auto-Synchronizing Model Context Protocol Tools for LLM Agents

Elias Lumer, Anmol Gulati, Vamse Kumar Subbiah · 2025

Read ScaleMCP: Dynamic and Auto-Synchronizing Model Context Protocol Tools for LLM Agents by Elias Lumer, Anmol Gulati, Vamse Kumar Subbiah (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06416.txt.

Introduction

Elias Lumer, Anmol Gulati, Vamse Subbiah, and co-authors present a system in 2025 for giving AI agents access to external tools. A new protocol lets agents call tools and data services, but current tool lists are updated by hand, which causes duplication and errors. Selection also happens before the agent starts, limiting its freedom in long conversations.

Their system, called ScaleMCP, lets the agent fetch tools itself during a task and keeps the tool store in sync with source servers automatically. A new scoring method weights the most telling parts of each tool description. Tests used five thousand finance tool servers across many models. The system improved both tool finding and correct tool use.

Catalog record

CREATORElias Lumer, Anmol Gulati, Vamse Kumar Subbiah
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-06416

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ScaleMCP: Dynamic and Auto-Synchronizing Model Context Protocol Tools for LLM Agents Elias Lumer, Anmol Gulati, Vamse Kumar Subbiah Abstract Recent advancements in Large Language Models (LLMs) and the introduction of the Model Context Protocol (MCP) have significantly expanded LLM agents’ capability to interact dynamically with external tools and APIs. However, existing tool selection frameworks do not integrate MCP servers, instead relying heavily on error-prone manual updates to monolithic local tool repositories, leading to duplication, inconsistencies, and inefficiencies. Additionally, current approaches abstract tool selection before the LLM agent is invoked, limiting its autonomy and hindering dynamic re-querying capabilities during multi-turn interactions. To address these issues, we introduce ScaleMCP, a novel tool selection approach that dynamically equips LLM agents with a MCP tool retriever, giving agents the autonomy to add tools into their memory, as well as an auto-synchronizing tool storage system pipeline through CRUD (create, read, update, delete) operations with MCP servers as the single source of truth. We also propose a novel embedding strategy, Tool Document Weighted Average (TDWA), designed to selectively emphasize critical components of tool documents (e.g. tool name or synthetic questions) during the embedding process. Comprehensive evaluations conducted on a created dataset of 5,000 financial metric MCP servers, across 10 LLM models, 5 embedding models, and 5 retriever types, demonstrate substantial improvements in tool retrieval and agent invocation performance, emphasizing ScaleMCP’s effectiveness in scalable, dynamic tool selection and invocation. Keywords: Tool Selection, Retrieval-Augmented Generation, Model Context Protocol, LLMs, AI Agents 1 Introduction Recent advancements in Large Language Models (LLMs) and tool learning have enabled LLM agents to dynamically interact with external tools and APIs. The introduction of the Model Context Protocol (MCP) standardizes this connection between LLMs and external tools, data sources, and prompts ( Anthropic, 2024 ) . Concurrently, to deal with LLM architecture limitations in calling the correct tools or model providers not allowing more than 128 tools to be equipped to the LLM, breakthroughs in tool-applied Retrieval-Augmented Generation (RAG) have enabled LLM agents to efficiently scale to a large number of tools ( Lumer et al., 2025a ; Chen et al., 2024 ) . Figure 1: ScaleMCP automatic indexing pipeline and LLM agent invocation. The auto-synchronization tool indexing pipeline reads the current single source of truth MCP server tools and compares its hashes to the MCP storage system hashes, with CRUD (create, read, update, delete) operations on the storage index. For the LLM agent invocation, after the user asks a question, the LLM agent calls the "MCP Retrieval Tool" in parallel 5 times (1 for each targeted tool to retrieve) to equip relevant MCP servers (tools) into its context. Then, after retrieving the relevant MCPs, the LLM agent decides to call 5 MCP servers in parallel, and the MCP server returns the tool response. Finally, the LLM agent returns a successful final answer to the user after reasoning through the MCP server responses. Despite technological advancements in tool selection and LLM invocation in prior work, three critical limitations remain. First, existing frameworks have not adopted the Model Context Protocol within their tool selection frameworks. Second, prior work heavily relies on manual updates to a monolithic tool repository to maintain synchronization between tool definitions and tool storage systems used for retrieval. This manual updating process is prone to human-error, inconsistencies, and relies on duplicated tool code. Lastly, current approaches abstract the tool selection process outside of the LLM invocation process, limiting the agents’ autonomy and preventing dynamic re-querying of the tool storage system during multi-turn user conversations. In this paper, we address these gaps by introducing ScaleMCP, a novel tool selection approach enabling LLM agents to dynamically discover and equip MCPs (as tools) during multi-turn interactions. Central to our framework is an auto-synchronization tool storage system pipeline (Figure 1 ) that treats MCP servers as the single source of truth, automatically detecting and reflecting updates in the storage system by using CRUD operations (create, read, update, delete). Additionally, we propose a new tool document embedding process, Tool Document Weighted Average (TDWA), which selectively emphasizes certain tool document components during vector embedding, overcoming the current concatenation or simple averaging method that treats all components equally in the vector space. Lastly, we validate our contributions with a newly curated dataset of 5,000 real-world financial metric MCP servers, with extensive experiments on 10 LLM models, 5 embedding models, and 5 retriever types for (1) tool retrieval, (2) comparison of TDWA, and (3) end-to-end LLM agent invocation. 2 Background 2.1 Model Context Protocol (MCP) The Model Context Protocol (MCP) is an open protocol developed to standardize the integration between Large Language Models (LLMs) and external tools, data sources, and prompts ( Anthropic, 2024 ) . Anthropic introduced MCP to provide a universal method to replace fragmented integrations with a unified protocol for AI agents. This architecture allows developers to expose their tools, APIs, data, or prompts through MCP servers or build AI applications (MCP clients) that connect to these servers, simplifying the process of granting AI systems access to necessary data. Recently, researchers have highlighted security and privacy considerations associated with the Model Context Protocol, such as malicious code execution, remote access control, credential theft, lack of authentication, authorization, and debugging ( Radosevich and Halloran, 2025 ; Hou et al., 2025 ) . Additionally, developers have noted a limitation with serverless deployments as MCP currently is a stateful protocol between a client and server, with benefits including push notifications and sampling ( Volo Builds, 2025 ) . Nonetheless, the Model Context Protocol is asserting itself as the current standard for LLM agent tool integration, with model providers (e.g. OpenAI and Google) and AI platforms (e.g. Cursor, Cline) adopting the protocol ( OpenAI, 2025 ; Google, 2025 ; Cursor, 2025 ; Cline, 2025 ) . 2.2 Tool Selection and Retrieval Large language models natively face limitations in the number of tools or functions they can directly access and invoke. On one hand, complex multi-hop tool usage constrains the reasoning capability of the LLM in deciding which tools to invoke and in what sequence. On the other hand, model providers such as OpenAI, Anthropic, and Google enforce strict API limits, preventing the integration of more than 128 tools at a time ( OpenAI, 2024 ) . To scale beyond this constraint, prior works ( Lumer et al., 2025a ; Chen et al., 2024 ) employ advanced RAG-based methods without fine-tuning, storing tools offline in vector databases or knowledge graphs ( Lumer et al., 2025b ; Peng et al., 2024 ) and dynamically equipping only relevant tools during inference. Alternatively, an agentic RAG approach equips LLMs with dedicated tool-searching functionalities, allowing dynamic self-directed tool discovery and invocation ( Singh et al., 2025 ; Li et al., 2023 ; Du et al., 2024 ) , contrasting with static, predefined retrieval pipelines ( Lumer et al., 2024 ; Chen et al., 2024 ) . However, Li et al. notes a limitation where earlier GPT-based models fail to utilize these dynamic tool-search functions effectively. Additional research emphasizes retriever fine-tuning over out-of-the-box embeddings provided by OpenAI ( OpenAI, 2025 ) or Google ( Google, 2025 ) for tool selection efficiency ( Wu et al., 2024 ; Qin et al., 2023 ; Anantha et al., 2023 ;

Read the complete text