Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces
Karan Gupta, Pranav Vajreshwari, Yash Pandya, Raghav Magazine, Akshay Nambi, Ahmed Awadallah · 2026
Read Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces by Karan Gupta, Pranav Vajreshwari, Yash Pandya, Raghav Magazine, Akshay Nambi, Ahmed Awadallah (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06713.txt.
Introduction
Karan Gupta and colleagues present ATLAS, a reinforcement fine-tuning frame that lets small language models work with very large tool sets. Instead of loading all tools at once, the model learns when to load tools step by step. It also learns to chain them in code, which bounds context growth and steadies long tasks. Success is split into rubric criteria scored by small judge models, and a 4B model nears frontier-agent scores.
Catalog record
| CREATOR | Karan Gupta, Pranav Vajreshwari, Yash Pandya, Raghav Magazine, Akshay Nambi, Ahmed Awadallah |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 97843 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.06713
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces Karan Gupta, Pranav Vajreshwari, Yash Pandya, Raghav Magazine, Akshay Nambi, Ahmed Awadallah Abstract Agentic systems operating over large tool ecosystems must plan and execute long-horizon workflows under weak or non-verifiable supervision. While frontier models mitigate these challenges through scale and large context budgets, small language models (SLMs) remain brittle: eager tool loading saturates context, execution errors compound over time, and sparse rewards limit learning. We introduce ATLAS , a reinforcement finetuning framework that enables SLMs to operate effectively in large-scale toolspace environments by learning how to acquire context and how to execute actions. Our approach makes two key contributions. First, we treat context control and execution structure as learnable decisions, combining iterative tool loading with programmatic tool orchestration to bound context growth and stabilize long-horizon trajectories. Second, we propose rubric-based reinforcement finetuning, which decomposes task success into structured, task-aligned criteria and enables scalable training using small judge models. Across MCP benchmarks, these design choices yield large and consistent gains over generic RL baselines, allowing a 4B SLM to approach frontier-agent performance under far tighter parameter and context budgets. Figure 1: Context growth and execution structure across MCP agent designs. Traditional MCP agents incur high context costs by loading all tools upfront. ISL and ITL progressively reduce context by scoping server and tool schemas, while ITL+PTC further minimizes prompt growth by moving execution state into programmatic orchestration. 1 Introduction Agentic systems are increasingly deployed in Model Context Protocol (MCP) environments, where a single user request may require coordinated planning, reasoning, and execution across multiple external services, tools, and enterprise workflows 1 . While MCP’s client–server abstraction enables large-scale integration, it shifts the core challenge from connectivity to decision-making at scale : selecting relevant servers and tools, maintaining state over long trajectories, and operating within tight context and cost budgets 31 ; 20 . In practice, agents are often connected to hundreds of tools across many MCP servers, and exposing the full tool registry upfront forces reasoning over large, heterogeneous schemas, causing tool definitions and outputs to dominate the context window and degrade efficiency as tool ecosystems scale 13 ; 4 ; 25 . Frontier agent architectures mitigate tool-scale and long-horizon execution through dynamic tool discovery and code-based orchestration 28 . Rather than preloading all tool schemas, agents retrieve only the servers and tools required at each step, reducing token overhead from verbose definitions and intermediate results. In parallel, Programmatic Tool Calling (PTC) replaces JSON-style loops with executable orchestration code, enabling explicit control flow while keeping large artifacts out of the prompt 14 ; 19 . While effective for frontier models, these mechanisms rely on assumptions about context capacity, state robustness, and code reliability that do not hold for small language models (SLMs) 16 ; 11 . SLMs are essential for latency-, cost-, and locality-constrained deployments, but naively adopting frontier agent designs exposes structural failure modes 15 . The limitation is not reduced reasoning ability, but a mismatch between the scale assumptions in current architectures and the operating constraints of efficient models. Challenges for SLMs in live MCP environments. Agentic systems operating over MCP face a combination of tool-space explosion , long-horizon execution , and weak supervision that disproportionately impacts SLMs 13 . Agents must reason over large, heterogeneous tool sets distributed across many servers, where dynamic discovery becomes brittle under tight context budgets and costly tool misselection 4 . Long-horizon workflows amplify early errors when limited context restricts stable tracking of goals and intermediate state 20 , while code-based orchestration exposes weaknesses in code synthesis, execution, and recovery 3 . Moreover, MCP tasks rarely admit a single verifiable outcome, making outcome-only rewards too sparse for effective credit assignment and leading to brittle behaviors such as premature termination or overuse of familiar tools 5 ; 23 . Together, these challenges indicate that the primary limitation of SLMs is the absence of mechanisms that explicitly regulate context growth and execution structure . This motivates a reframing of agentic reasoning as deciding what context to acquire, when to acquire it, and how to represent execution compactly , rather than expanding the prompt 31 . We introduce ATLAS ( Adaptive Tool Loading and Scoped Context ), a reinforcement finetuning (RFT) framework for enabling efficient agentic reasoning with SLMs in large-scale MCP environments. ATLAS is complementary to frontier agent architectures in that it adopts the same high-level principles of on-demand tool access and structured execution, while explicitly targeting the efficiency-constrained regime where context, computation, and supervision are scarce. Its core novelty lies in treating context acquisition and execution structure as learnable decisions , optimized through reinforcement learning rather than fixed architectural choices. ATLAS is built around two abstractions. First, adaptive tool loading exposes a compact capability overview and incrementally materializes detailed tool schemas only when required, bounding the tool search space at each step while supporting multi-server workflows. Second, orchestration through code represents long-horizon tool use as executable programs rather than turn-by-turn natural language interactions, enabling compact state representation and reliable control flow without repeatedly surfacing intermediate results to the model. While these abstractions define how efficient agentic behavior can be represented, learning such behavior remains challenging due to non-verifiable objectives and delayed supervision. Existing RFT approaches for agents typically rely on frontier LLM judges producing synthetic trajectory-level rewards for dimensions such as task success or tool correctness 5 ; 34 ; 10 ; 22 . Although effective at small scale, these rewards are coarse, obscure task-specific structure, and require repeated use of expensive frontier judges, limiting scalability. ATLAS introduces rubric-based reinforcement finetuning , where an LLM judge evaluates agent behavior using structured, task-level rubrics instead of directly producing a single score. These rubrics decompose evaluation along task-aligned criteria such as completeness, grounding in tool outputs, appropriateness of tool choices, and parameter precision; providing a richer learning signal without step-level annotations. Crucially, once rubrics are generated, evaluation can be reliably performed by SLMs , enabling scalable RFT without continued reliance on frontier judges. We further introduce an automated pipeline for rubric generation, allowing task-specific supervision to be produced once and reused throughout training. We evaluate ATLAS on a suite of synthetic but realistic MCP tasks constructed using live MCP servers 27 . The training set comprises approximately 300 tasks spanning 28 MCP servers, and the held-out evaluation set includes tasks from both seen and previously unseen servers. We find that task-level rubric rewards yield more stable and effective reinforcement finetuning than generic outcome rewards 22 ; 10 . Notably, under rubric-based evaluation, an open-source SLM judge (Qwen3-30B-Instruct) outperforms a GPT-4o-based generic judge and matches stronger frontier judges such as o4-mini. When combined with programmatic orchestration, ATLAS consistently outperforms JSON-style tool calling in both task completion and efficiency. Our key contributions are 1. We introduce ATLAS , a reinforcement finetuning framework that enables efficient agentic behavior through adaptive context control and code-based orchestration for long-horizon tool use. 2. We show that learning these behaviors is essential: task-level, structured rubric rewards generated by an LLM judge provide stable supervision under non-verifiable objectives without requiring step-level annotations. 3. We demonstrate that rubric-based evaluation enables SLMs to serve as effective judges, outperforming GPT-4o-based generic judging, enabling scalable and cost-efficient