Can Small Agent Collaboration Beat a Single Big LLM?
Agata Żywot, Xinyi Chen, Maarten de Rijke · 2026
Read Can Small Agent Collaboration Beat a Single Big LLM? by Agata Żywot, Xinyi Chen, Maarten de Rijke (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-11327.txt.
Introduction
Agata Zywot, Xinyi Chen, and Maarten de Rijke test teams of small AI agents. Their 2026 report asks if such teams can beat one big model. The test bed is GAIA, a benchmark of hard general tasks. The agents are Qwen3 models with tools in hand.
Model sizes in the tests run from 4B to 32B. The team varies three things to see what counts most. One is scale, the raw size of the model used. One is explicit thinking, from none to planner-only to full. The third is tool use: search, code, and mind-maps. Tool use gave the largest and most steady gains. That finding favors smart setups over raw size alone.
Catalog record
| CREATOR | Agata Żywot, Xinyi Chen, Maarten de Rijke |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-11327
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can Small Agent Collaboration Beat a Single Big LLM? Agata Żywot, Xinyi Chen, Maarten de Rijke Abstract This report studies whether small, tool-augmented agents can match or outperform larger monolithic models on the GAIA benchmark. Using Qwen3 models (4B–32B) within an adapted Agentic-Reasoning framework, we isolate the effects of model scale, explicit thinking (no thinking, planner-only, or full), and tool use (search, code, mind-map). Tool augmentation provides the largest and most consistent gains. Using tools, 4B models can outperform 32B models without tool access on GAIA in our experimental setup. In contrast, explicit thinking is highly configuration- and difficulty-dependent: planner-only thinking can improve decomposition and constraint tracking, while unrestricted full thinking often degrades performance by destabilizing tool orchestration, leading to skipped verification steps, excessive tool calls, non-termination, and output-format drift. 1 Introduction Agentic-AI systems are increasingly leveraging language models to autonomously plan, reason, and act. While Large Language Models (LLMs) have traditionally served as the backbone for these systems, recent work argues that smaller language models (SLMs) may be better suited for many agentic tasks due to their efficiency, cost-effectiveness, and modular architecture 1 . Despite these advantages, SLMs are often perceived as fundamentally limited by weaker reasoning and planning abilities compared to frontier-scale models. A promising alternative to pure scaling is to augment models with explicit tools and structured reasoning mechanisms. Frameworks such as Agentic-Reasoning demonstrate that equipping language models with specialized tool-agents—such as web search, code execution, and external memory—can substantially improve performance on multi-step reasoning tasks 6 . This raises a question: can architectural and agentic design compensate for smaller model size? In this work, we investigate whether an agentic decomposition using small, tool-augmented models can match or outperform a single large monolithic model. We focus specifically on three factors: model size , thinking , and tool usage . Understanding their effects is crucial for designing efficient agentic systems that do not rely exclusively on scaling. We study these questions using the GAIA benchmark 3 , which targets agentic capabilities across three difficulty levels (L1–L3), including multi-hop reasoning, planning, and tool use. Specifically, we address the following research questions: • RQ1: What are the capabilities of small models when embedded in a collaborative agentic framework? • RQ2: Can a collaboration of small agents outperform a single larger model? • RQ3: How do explicit thinking and tool usage contribute to performance across model scales? Our contributions can be summarized as the following findings: • Small models equipped with tool-augmented agentic architectures can outperform much larger monolithic models on GAIA. • Tool usage provides larger and more consistent gains than explicit thinking alone, particularly for smaller models. • The effectiveness of thinking depends on both model scale and agent role: planner-only thinking benefits instruction-tuned small models (4B-Instruct), but does not consistently help base 4B models, while full thinking frequently degrades accuracy by disrupting tool coordination—particularly on computation-heavy and long-horizon coordination tasks. 2 Related Work 2.1 Tool-Augmented Language Models A growing body of work demonstrates that external tools significantly enhance the problem-solving abilities of language models. Systems integrating web search, code execution, or retrieval mechanisms enable models to overcome knowledge cutoffs and computational limitations. The Agentic-Reasoning framework 6 exemplifies this approach by decomposing reasoning into planning and tool-invocation stages. While prior work shows strong gains for large models, it remains unclear to what extent such gains transfer to smaller backbones. 2.2 Explicit Thinking and Planning Explicit reasoning mechanisms, such as Chain-of-Thought (CoT) prompting and planner-style decomposition, have been shown to improve performance on arithmetic and symbolic reasoning benchmarks by encouraging step-by-step problem decomposition 5 . However, recent work highlights important limitations of explicit thinking. CoT prompting can obscure hallucination cues and complicate model behavior, e.g., leading models to overcommit to intermediate steps that are incorrect 2 . Moreover, explicit reasoning does not universally improve performance and can even degrade accuracy by introducing unnecessary or noisy intermediate steps, particularly when model capacity is limited 9 . As a result, when and where explicit thinking is beneficial—especially in tool-augmented agentic settings—remains an open research question. 2.3 Multi-Agent Collaboration Multi-agent systems distribute reasoning across specialized components or roles. Approaches such as ToolOrchestra 4 and MaAS 8 show that collaboration can outperform monolithic systems while reducing cost. However, most existing work focuses on architectural search, leaving open how collaboration interacts with explicit thinking and tool usage across model sizes. Our work directly addresses this gap. 3 Methodology 3.1 Framework Adaptation We build on the Agentic-Reasoning framework 6 , adapting it for local execution with open-source models. A key constraint is GPU memory, which motivates our decision to use a single shared model instance with locking. Instead of deploying separate models for each agent role, we dynamically re-role a single backbone via specialized system prompts. This preserves modular reasoning while remaining feasible on academic hardware. 3.2 Tool-Agent Integration We integrated and adapted three primary tool-agents to handle the multi-step requirements of complex reasoning tasks, build upon the Agentic Reasoning implementation. The tools are defined as follows: 1. Web-Search Agent: Utilizes a RAG-based query breakdown to decompose complex questions into searchable sub-queries. It iteratively retrieves and filters web content to provide the Planner with grounded context. 2. Coding Agent: Provides a sandbox environment for algorithmic execution. It is triggered when tasks require precise mathematical calculations, data manipulation, or the parsing of structured formats. 3. Mind-Map Agent: Designed for long-context management, constructs a dynamic knowledge graph (Mind-Map) during the reasoning process. It allows the system to store, link, and retrieve intermediate findings, preventing "forgetting" during long-horizon tasks. We compare standard single-model inference with an agentic setup where the same backbone model alternates between planning, searching, coding, and memory roles using different prompts. Tools are kept identical across all runs. 4 Experiments 4.1 Experimental Setup Models. We evaluate Qwen3 models ranging from 4B to 32B parameters 7 . This range allows us to test whether scaling alone can replace explicit tool usage and whether smaller models benefit more from agentic augmentation. We additionally include the 4B-Instruct variant to assess the impact of tool- and instruction-focused fine-tuning. Datasets. We use the GAIA benchmark 3 , which is explicitly designed to evaluate agentic behavior. GAIA Level 1 primarily tests factual retrieval and shallow reasoning, Level 2 emphasizes multi-hop reasoning and planning, and Level 3 requires complex coordination between reasoning and tools. This stratification enables analysis of capability breakdowns. We use the validation split, comprising 165 questions (53, 86 and 26 per Level 1-3, respectively). Evaluation Metrics. We report accuracy (ACC) overall and per difficulty level. Accuracy directly reflects whether an agentic system successfully completes the task end-to-end, aligning with our research questions about practical capability rather than intermediate reasoning quality. Reasoning and Tool Usage. We evaluate three reasoning configurations: no explicit thinking, planner-only thinking, and full thinking across all agents. This allows us to isolate whether explicit reasoning helps planning, execution, or both. Model Config Thinking ACC [%] L1 [%] L2 [%] L3 [%] 4B-Instruct No-Tools NO 9.70 20.75 5.81 0.00 YES 10.91 18.87 9.30 0.00 Agentic NO 16.36 30.19 12.79 0.00 PLANNER 18.18 30.19 15.12 3.85 YES 15.76 26.42 13.95 0.00 4B No-Tools NO 6.06 9.43 4.65 3.85 YES 9.09 15.09 8.14 0.00 Agentic NO 13.33 15.09 16.28 0.00