SearchSkill : Teaching LLMs to Use Search Tools with Evolving Skill Banks

Hu Jinchao, Meizhi Zhong, Kehai Chen, Min Zhang · 2026

Read SearchSkill : Teaching LLMs to Use Search Tools with Evolving Skill Banks by Hu Jinchao, Meizhi Zhong, Kehai Chen, Min Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09038.txt.

Introduction

Jinchao Hu and three co-authors address a weak point of language models that answer questions by searching: the queries they write are often broad or copied from the question. SearchSkill teaches models to plan each search step by first picking a reusable skill from a growing bank of skill cards.

The bank is not fixed. When the same kind of failure keeps appearing, the system adds or sharpens skills, rebuilds the affected practice examples, and trains again. Across open and closed models, the method raised exact-match scores on hard question-answering tests. It also produced fewer copied first queries, more focused follow-up queries, and more correct answers within a small search budget.

Catalog record

CREATORHu Jinchao, Meizhi Zhong, Kehai Chen, Min Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.09038

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SearchSkill : Teaching LLMs to Use Search Tools with Evolving Skill Banks Hu Jinchao, Meizhi Zhong, Kehai Chen, Min Zhang Abstract Teaching language models to use search tools is not only a question of whether they search, but also of whether they issue good queries. This is especially important in open-domain question answering, where broad or copied queries often waste retrieval budget and derail later reasoning. We propose SearchSkill , a framework that makes query planning explicit through reusable search skills. At each step, the model first selects a skill, then generates a search or answer action conditioned on the selected skill card. The skill inventory itself is not fixed: SearchSkill maintains an evolving SkillBank, expands or refines it from recurrent failure patterns, and reconstructs affected trajectories before supervised training. The resulting two-stage SFT recipe aligns training with the inference-time protocol of skill selection followed by skill-grounded execution. Across open-source and closed-source models, SearchSkill improves exact match on knowledge-intensive QA benchmarks and yields better retrieval behavior, including fewer copied first queries, more atomic hop-focused queries, and more correct answers within a small search budget. These results suggest that explicit skill-conditioned query planning is a lightweight alternative to treating search as an undifferentiated action. Code and data are available at here . 1 Introduction Large language models (LLMs) exhibit strong reasoning ability and broad parametric knowledge, but that knowledge remains fundamentally static: it can be outdated, incomplete, or simply missing the long-tail facts required by knowledge-intensive tasks. This limitation is especially visible in open-domain and multi-hop question answering, where solving a problem often requires gathering fresh evidence, following hidden bridge entities, and verifying intermediate conclusions against external sources. Search tools therefore play an indispensable role in extending the knowledge boundary of LLMs and turning them into reliable systems for real-world information seeking. Recent years have seen rapid progress on retrieval-augmented and tool-using LLMs. Retrieval-augmented generation (RAG) augments generation with external passages for knowledge-intensive prediction [ 15 ] . Tool-use methods such as Toolformer [ 25 ] and ReAct [ 35 ] further enable models to interleave reasoning with external actions. More recent search-centered systems train this capability more directly: Search-R1 [ 11 ] optimizes long-horizon search behavior with reinforcement learning, while ZeroSearch [ 27 ] incentivizes search capability without relying on live search during training. These advances have substantially improved LLM–retriever interaction, but they also expose a practical bottleneck. Challenge 1: most existing methods teach the model to search, but devote much less modeling capacity to how to formulate high-quality search queries. In multi-hop settings, poor queries lead to longer interaction traces, redundant tool calls, noisy evidence, and lower final answer accuracy. In other words, the issue is often not whether a model uses a retriever, but whether it uses the retriever intelligently. Recent work on agent skills offers a complementary angle. Anthropic’s Agent Skills 1 1 1 https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills made skills a first-class interface for packaging instructions, code, and resources that an agent can load on demand. Follow-up analyses show that skills rapidly became a practical mechanism for extending model functionality, while also raising ecosystem-level questions about organization and safe reuse [ 18 , 19 ] . AgentSkillOS studies selection and benchmarking over large skill ecosystems [ 16 ] , while Reinforcement Learning for Self-Improving Agent with Skill Library, MemSkill, and SkillRL explore how agents can maintain and evolve skill libraries or skill banks over training [ 30 , 36 , 33 ] . However, applying this intuition to search tool use is still non-trivial. Challenge 2: current skill frameworks do not directly tell us how skills should be represented, invoked, and updated inside search trajectories. A useful search skill must shape not only a high-level strategy, but also the next query, the arguments passed to the retriever, the verification behavior after retrieval, and the stopping decision before answering. To address these challenges, we propose SearchSkill , a framework that teaches LLMs to use search tools through an evolving bank of reusable search skills. Rather than treating search as an undifferentiated action, SearchSkill factorizes each interaction step into explicit skill selection and tool execution. Concretely, the model first emits a <skill> … </skill> tag to declare the skill(s) it will use at the current turn, and then either issues a <search> … </search> query or returns an <answer> … </answer> ; retrieved evidence is fed back through <information> … </information> . This token-level interface makes the skill choice an explicit, trainable control variable before every retrieval or answer action. We instantiate the bank with search-specific skills such as bridge-entity search, parallel attribute comparison, temporal range extraction, conflict checking, verbatim evidence extraction, and answer grounding. Training is stage-wise: we first iteratively refine the SkillBank through a failure-driven outer loop, where current rollouts expose stable failure patterns, accepted add/refine updates revise the bank, and affected trajectories are reconstructed under each accepted update. Once the bank stabilizes, we perform skill-conditioned supervised fine-tuning on high-quality positive trajectories, distilling mature bank-consistent behavior into the policy.Our main contributions are as follows: • We propose a skill-conditioned search-tool interface that improves query formulation by making the model choose a reusable search strategy before each tool call, addressing the common failure of issuing broad or copied multi-hop queries. • We introduce an evolvable Search SkillBank that is updated through failure analysis, add/refine operations, and trajectory reconstruction, providing a reusable tool-use prior that can extend beyond the QA setting. • We show that a two-stage supervised fine-tuning recipe can obtain performance close to or stronger than large-scale RL-based search agents, while avoiding the high rollout and optimization cost of training search behavior primarily through RL. • We demonstrate effectiveness on both open-source and closed-source models: fine-tuned Qwen2.5 3B/7B models improve QA performance, and the frozen SkillBank also benefits proprietary models as an external planning guide. 2 Related work 2.1 RAG and RL Retrieval-augmented generation for knowledge-intensive QA began with retrieval-aware pretraining and dense retrieval backbones such as REALM and DPR [ 4 , 13 ] . RAG and FiD then made retrieved passages central to generation and reading [ 15 , 7 ] . Later systems such as RETRO, Atlas, and IRCoT further tightened the coupling between retrieval and reasoning [ 2 , 8 , 29 ] . Dynamic retrieval policies were subsequently explored by Active RAG, Self-RAG, and unified active retrieval [ 10 , 1 , 3 ] . In parallel, search became an explicit action space rather than a passive back-end. WebGPT and ReAct framed browsing and search as sequential reasoning-and-acting [ 21 , 35 ] , while Toolformer and Tool Learning generalized this pattern to broader tool use [ 25 , 23 ] . Search-o1 further embeds agentic search into long reasoning traces without reducing the problem to a static retrieve-then-read pipeline [ 17 ] . RL-based search agents then push this line further: R1-Searcher, DeepRetrieval, and Search-R1 directly optimize search behavior with reinforcement learning [ 26 , 9 , 11 ] . ZeroSearch complements them by improving search capability without live search during training [ 27 ] . Together, these works frame search as a trainable decision process rather than a fixed retrieval module. 2.2 Skills A particularly relevant notion of skill is the recent agent-skill

Read the complete text