Skill0 : In-Context Agentic Reinforcement Learning for Skill Internalization

Zhengxi Lu 1,2⋆ , Zhiyuan Yao 2 , Jinyang Wu 3 , Chengcheng Han 2 , Qi Gu 2† Xunliang Cai 2 , Weiming Lu 1 , Jun Xiao 1 , Yueting Zhuang 1 , Yongliang Shen 1† 1 Zhejiang University 2 Meituan 3 Tsinghua University {zhengxilu, syl}@zju.edu.cn [email protected] · 2026

Read Skill0 : In-Context Agentic Reinforcement Learning for Skill Internalization by Zhengxi Lu 1,2⋆ , Zhiyuan Yao 2 , Jinyang Wu 3 , Chengcheng Han 2 , Qi Gu 2† Xunliang Cai 2 , Weiming Lu 1 , Jun Xiao 1 , Yueting Zhuang 1 , Yongliang Shen 1† 1 Zhejiang University 2 Meituan 3 Tsinghua University {zhengxilu, syl}@zju.edu.cn [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-02268.txt.

Introduction

Lu and colleagues internalize agent skills into model weights instead of retrieving them at run time. Their SKILL0 curriculum starts with full skill context and withdraws it step by step, grouping skills by category and rendering them with interaction history as compact visual context. A dynamic curriculum keeps only skills the current policy still gains from, within a shrinking budget, until the agent acts zero shot. Gains over standard reinforcement learning on ALFWorld, Search-QA and WebShop tasks come with tiny per step context.

Catalog record

CREATORZhengxi Lu 1,2⋆ , Zhiyuan Yao 2 , Jinyang Wu 3 , Chengcheng Han 2 , Qi Gu 2† Xunliang Cai 2 , Weiming Lu 1 , Jun Xiao 1 , Yueting Zhuang 1 , Yongliang Shen 1† 1 Zhejiang University 2 Meituan 3 Tsinghua University {zhengxilu, syl}@zju.edu.cn [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48815
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.02268

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Skill0 : In-Context Agentic Reinforcement Learning for Skill Internalization Zhengxi Lu 1,2⋆ , Zhiyuan Yao 2 , Jinyang Wu 3 , Chengcheng Han 2 , Qi Gu 2† Xunliang Cai 2 , Weiming Lu 1 , Jun Xiao 1 , Yueting Zhuang 1 , Yongliang Shen 1† 1 Zhejiang University 2 Meituan 3 Tsinghua University {zhengxilu, syl}@zju.edu.cn [email protected] Abstract Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a reliable mechanism for augmenting LLM agents. Yet inference-time skill augmentation is fundamentally limited: retrieval noise introduces irrelevant guidance, injected skill content imposes substantial token overhead, and the model never truly acquires the knowledge it merely follows. We ask whether skills can instead be internalized into model parameters, enabling zero-shot autonomous behavior without any runtime skill retrieval. We introduce Skill0 , an in-context reinforcement learning framework designed for skill internalization . Skill0 introduces a training-time curriculum that begins with full skill context and progressively withdraws it. Skills are grouped offline by category and rendered with interaction history into a compact visual context, teaching he model tool invocation and multi-turn task completion. A Dynamic Curriculum then evaluates each skill file’s on-policy helpfulness , retaining only those from which the current policy still benefits within a linearly decaying budget, until the agent operates in a fully zero-shot setting. Extensive agentic experiments demonstrate that Skill0 achieves substantial improvements over the standard RL baseline (+9.7% for ALFWorld and +6.6% for Search-QA), while maintaining a highly efficient context of fewer than 0.5k tokens per step. Our code is available at https://github.com/ZJU-REAL/SkillZero . 1 Introduction “ Skills at training, zero at inference.” — Skill0 Large Language Models (LLMs) ( 11 ; 43 ; 55 ; 6 ; 44 ) have demonstrated strong decision-making capabilities across complex real-world tasks ( 37 ; 39 ; 14 ; 19 ) . With the emergence of agent scaffolds like Claude Code and OpenClaw, structured Agent Skills ( 54 ; 23 ; 13 ) have become the standard mechanism for extending agent capabilities on specialized tasks. Figure 1: Comparison of (a) Skill Augmentation methods and (b) our Skill Internalization method. The prevailing paradigm is inference-time skill augmentation: relevant skills are retrieved from a skill bank and injected into the model’s context as natural language guidance at each step ( 24 ; 26 ) . This approach has proven effective and is now well-established, with growing ecosystems of skill libraries, retrieval pipelines, and evolution mechanisms ( 54 ; 53 ) . Yet this practical success obscures a more fundamental limitation. First, retrieval noise introduces irrelevant or misleading guidance that corrupts the agent’s context ( 58 ; 50 ) . Second, injected skill content imposes token overhead that compounds across multi-turn interactions, limiting scalability ( 30 ; 16 ) . Third, and most critically, a model that follows skill descriptions in its prompt is executing skills, not learning them: competence resides in the context, not in the model ( 48 ; 12 ) . This observation suggests a different question: rather than asking how to better retrieve and inject skills, can skills be internalized into model parameters, rendering retrieval unnecessary at inference time? Skill acquisition in humans follows a familiar progression: an explicit instruction phase gives way to an internalized phase in which the same behavior is executed autonomously from memory ( 2 ; 62 ) . Inference-time skill augmentation permanently anchors agents in the first stage. Reinforcement learning offers a natural path to the second, driving the agent to consolidate effective strategies as intrinsic policy rather than reading them from context ( 10 ; 36 ) . Yet a naive application of RL fails in both directions: without skill context, the agent lacks the structured guidance necessary to learn complex multi-step behavior; with full skill context throughout, the model remains dependent on external knowledge it has never been required to internalize. What is needed is a training regime that starts with skills and ends without them, systematically transferring competence from context to parameters. We propose Skill0 , the first RL framework that formulates skill internalization as an explicit training objective. Skill0 realized this curriculum through In-Context Reinforcement Learning (ICRL): skills are provided as in-context guidance during training rollouts but removed entirely at inference, so that RL optimization directly drives the transition from context-dependent execution to autonomous behavior. Concretely, skills are grouped offline by category and rendered with interaction history into a compact visual context, teaching the model tool invocation and multi-turn task completion. Dynamic Curriculum evaluates each skill file’s on-policy helpfulness by comparing agent performance with and without it on a matched validation sub-task. Skills are retained only where the current policy still benefits, and discarded otherwise, until the budget reaches zero and the agent operates without any skill context. Extensive experiments demonstrate that Skill0 achieves substantial improvements over strong baselines like AgentOCR (+9.7% for ALFWorld and +6.6% for Search-QA), and competitive performance against skill-augmented methods like SkillRL. Notably, by eliminating skill reliance at inference time, Skill0 maintains a highly efficient context of fewer than 0.5k tokens per step, significantly reducing inference overhead without sacrificing task performance. • We propose Skill0 , the first RL framework that formulates skill internalization as an explicit training objective, moving agents from inference-time skill dependence to fully autonomous zero-shot behavior. • We introduce in-context reinforcement learning, which provides structured skill guidance during training rollouts and removes it entirely at inference, directly optimizing the transition from context-dependent execution to intrinsic competence. • We propose Dynamic Curriculum, a helpfulness-driven annealing mechanism that withdraws each skill only when the current policy no longer benefits from it, replacing rigid schedules with adaptive internalization. 2 Related Work 2.1 LLM Agents Recent advancements in instruction-tuned LLMs have enabled autonomous agents to operate across a wide range of dynamic, open-world environments, including code generation ( 19 ; 46 ) , GUI automation ( 60 ; 28 ) , gameplay ( 41 ) , and embodied control ( 47 ) . With the recent development of reinforcement learning for LLMs ( 61 ; 66 ; 59 ; 4 ) , agentic RL has emerged as a crucial post-training recipe for equipping LLM agents with robust decision-making capabilities ( 31 ; 32 ; 8 ) . Figure 2: Overview of Skill0 . (a) Relevance-Driven Skill Grouping; (b) In-Context Reinforcement Learning with skill-enhanced agent loop; (c) Dynamic curriculum learning during training process. 2.2 Agentic Skills Early memory-based approaches store raw trajectories directly into external databases during the sampling process, serving as references for experience replay ( 64 ; 40 ) . However, such raw trajectories are often lengthy, redundant, and noisy, making direct injection into the context window inefficient ( 5 ) . To address this limitation, a growing line of work has explored skills —reusable, abstracted, and structured behavioral primitives distilled from historical trajectories ( 54 ; 23 ; 13 ) . Skills serve as a form of episodic memory that agents can consult at decision time ( 24 ; 27 ; 26 ) , and have further been shown to provide efficient guidance within reinforcement learning frameworks ( 53 ; 48 ; 18 ) . Despite these advances, existing methods predominantly focus on skill extraction, organization,

Read the complete text