SkillC : Learning Autonomous Skill Internalization in LLM Agents via Contrastive Credit Assignment
Hongxiang Lin, Zhirui Kuai, Erpeng Xue, Lei Wang, {kuaizhirui, wanglei46, xueerpeng}@meituan.com · 2026
Read SkillC : Learning Autonomous Skill Internalization in LLM Agents via Contrastive Credit Assignment by Hongxiang Lin, Zhirui Kuai, Erpeng Xue, Lei Wang, {kuaizhirui, wanglei46, xueerpeng}@meituan.com (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27899.txt.
Introduction
Lin and colleagues train language-model agents to internalize skills so they no longer need skill prompts at run time. SkillC pairs rollouts with and without skill help inside the same update, turning the contrast into a direct learning signal for self-reliance. On two agent benchmarks it beat the best prior internalization method while staying close to skill-assisted rivals.
Catalog record
| CREATOR | Hongxiang Lin, Zhirui Kuai, Erpeng Xue, Lei Wang, {kuaizhirui, wanglei46, xueerpeng}@meituan.com |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38740 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.27899
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SkillC : Learning Autonomous Skill Internalization in LLM Agents via Contrastive Credit Assignment Hongxiang Lin, Zhirui Kuai, Erpeng Xue, Lei Wang, {kuaizhirui, wanglei46, xueerpeng}@meituan.com Abstract Structured skill prompts improve exploration in long-horizon agentic reinforcement learning (RL). Skill-augmented RL methods retain external skills at inference, while skill-internalization RL methods withdraw them during training to enable autonomous performance. However, existing internalization approaches only use skill-helpfulness contrast for curriculum control, leaving the policy update unchanged and unable to distinguish skill-dependent from autonomous success. We propose SkillC , a framework based on Contrastive Skill Credit Assignment (CSCA) that converts this contrast into a direct learning signal for internalization. SkillC samples paired skill-injected and skill-free rollouts for tasks from active skill types within the same policy update, and injects their task-level contrast into optimization via a dual-stream advantage estimator that preserves global ranking while applying a one-sided correction toward skill-free success. A smoothed validation-level signal further drives an adaptive curriculum over attribution strength, rollout allocation, and monotonic active-set pruning. Experiments on ALFWorld and WebShop show that, without runtime skill access, SkillC surpasses the strongest prior skill-internalization RL baseline by 5.5% and 4.4%, respectively, while remaining competitive with skill-augmented RL methods. 1 Introduction Figure 1: Comparison of paradigms for skill use in Agentic RL. (a) Skill-augmented RL methods keep skills available at inference time and optimize performance with runtime skill support. (b) Skill-internalization RL methods withdraw skills during training and may estimate skill helpfulness for control, but leave task-level credit assignment unchanged. (c) SkillC turns helpfulness comparison into contrastive credit assignment for autonomous success, while adapting attribution strength, rollout allocation, and the active skill set through a validation-driven curriculum. Structured skill prompts 27 ; 32 ; 10 have become a practical mechanism for training long-horizon LLM agents. By providing reusable procedural guidance during interaction, they improve exploration and sample efficiency in agentic reinforcement learning (RL) 7 ; 8 ; 21 ; 3 . Recent work has advanced this paradigm through skill augmentation and co-evolving skill banks 8 , skill distillation 24 ; 26 , and skill lifecycle management 14 ; 13 ; 17 . Yet most of these approaches remain in the skill-augmented regime shown in Figure 1 (a): skills are treated as external runtime support, and autonomous performance without retrieval is not the optimization target. A more ambitious goal is skill internalization, illustrated in Figure 1 (b), where skills are used only during training and gradually withdrawn so that the final policy can act autonomously. Skill0 12 is a representative example. Current internalization methods can already estimate whether a skill remains useful and use that information to control withdrawal. What remains unchanged, however, is the policy objective. Within each GRPO update 16 , all rollouts for a task are still generated under a single prompt condition, so optimization reinforces success under the current regime rather than autonomy after withdrawal. The training loop may know that a skill is still globally useful, yet it cannot determine whether a particular successful trajectory is already independent of that skill. We term this optimization mismatch internalization blindness . To address it, we propose SkillC , a framework based on Contrastive Skill Credit Assignment (CSCA) that converts this contrast into a direct learning signal for internalization. For tasks from active skill types, SkillC samples paired skill-injected and skill-free rollouts within the same policy update, injecting their task-level contrast into optimization via a dual-stream advantage estimator that preserves global ranking while applying one-sided correction toward skill-free success. A smoothed validation-level counterpart of the same contrastive quantity further drives a hybrid adaptive curriculum. For multi-category environments like ALFWorld, this curriculum jointly controls attribution strength, rollout allocation, and monotone active-set pruning; for single-skill environments like WebShop, it controls attribution strength and rollout allocation uniformly. Together, these mechanisms yield the contrastive internalization regime illustrated in Figure 1 (c). Our main contributions are as follows: • We identify trajectory-level credit assignment limitations in fixed-schedule skill internalization where all rollouts share a single prompt condition, preventing the gradient from distinguishing autonomous versus skill-dependent success. We propose SkillC with paired contrastive rollouts and adaptive curriculum scheduling to address this. • We implement CSCA in SkillC through three coordinated mechanisms. Paired skill-injected and skill-free rollouts expose task-level internalization gaps, a dual-stream advantage estimator with per-condition normalization redirects credit toward autonomous success, and a validation-driven curriculum adapts attribution strength and rollout allocation. • Critical characterization of applicability reveals CSCA’s effectiveness is signal-quality-dependent. Strong contrastive signals enable substantial gains (Heat +10.5\% , Cool +11.1\% , Pick2 +33.4\% ), while weak signals on ceiling-effect tasks cause regression (Pick -11.5\% ), and single-skill environments yield only marginal improvements (WebShop +3.1\% ). 2 Related Work 2.1 Agentic Reinforcement Learning Reinforcement learning has become a key post-training paradigm for language model agents. GRPO 16 introduced critic-free policy optimization using group-relative advantages. Building on this foundation, agentic RL methods address agent-specific challenges through hierarchical RL for long-horizon credit assignment 8 , self-evolution mechanisms for policy co-adaptation 4 ; 18 , experience reuse for continuous adaptation 2 ; 9 , and prompt-based reasoning like ReAct 31 and Reflexion 19 . However, a key challenge remains: how to leverage intermediate guidance during training while ensuring the final policy acquires genuine autonomous competence rather than guidance-dependent behaviors. 2.2 Skill Libraries for Agents Structured skills have become a core mechanism for extending agent capabilities beyond parametric memory. Early work recognized that raw trajectory retrieval incurs substantial token overhead and noise, motivating abstraction into reusable procedural knowledge 26 ; 12 . Recent skill-based systems span multiple dimensions: hierarchical skill organization and discovery through reflection 22 ; 23 , dependency-aware retrieval via graph structures 6 , and dynamic lifecycle management including retention and retirement 17 ; 14 . For skill selection, approaches range from frozen embedding-based retrieval to learned routing and policy-driven ranking 26 ; 3 . For utilization, RL-based methods condition policies on selected skills with hierarchical reward structures 22 ; 18 . For distillation, methods span prompt-based extraction and self-reflection to co-evolving skill generators 26 ; 24 . However, existing skill-augmented RL methods do not jointly optimize all three stages with unified learning signals, and existing skill internalization approaches progressively withdraw skills according to curriculum schedules but leave the policy update itself unchanged during training. Within each gradient step, all rollouts share the same prompt condition, preventing the gradient from observing which behaviors are already autonomous versus skill-dependent. This creates an optimization mismatch we term internalization blindness. 3 Method Figure 2: Overview of SkillC . Paired contrastive rollouts expose residual skill dependence for each task, a dual-stream advantage estimator redirects credit toward autonomous success without mixed-normalization bias, and a internalization-aware curriculum adapts attribution strength, rollout allocation, and active skill set. We propose SkillC , addressing a key limitation in skill internalization: within each policy update, standard GRPO jointly normalizes skill-injected and skill-free trajectories, inflating the reward baseline when skill-injected rollouts dominate and suppressing credit for autonomous success. SkillC resolves this through three mechanisms: (1) sampling paired skill-injected and skill-free rollouts to measure task-level internalization gaps, (2) using per-condition advantage normalization to redirect credit toward autonomous behaviors, and (3) adapting the curriculum via validation-level contrastive signals to balance attribution strength, rollout allocation, and skill retention. 3.1 Problem Formulation For a