CODESKILL : Learning Self-Evolving Skills for Coding Agents

Yanzhou Li, Yiran Zhang, Xiaoyu Zhang, Xiaoxia Liu, Yang Liu · 2026

Read CODESKILL : Learning Self-Evolving Skills for Coding Agents by Yanzhou Li, Yiran Zhang, Xiaoyu Zhang, Xiaoxia Liu, Yang Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25430.txt.

Introduction

Yanzhou Li and colleagues let coding agents evolve reusable skills, not fixed prompts and heuristic updates. Their CODESKILL framework treats skill extraction and bank upkeep as a learnable policy, trained with reinforcement learning. Skills are extracted at several granularities from trajectories, updated with experience, and kept in a compact bank for future tasks. A hybrid reward mixes rubric skill-quality feedback with sparse execution feedback from a frozen agent. Tests on three benchmarks show the method beats no-skill setups by 11.03 and the strongest prompt or memory baseline by 5.10.

Catalog record

CREATORYanzhou Li, Yiran Zhang, Xiaoyu Zhang, Xiaoxia Liu, Yang Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70950
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.25430

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CODESKILL : Learning Self-Evolving Skills for Coding Agents Yanzhou Li, Yiran Zhang, Xiaoyu Zhang, Xiaoxia Liu, Yang Liu Abstract Coding agents produce rich trajectories while solving software-engineering tasks. To enable agent self-evolution, these trajectories can be distilled into reusable procedural skills that compactly encode experience to guide future behavior. However, existing skill construction and maintenance methods often rely on fixed prompts and heuristic update rules, leaving it unclear how knowledge should be selected, abstracted, and maintained to best serve downstream agents. We propose CODESKILL , an LLM-based framework that reformulates skill extraction and skill-bank maintenance as a learnable management policy. CODESKILL extracts multi-granularity procedural skills from coding-agent trajectories, evolves skills with new experience, and maintains a compact skill bank for future task solving. We train CODESKILL with reinforcement learning, using a hybrid reward that combines dense rubric-based skill-quality feedback with sparse verifiable execution feedback from the frozen downstream agent. Experiments on EnvBench, SWE-Bench Verified, and Terminal-Bench 2 show that CODESKILL improves average pass rate by 9.69 over the no-skill baseline and by 4.01 over the strongest prompt-based or memory baseline, while maintaining the skill bank at a stable size. 1 Introduction Large language model (LLM) agents have shown strong capabilities in complex interactive tasks, such as coding agents that resolve realistic software issues. These long-horizon interactions produce rich trajectories with reusable experience beyond individual tasks. To reuse such experience, recent agent systems explore memory mechanisms that store prior episodes, retrieve relevant cases, or summarize past trajectories ( 2 ; 24 ; 42 ; 21 ) . Beyond raw episodic memory, higher-level procedural knowledge such as skills provides a more compact and reusable way to specify when a recurring situation arises and what actions should be taken. Skill-based agents and skill libraries have been shown effective in long-horizon and tool-use settings, making skills an important form of reusable agent knowledge for improving future behavior ( 26 ; 11 ; 10 ) . Inspired by the effectiveness of skills, recent work has begun to automate the distillation of skills and other reusable knowledge from past trajectories to support self-evolving agents. Across general agents and coding agents, these methods use LLMs to analyze prior successes and failures, extract reusable lessons, workflows, or reasoning patterns, and update memory over time for future task solving ( 14 ; 30 ; 33 ; 2 ; 17 ) . Despite this progress, existing approaches still rely heavily on fixed prompts and heuristic criteria ( 30 ; 33 ; 21 ) , which predefine what to extract and how to update memory rather than adapting these decisions to the downstream agent. This limitation is especially salient for software-engineering (SWE) agents, where long-horizon, tool-driven trajectories contain dense and heterogeneous feedback, making it difficult to distinguish reusable procedural knowledge from task-specific or accidental details. As a result, it remains unclear what evidence should be extracted, how abstract a skill should be, and how the skill bank should be revised so that the resulting skills are both reusable across tasks and actionable for concrete agent decisions. To this end, we propose CODESKILL , an LLM-based framework optimized to generate reusable skills from coding-agent trajectories and maintain a skill bank for augmenting future software-engineering tasks. Given past trajectories and an existing skill bank, CODESKILL extracts reusable procedural knowledge as skills, evolves existing skills based on new or failed experience, and maintains the skill bank by adding useful candidates, merging redundant ones, or dropping unhelpful skills. To train CODESKILL , we first warm-start it with supervised data constructed from coding-agent trajectories and teacher-generated skill operations, and then optimize it with Group Relative Policy Optimization (GRPO) using feedback from a frozen downstream coding agent ( 20 ) . The reward combines verifiable task feedback on whether the skill helps the downstream agent solve the task with rubric-based judgments of skill quality and behavior-skill alignment ( 40 ) . This enables CODESKILL to learn a management policy for extracting, refining, and updating skills that remain reusable and actionable. We validate CODESKILL by instantiating it with Qwen3.5-4B and evaluating it on EnvBench, SWE-Bench Verified, and Terminal-Bench 2. Across these benchmarks, CODESKILL improves average pass rate by 9.69 over the no-skill baseline and by 4.01 over the strongest prompt-based skill extraction or memory-construction baseline, corresponding to relative gains of about 33% and 11%, respectively. The gains hold across both in-domain and out-of-distribution software-engineering tasks. Our contributions are summarized as follows. • We propose CODESKILL , an LLM-based framework that analyzes coding-agent trajectories across diverse tasks, extracts reusable skills, and maintains an evolving skill bank for software-engineering tasks. • We reformulate skill management as a learnable management policy, and train CODESKILL with reinforcement learning. Our reward combines sparse verifiable feedback with dense rubric-based judgments, balancing executable outcomes with informative supervision when task success is sparse. • We validate CODESKILL on EnvBench, SWE-Bench Verified, and Terminal-Bench 2, showing that it achieves state-of-the-art performance among memory-based methods. 2 Related Work 2.1 Self-evolving Long-term Memory Long-term memory enables LLM agents to retain experience beyond a single interaction and improve over time. Early memory-augmented agents store reflections, past experiences, or persistent user information for later retrieval ( 22 ; 38 ; 3 ; 41 ) . Recent self-evolving agents further learn utility estimates for episodic memories, distill trajectories into reusable principles, or construct reasoning and workflow memories from successes and failures ( 36 ; 29 ; 17 ; 28 ) . Procedural memory summarizes agent behaviors into reusable action knowledge ( 5 ) . Beyond general agent settings, several works augment software-engineering agents with memories distilled from past repair trajectories or shared across agent frameworks ( 2 ; 24 ) . Another line studies how coding agents can retrieve and reuse related issue contexts or subtask-level histories ( 42 ; 21 ) . These works show that coding agents can benefit from past experience, but they mainly retrieve or summarize prior cases. CODESKILL instead learns to extract, evolve, and maintain procedural memories using downstream feedback. 2.2 Agent Skills & Automatic Skill Management Agent skills have emerged as a procedural form of memory that augments agents with reusable workflows, instructions, scripts, or tool-use knowledge. Unlike episodic memories that mainly recall past interactions, skills are designed to provide actionable guidance when similar situations arise ( 26 ; 11 ; 10 ) . Recent work has therefore studied automatic skill construction from trajectories and interaction histories, where agents distill reusable lessons, workflows, or failure patterns into skill banks that can be reused and updated over time ( 14 ; 33 ; 39 ; 1 ; 35 ; 12 ; 37 ) . Several methods further combine skills with reinforcement learning or agent self-improvement loops ( 30 ; 27 ; 25 ; 9 ) . These works typically improve the task-solving policy with skills, train agents to use skill libraries, or co-evolve skills with the solver. In contrast, CODESKILL focuses on skill management itself, learning how to generate and maintain skills that are adapted to a frozen downstream coding agent. Recent studies on software-engineering skills also show that static skill injection does not always improve coding agents, and that skill usefulness depends strongly on task fit and context ( 6 ) . This further motivates adaptive

Read the complete text