SkillOps: Managing LLM Agent Skill Libraries as Self-Maintaining Software Ecosystems

Xinyuan Song, Hongji Pu, Liang Zhao · 2026

Read SkillOps: Managing LLM Agent Skill Libraries as Self-Maintaining Software Ecosystems by Xinyuan Song, Hongji Pu, Liang Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13716.txt.

Introduction

Hongji Pu, Xinyuan Song and Liang Zhao offer SkillOps, a method-agnostic plug-in system for maintaining large language model agent skill libraries. SkillOps casts each skill as a typed Skill Contract, organises skills via a Hierarchical Skill Ecosystem Graph, while diagnosing library health across four dimensions. On ALFWorld, SkillOps reaches 79.5 percent task success as a standalone agent, beating the strongest baseline by 8.8 points with no extra task-time language model calls.

Catalog record

CREATORXinyuan Song, Hongji Pu, Liang Zhao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66191
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2605.13716

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SkillOps: Managing LLM Agent Skill Libraries as Self-Maintaining Software Ecosystems Xinyuan Song, Hongji Pu, Liang Zhao Abstract LLM agents increasingly rely on skill libraries for multi-step tasks, yet these libraries can accumulate persistent defects as skills are added, reused, patched, and linked to changing dependencies. We call this failure mode skill technical debt : library-level defects that may not break a single skill locally but can harm future retrieval, composition, and execution. Existing skill-based agents mainly focus on task-time retrieval, planning, and repair, while library-time maintenance remains underexplored. We propose SkillOps , a method-agnostic plug-in framework for maintaining skill libraries. SkillOps represents each skill as a typed Skill Contract ( P , O , A , V , F ) (P,O,A,V,F) , organizes skills with a Hierarchical Skill Ecosystem Graph (HSEG) , and diagnoses library health across utility, compatibility, risk, and validation dimensions. Given a raw skill library, SkillOps produces a maintained library that can be used by existing retrieval or planning agents without changing their internal code. On ALFWorld, SkillOps achieves 79.5 % 79.5\% task success as a standalone agent, outperforming the strongest baseline by + 8.8 +8.8 percentage points with no additional task-time LLM calls. As a plug-in layer, it improves retrieval-heavy baselines by + 0.68 +0.68 – + 2.90 +2.90 percentage points. The current rule-based maintenance implementation uses nearly zero library-time LLM calls or tokens, showing that skill-library maintenance can be added as a low-overhead architectural layer. Code is publicly available at https://github.com/Hik289/SkillOps.git . 1 Introduction LLM agents increasingly rely on skill libraries to solve complex, multi-step tasks ( Wang et al. 2024 ; Zheng et al. 2025 ; Shen et al. 2026 ) . A skill library stores reusable executable procedures, such as parsers, controllers, API wrappers, manipulation routines, validators, and data-processing scripts. Recent benchmarks show that access to skills improves task success ( BenchFlow AI 2026 ) , and recent systems further improve skill use through retrieval, dependency-aware graph search, and task-time composition ( Liu et al. 2026 ; Xia et al. 2026 ) . However, as agents are deployed for longer periods, their libraries do not remain static: skills are repeatedly added, patched, reused in new contexts, and connected to changing downstream dependencies. This turns the skill library from a fixed retrieval pool into a persistent software asset that requires management. We identify this library-level failure mode as skill technical debt . Following technical debt in software and ML systems ( Cunningham 1992 ; Sculley et al. 2015 ) , skill technical debt refers to persistent defects in a skill library, such as redundancy, missing validation, interface drift, or stale implementations, that may not break a single skill locally but can reduce future retrieval, composition, and execution reliability. The key issue is persistence: task-time repair may fix one failed episode, while the underlying library defect remains and can cause future failures when the skill is reused. Existing work on skill-based agents focuses mostly on task-time use of a library: how to retrieve the right skill for the current task ( Liu et al. 2026 ; Xia et al. 2026 ; Qin et al. 2024 ) , how to compose retrieved skills into executable plans ( Shen et al. 2026 ; Liu et al. 2026 ; Qin et al. 2024 ) , or how to validate and repair outputs during execution ( Zheng et al. 2025 ; Shen et al. 2026 ) . These methods are important, but they do not directly solve the library-time problem. A task-time repair may fix the current episode without updating the library; a missing validator remains missing; an incompatible interface remains exposed; and redundant implementations remain available to future retrieval. Thus, current skill-library systems often assume that the library is healthy, even though this assumption becomes weaker as the library grows. Motivated by this gap, we study skill-library maintenance under technical debt as a plug-in problem for LLM agent systems. The goal is to provide a library-management layer that runs before downstream agents use the library: given a raw skill library, it diagnoses persistent defects, applies typed repairs, and returns a cleaned library that existing retrieval-based or task-oriented agents can use without changing their internal code. To this end, we propose SkillOps , a drop-in maintenance framework that combines contract-based skill representation, graph-structured library organization, health diagnosis, and feedback-driven maintenance. SkillOps represents each skill as a Skill Contract (P,O,A,V,F) , where P denotes preconditions, O denotes the executable operation, A denotes produced artifacts, V denotes validators, and F denotes known failure modes. These contracts are organized into a Hierarchical Skill Ecosystem Graph (HSEG) , where skills are connected through typed dependency, compatibility, redundancy, and alternative edges. Based on HSEG, SkillOps computes library health along utility, redundancy, compatibility, failure-risk, and validation-gap dimensions, then applies maintenance actions such as merge, repair, and retire. Central Interface cleaned_lib = run_maintenance ( raw_lib ) This central interface produces a repaired librarythat can be passed to any downstream retrieval or planning method. In the current implementation, the maintenance loop uses observable signals such as utility logs, body-hash collisions, missing validators, failure logs, and type mismatches, and therefore incurs nearly zero LLM calls at library time. SkillOps also includes an optional task-time planner for typed skill matching, dependency stitching, validator or adapter insertion, and local repair, but this planner can be replaced by any downstream agent that reads the maintained library. Our experiments support SkillOps from four aspects. First, as a standalone agent, SkillOps reaches 79.5\% task success on ALFWorld Shridhar et al. 2021 , outperforming the strongest baseline by +8.8 percentage points while using zero additional LLM calls. Second, as a plug-in layer, SkillOps improves retrieval-heavy baselines consistently, with gains of +0.68 to +2.90 percentage points across BM25, dense, and hybrid retrieval settings. Third, the rule-based maintenance pass has nearly zero library-time LLM cost and is neutral-to-negative in task-time token usage. Our contributions are as follows: 1. We formalize skill-library maintainability as a library-time problem for LLM agents, defining HSEG together with health dimensions that capture utility, redundancy, compatibility, failure risk, and validation gaps. 2. We introduce SkillOps as a method-agnostic plug-in maintenance framework, where a raw library is transformed into a maintained library through typed actions such as merge , repair , retire , add_validator , and add_adapter . 3. We empirically show that SkillOps has method-conditional effects across agent types, helping retrieval-heavy agents while revealing when library-time maintenance is neutral or conflicts with task-time self-repair. 4. We show that the current rule-based implementation performs library maintenance without additional LLM calls or token cost, making maintenance a low-overhead architectural layer rather than an extra inference-time burden. 2 Problem Setup Skill Contract. As shown in Figure 1 , SkillOps models each skill as an executable contract rather than only a name or text description. Each skill s\in\mathcal{S} is written as s=(P,O,A,V,F) , where P is the precondition for calling the skill, O is the executable operation, A is the typed artifact produced by the skill, V is a validator over A , and F is the set of known failure modes. When V=\emptyset , the skill has no local correctness check, which we call a validation gap. This contract form allows SkillOps to check relevance,

Read the complete text