How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings

Yujian Liu Jiabao Ji Li An Tommi Jaakkola Yang Zhang Shiyu Chang · 2026

Read How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings by Yujian Liu Jiabao Ji Li An Tommi Jaakkola Yang Zhang Shiyu Chang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-04323.txt.

Description

Liu and colleagues benchmark reusable agent skills outside ideal lab conditions. Skills are domain-specific knowledge packs extending language-model agents, but past tests hand agents narrow skills tailored to each task. In realistic settings agents must find and pick skills themselves from 34 thousand real-world skills, with no hand-curated help. Gains fade steadily as settings grow more realistic, nearly reaching no-skill baselines. Query-specific refinement recovers much lost ground when starting skills are decent, and on Terminal-Bench 2.0 it lifts Claude Opus 4.6 from 58 percent to 66 percent.

Catalog record

CREATORYujian Liu Jiabao Ji Li An Tommi Jaakkola Yang Zhang Shiyu Chang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50405
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.04323

Browse the full library · About the Tower of Babel