Automated Capability Discovery via Foundation Model Self-Exploration

Cong Lu, Shengran Hu, Jeff Clune, * Equal Contribution University of British Columbia · 2025

Read Automated Capability Discovery via Foundation Model Self-Exploration by Cong Lu, Shengran Hu, Jeff Clune, * Equal Contribution University of British Columbia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-07577.txt.

Description

Lu, Hu, and Clune propose letting one AI model systematically probe what another can and cannot do, in a 2025 paper. Human-made tests take growing effort as models improve, and they cover only a fraction of possible abilities. Their framework assigns a foundation model the role of scientist, inventing open-ended tasks for a subject model to attempt.

Across the GPT, Claude, and Llama families, the system generated thousands of distinct tasks. Grouping them revealed dozens of broader skill areas and failure modes that a single team would struggle to find. Human surveys agreed closely with the automatic scoring. The code and logs are openly released.

Catalog record

CREATORCong Lu, Shengran Hu, Jeff Clune, * Equal Contribution University of British Columbia
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE117 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-07577

Browse the full library · About the Tower of Babel