Automated Capability Discovery via Foundation Model Self-Exploration
Cong Lu, Shengran Hu, Jeff Clune, * Equal Contribution University of British Columbia · 2025
Read Automated Capability Discovery via Foundation Model Self-Exploration by Cong Lu, Shengran Hu, Jeff Clune, * Equal Contribution University of British Columbia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-07577.txt.
Description
Lu, Hu, and Clune propose letting one AI model systematically probe what another can and cannot do, in a 2025 paper. Human-made tests take growing effort as models improve, and they cover only a fraction of possible abilities. Their framework assigns a foundation model the role of scientist, inventing open-ended tasks for a subject model to attempt.
Across the GPT, Claude, and Llama families, the system generated thousands of distinct tasks. Grouping them revealed dozens of broader skill areas and failure modes that a single team would struggle to find. Human surveys agreed closely with the automatic scoring. The code and logs are openly released.
Catalog record
| CREATOR | Cong Lu, Shengran Hu, Jeff Clune, * Equal Contribution University of British Columbia |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 117 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-07577