EvaLearn: Quantifying the Learning Capability and Efficiency of LLMs via Sequential Problem Solving
2025
Read EvaLearn: Quantifying the Learning Capability and Efficiency of LLMs via Sequential Problem Solving (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02672.txt.
Description
Most AI benchmarks test what a model already knows, not how fast it learns. A large research team introduces EvaLearn, a benchmark of 648 hard problems in 182 sequences. Models must solve each sequence in order, so later problems can benefit from lessons learned on earlier ones.
Five automatic scores measure both learning power and efficiency. Nine top models showed wildly different profiles: some started middling but learned strongly, while others gained nothing or even got worse from experience. Strong static skill did not guarantee strong learning. Extra rubrics and teacher feedback helped models improve. The 2025 benchmark opens a new dimension for comparing language models.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-02672