General Scales Unlock AI Evaluation with Explanatory and Predictive Power
Lexin Zhou 1,2,3 Lorenzo Pacchiardi 1 Fernando Martínez-Plumed 3 Katherine M. Collins 4 Yael Moros-Daval 3 Seraphina Zhang 1,5 Qinlin Zhao 2 Yitian Huang 2 Luning Sun 6 Jonathan E. Prunty 1 Zongqian Li 7 Pablo Sánchez-García 8 Kexin Jiang Chen 3 Pablo A. M. Casares 3 Jiyun Zu 9 John Burden 1 Behzad · 2025
Read General Scales Unlock AI Evaluation with Explanatory and Predictive Power by Lexin Zhou 1,2,3 Lorenzo Pacchiardi 1 Fernando Martínez-Plumed 3 Katherine M. Collins 4 Yael Moros-Daval 3 Seraphina Zhang 1,5 Qinlin Zhao 2 Yitian Huang 2 Luning Sun 6 Jonathan E. Prunty 1 Zongqian Li 7 Pablo Sánchez-García 8 Kexin Jiang Chen 3 Pablo A. M. Casares 3 Jiyun Zu 9 John Burden 1 Behzad (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-06378.txt.
Description
Zhou and colleagues proposed a new way in twenty twenty-five to evaluate artificial intelligence systems, built on general scales rather than fixed test scores. Benchmarks drive progress, but a score on one test says little about what a system can do on a fresh task.
Their method rates every test item by the demands it places on eighteen general ability scales, from knowledge to reasoning. Fifteen large language models were profiled across sixty-three tasks. The profiles explained what each benchmark truly measures and exposed how skills shift with model size and thinking style. Better still, demand ratings predicted a model's success on brand-new items, beating standard black-box predictors, above all on unfamiliar tasks. The authors call it a step toward evaluation that explains and forecasts, not just ranks.
Catalog record
| CREATOR | Lexin Zhou 1,2,3 Lorenzo Pacchiardi 1 Fernando Martínez-Plumed 3 Katherine M. Collins 4 Yael Moros-Daval 3 Seraphina Zhang 1,5 Qinlin Zhao 2 Yitian Huang 2 Luning Sun 6 Jonathan E. Prunty 1 Zongqian Li 7 Pablo Sánchez-García 8 Kexin Jiang Chen 3 Pablo A. M. Casares 3 Jiyun Zu 9 John Burden 1 Behzad |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 286 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-06378