Predictable Emergent Abilities of LLMs: Proxy Tasks Are All You Need
Bo-Wen Zhang, Yan Yan, Boxiang Yang, Yifei Xue, Guang Liu · 2024
Read Predictable Emergent Abilities of LLMs: Proxy Tasks Are All You Need by Bo-Wen Zhang, Yan Yan, Boxiang Yang, Yifei Xue, Guang Liu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07111.txt.
Description
Bo-Wen Zhang, Yan Yan, Boxiang Yang, Yifei Xue, and Guang Liu forecast sudden new skills in large language models with stand-in tasks, in a paper from two thousand twenty-four. Scale rules tune training well but cannot foresee skills that small models never show. Stand-in tasks that track the target skill offer a detour.
Candidate tasks are first scored by how their results track the target across many models. Small model groups then vet the best stand-ins for trust. Scores from the chosen stand-ins combine into a forecast for the target. A tool-use case study showed forecasts that tracked real results closely. Readers who plan model training get a way to see jumps coming.
Catalog record
| CREATOR | Bo-Wen Zhang, Yan Yan, Boxiang Yang, Yifei Xue, Guang Liu |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-07111