Predictable Emergent Abilities of LLMs: Proxy Tasks Are All You Need

Bo-Wen Zhang, Yan Yan, Boxiang Yang, Yifei Xue, Guang Liu · 2024

Read Predictable Emergent Abilities of LLMs: Proxy Tasks Are All You Need by Bo-Wen Zhang, Yan Yan, Boxiang Yang, Yifei Xue, Guang Liu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07111.txt.

Description

Bo-Wen Zhang, Yan Yan, Boxiang Yang, Yifei Xue, and Guang Liu forecast sudden new skills in large language models with stand-in tasks, in a paper from two thousand twenty-four. Scale rules tune training well but cannot foresee skills that small models never show. Stand-in tasks that track the target skill offer a detour.

Candidate tasks are first scored by how their results track the target across many models. Small model groups then vet the best stand-ins for trust. Scores from the chosen stand-ins combine into a forecast for the target. A tool-use case study showed forecasts that tracked real results closely. Readers who plan model training get a way to see jumps coming.

Catalog record

CREATORBo-Wen Zhang, Yan Yan, Boxiang Yang, Yifei Xue, Guang Liu
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-07111

Browse the full library · About the Tower of Babel