Self-Improvement in Language Models: The Sharpening Mechanism

Audrey Huang [email protected] Adam Block [email protected] Dylan J. Foster [email protected] Dhruv Rohatgi [email protected] and Cyril Zhang [email protected] Max Simchowitz [email protected] Jordan T. Ash [email protected] Akshay Krishnamurthy akshaykr@microsof · 2024

Read Self-Improvement in Language Models: The Sharpening Mechanism by Audrey Huang [email protected] Adam Block [email protected] Dylan J. Foster [email protected] Dhruv Rohatgi [email protected] and Cyril Zhang [email protected] Max Simchowitz [email protected] Jordan T. Ash [email protected] Akshay Krishnamurthy akshaykr@microsof (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01951.txt.

Description

Audrey Huang and seven co-authors explained in 2024 how language models can improve their own answers without outside help. A model cannot create knowledge from nothing, yet picking its better drafts seems to lift quality. The authors name the effect sharpening: the model judges better than it generates, so training on its own best outputs concentrates skill.

Self-judgment during training squeezes costly trial-and-error into the model itself. The account gives theory for when self-improvement works and when it stalls. It also guides builders spending compute on self-training rather than fresh data.

Catalog record

CREATORAudrey Huang [email protected] Adam Block [email protected] Dylan J. Foster [email protected] Dhruv Rohatgi [email protected] and Cyril Zhang [email protected] Max Simchowitz [email protected] Jordan T. Ash [email protected] Akshay Krishnamurthy akshaykr@microsof
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE180 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-01951

Browse the full library · About the Tower of Babel