Self-Improvement in Language Models: The Sharpening Mechanism
Audrey Huang [email protected] Adam Block [email protected] Dylan J. Foster [email protected] Dhruv Rohatgi [email protected] and Cyril Zhang [email protected] Max Simchowitz [email protected] Jordan T. Ash [email protected] Akshay Krishnamurthy akshaykr@microsof · 2024
Read Self-Improvement in Language Models: The Sharpening Mechanism by Audrey Huang [email protected] Adam Block [email protected] Dylan J. Foster [email protected] Dhruv Rohatgi [email protected] and Cyril Zhang [email protected] Max Simchowitz [email protected] Jordan T. Ash [email protected] Akshay Krishnamurthy akshaykr@microsof (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01951.txt.
Description
Audrey Huang and seven co-authors explained in 2024 how language models can improve their own answers without outside help. A model cannot create knowledge from nothing, yet picking its better drafts seems to lift quality. The authors name the effect sharpening: the model judges better than it generates, so training on its own best outputs concentrates skill.
Self-judgment during training squeezes costly trial-and-error into the model itself. The account gives theory for when self-improvement works and when it stalls. It also guides builders spending compute on self-training rather than fresh data.
Catalog record
| CREATOR | Audrey Huang [email protected] Adam Block [email protected] Dylan J. Foster [email protected] Dhruv Rohatgi [email protected] and Cyril Zhang [email protected] Max Simchowitz [email protected] Jordan T. Ash [email protected] Akshay Krishnamurthy akshaykr@microsof |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 180 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-01951