Reasoning Bias of Next Token Prediction Training
Pengxiao Lin, Zhongwang Zhang, Zhi-Qin John Xu · 2025
Read Reasoning Bias of Next Token Prediction Training by Pengxiao Lin, Zhongwang Zhang, Zhi-Qin John Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-02007.txt.
Description
Pengxiao Lin, Zhongwang Zhang, and Zhi-Qin John Xu compare two ways of training large language models to reason. The standard way, next token prediction, trains the model on every word of a text, noise included. A rival way, critical token prediction, trains only on key tokens such as the final answer. Many assumed the focused way would reason better.
The authors find the opposite. Models trained on every token reasoned better across a range of benchmark tasks, even though their training data held more noise. They argue the noise acts as a regularizer, keeping the model from leaning too hard on stray details. Such models also coped better with small disturbances and settled into flatter spots in the loss landscape. The focused method still has a place, the authors say, but at the fine-tuning stage rather than in pretraining.
Catalog record
| CREATOR | Pengxiao Lin, Zhongwang Zhang, Zhi-Qin John Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-02007