Consistency Is Not Always Correct: Towards Understanding the Role of Exploration in Post-Training Reasoning
Dake Bu, Wei Huang, Andi Han, Atsushi Nitanda, Bo Xue, Qingfu Zhang, Hau-San Wong, Taiji Suzuki · 2025
Read Consistency Is Not Always Correct: Towards Understanding the Role of Exploration in Post-Training Reasoning by Dake Bu, Wei Huang, Andi Han, Atsushi Nitanda, Bo Xue, Qingfu Zhang, Hau-San Wong, Taiji Suzuki (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-07368.txt.
Introduction
Dake Bu and co-authors, in work cataloged under a title on exploration and consistency, model how post-training reshapes reasoning. Foundation models know much but reason unevenly on set tasks. Reinforcement methods and test-time scaling help, yet they seem to strengthen old paths rather than find new ones.
Viewing reasoning steps as Markov moves helps explain the puzzle. Easy steps are likely moves, hard insights are rare moves, and training reweights their odds. Exploration helps by raising the chance that rare good moves get sampled and kept. The analysis reframes debates about consistency versus search. It guides choices about sampling budgets during training.
Catalog record
| CREATOR | Dake Bu, Wei Huang, Andi Han, Atsushi Nitanda, Bo Xue, Qingfu Zhang, Hau-San Wong, Taiji Suzuki |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 227 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-07368