Consistency Is Not Always Correct: Towards Understanding the Role of Exploration in Post-Training Reasoning

Dake Bu, Wei Huang, Andi Han, Atsushi Nitanda, Bo Xue, Qingfu Zhang, Hau-San Wong, Taiji Suzuki · 2025

Read Consistency Is Not Always Correct: Towards Understanding the Role of Exploration in Post-Training Reasoning by Dake Bu, Wei Huang, Andi Han, Atsushi Nitanda, Bo Xue, Qingfu Zhang, Hau-San Wong, Taiji Suzuki (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-07368.txt.

Introduction

Dake Bu and co-authors, in work cataloged under a title on exploration and consistency, model how post-training reshapes reasoning. Foundation models know much but reason unevenly on set tasks. Reinforcement methods and test-time scaling help, yet they seem to strengthen old paths rather than find new ones.

Viewing reasoning steps as Markov moves helps explain the puzzle. Easy steps are likely moves, hard insights are rare moves, and training reweights their odds. Exploration helps by raising the chance that rare good moves get sampled and kept. The analysis reframes debates about consistency versus search. It guides choices about sampling budgets during training.

Catalog record

CREATORDake Bu, Wei Huang, Andi Han, Atsushi Nitanda, Bo Xue, Qingfu Zhang, Hau-San Wong, Taiji Suzuki
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE227 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-07368

Browse the full library · About the Tower of Babel