Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process

Zhenyu Zhang, Shujian Zhang, John Lambert, Wenxuan Zhou, Zhangyang Wang, Mingqing Chen, Andrew Hard, Rajiv Mathews, Lun Wang · 2025

Read Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process by Zhenyu Zhang, Shujian Zhang, John Lambert, Wenxuan Zhou, Zhangyang Wang, Mingqing Chen, Andrew Hard, Rajiv Mathews, Lun Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-23988.txt.

Description

Zhenyu Zhang and eight co-authors go looking for the gears of machine reasoning without deciding in advance what they should be called. Researchers studying how language models reason usually start from human labels such as overthinking or reflection, defined at the word level. The space of possible behaviors is larger than anyone's vocabulary for it. The team's framework, RISE, discovers behaviors directly in the model's activation space.

They slice chain-of-thought traces into sentence-sized steps and train sparse autoencoders on the activations of those steps. The learned features sort out into readable behaviors including reflection and backtracking, occupying separate regions of the decoder space. Nudging a feature up or down controllably amplifies or suppresses its behavior during inference. The autoencoders also exposed structural signals such as response length, and confidence-related directions that let the authors steer how confident a model sounds.

Catalog record

CREATORZhenyu Zhang, Shujian Zhang, John Lambert, Wenxuan Zhou, Zhangyang Wang, Mingqing Chen, Andrew Hard, Rajiv Mathews, Lun Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-23988

Browse the full library · About the Tower of Babel