Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models

Siyuan Zhang, Jialian Li, Yichi Zhang, Xiao Yang, Yinpeng Dong, Hang Su · 2026

Read Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models by Siyuan Zhang, Jialian Li, Yichi Zhang, Xiao Yang, Yinpeng Dong, Hang Su (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00770.txt.

Description

Siyuan Zhang and colleagues study how reasoning evolves during training in large language models, looking inside the model rather than only at generation outcomes. Across models at various training stages, post-training improves static initial representation quality only slightly. Reasoning, unlike other tasks, involves a large continuous shift in representations during generation. Post-training helps drive that shift toward a distribution better suited to solving tasks.

Catalog record

CREATORSiyuan Zhang, Jialian Li, Yichi Zhang, Xiao Yang, Yinpeng Dong, Hang Su
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66748
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2602.00770

Browse the full library · About the Tower of Babel