CoT-Space: A Theoretical Framework for Internal Slow-Thinking via Reinforcement Learning

Zeyu Gan, Hao Yi, Yong Liu · 2025

Read CoT-Space: A Theoretical Framework for Internal Slow-Thinking via Reinforcement Learning by Zeyu Gan, Hao Yi, Yong Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-04027.txt.

Description

Zeyu Gan and two co-authors' 2025 research paper explains why the length of chain-of-thought reasoning settles at an optimum. Modern language models reason better with long chains of thought, and reinforcement learning tunes how long those chains grow. Empirically the length converges instead of expanding forever, but token-level analysis cannot explain the pattern.

Their framework, CoT-Space, recasts reasoning as optimisation inside a continuous semantic space at the level of reasoning steps. The trajectory is modelled through two lenses, noise and risk, reviving classical learning theory. Convergence to an optimal chain length then falls out naturally from a fundamental tension: shorter chains underfit the problem, longer chains overfit it. Reinforcement learning elicits and verifies the predictions experimentally. The findings explain test-time scaling through reinforcement learning. Practitioners gain a foundation for shaping reasoning trajectories instead of tuning lengths by trial and error.

Catalog record

CREATORZeyu Gan, Hao Yi, Yong Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-04027

Browse the full library · About the Tower of Babel