The Kinetics of Reasoning: How Chain-of-Thought Shapes Learning in Transformers?

Zihan Pengmei, Costas Mavromatis, Zhengyuan Shen, Yunyi Zhang, Vassilis N. Ioannidis, Huzefa Rangwala · 2025

Read The Kinetics of Reasoning: How Chain-of-Thought Shapes Learning in Transformers? by Zihan Pengmei, Costas Mavromatis, Zhengyuan Shen, Yunyi Zhang, Vassilis N. Ioannidis, Huzefa Rangwala (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-25791.txt.

Description

Zihan Pengmei led five co-authors studying thinking kinetics. Costas Mavromatis, Zhengyuan Shen, and Yunyi Zhang helped. Vassilis N Ioannidis and Huzefa Rangwala complete six. Their 2025 paper asks how chain-of-thought shapes transformer learning. Chain-of-thought supervision can improve transformer results largely. Mechanisms by which models benefit remain poorly understood. They study learning changes through grokking phenomena.

Transformers were pretrained on symbolic thinking tasks. Algorithmic hardness was tunable in their setup. Data composition was controllable for systematic study. Two settings compare final answers versus explicit traces. Chain-of-thought generally improves task results, they find. Benefits depend on task hardness systematically. A three-setting logistic curve models quality growth. Learning speed and shape vary with hardness. A transient trace unfaithfulness phase is uncovered. Early models answer correctly while contradicting steps. Later, traces align with answers. Chain-of-thought accelerates generalization but not hardest tasks.

Catalog record

CREATORZihan Pengmei, Costas Mavromatis, Zhengyuan Shen, Yunyi Zhang, Vassilis N. Ioannidis, Huzefa Rangwala
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-25791

Browse the full library · About the Tower of Babel