The Kinetics of Reasoning: How Chain-of-Thought Shapes Learning in Transformers?
Zihan Pengmei, Costas Mavromatis, Zhengyuan Shen, Yunyi Zhang, Vassilis N. Ioannidis, Huzefa Rangwala · 2025
Read The Kinetics of Reasoning: How Chain-of-Thought Shapes Learning in Transformers? by Zihan Pengmei, Costas Mavromatis, Zhengyuan Shen, Yunyi Zhang, Vassilis N. Ioannidis, Huzefa Rangwala (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-25791.txt.
Description
Zihan Pengmei led five co-authors studying thinking kinetics. Costas Mavromatis, Zhengyuan Shen, and Yunyi Zhang helped. Vassilis N Ioannidis and Huzefa Rangwala complete six. Their 2025 paper asks how chain-of-thought shapes transformer learning. Chain-of-thought supervision can improve transformer results largely. Mechanisms by which models benefit remain poorly understood. They study learning changes through grokking phenomena.
Transformers were pretrained on symbolic thinking tasks. Algorithmic hardness was tunable in their setup. Data composition was controllable for systematic study. Two settings compare final answers versus explicit traces. Chain-of-thought generally improves task results, they find. Benefits depend on task hardness systematically. A three-setting logistic curve models quality growth. Learning speed and shape vary with hardness. A transient trace unfaithfulness phase is uncovered. Early models answer correctly while contradicting steps. Later, traces align with answers. Chain-of-thought accelerates generalization but not hardest tasks.
Catalog record
| CREATOR | Zihan Pengmei, Costas Mavromatis, Zhengyuan Shen, Yunyi Zhang, Vassilis N. Ioannidis, Huzefa Rangwala |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-25791