How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation
Hao Yang, Qinghua Zhao, Lei Li · 2025
Read How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation by Hao Yang, Qinghua Zhao, Lei Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-20758.txt.
Introduction
Asking a language model to think step by step, the chain-of-thought trick, plainly improves its answers, but why it works inside the network is murky. Hao Yang and co-authors trace the flow of information backward through the model's decoding, projection, and activation stages. They look for what the written reasoning actually changes.
One finding casts the reasoning text as a pruning device: it narrows the space of likely outputs by leaning on answer templates. The closer the output hugs the template, the better the score. A second finding surprised them: the trick quiets neuron activity on open-ended tasks yet stirs more activity on closed ones. The authors frame these results as handles for designing sharper prompts on purpose, rather than by folklore. Code and data are released.
Catalog record
| CREATOR | Hao Yang, Qinghua Zhao, Lei Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-20758
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation Hao Yang, Qinghua Zhao, Lei Li Abstract Chain-of-Thought (CoT) prompting significantly enhances model reasoning, yet its internal mechanisms remain poorly understood. We analyze CoT’s operational principles by reversely tracing information flow across decoding, projection, and activation phases. Our quantitative analysis suggests that CoT may serve as a decoding space pruner, leveraging answer templates to guide output generation, with higher template adherence strongly correlating with improved performance. Furthermore, we surprisingly find that CoT modulates neuron engagement in a task-dependent manner: reducing neuron activation in open-domain tasks, yet increasing it in closed-domain scenarios. These findings offer a novel mechanistic interpretability framework and critical insights for enabling targeted CoT interventions to design more efficient and robust prompts. We released our code and data at https://anonymous.4open.science/r/cot-D247 . 1 Introduction Large language models (LLMs) excel in diverse tasks but falter in multi-step reasoning. Chain-of-Thought (CoT) prompting, introduced by ( Wei et al. 2022 ) , mitigates this by guiding models through step-by-step reasoning, markedly improving performance in arithmetic, commonsense, and symbolic reasoning tasks ( Wang et al. 2023 ; Jin et al. 2024 ; Prabhakar et al. 2024 ; Nayab et al. 2025 ) . Despite its effectiveness, CoT’s underlying mechanisms remain poorly understood. Prior studies propose hypotheses for CoT’s efficacy. For instance, Madaan et al. 2023 argue that CoT reduces task complexity, making tasks more manageable for models. Similarly, Madaan and Yazdanbakhsh 2022 propose that models imitate answer templates in CoT prompts, and Schaeffer et al. 2023 suggest that prompt features unrelated to logical reasoning drive performance gains. Although these insights are intuitively compelling, they lack direct experimental support linking models’ internal states to observed outcomes. This work advances our understanding of CoT’s mechanisms by adopting a mechanistic interpretability approach ( Dumas et al. 2024 ; Lee et al. 2024 ) . Mechanistic interpretability aims to “open” the model’s “black box”, investigating how its internal “ parts ” (computational units) and “ wiring ” (connection units) enable information to be sequentially processed and flow. It is crucial for CoT that significantly alters model behavior. Consistent with prior work ( Chia et al. 2023 ; Madaan et al. 2023 ; Jin et al. 2024 ) , we focus on vanilla CoT ( Wei et al. 2022 ) , as its internal mechanisms remain underexplored. Figure 7 shows our framework. To investigate CoT’s mechanisms, we select six models (ranging from 3B to 70B) and nine datasets spanning three core reasoning types: arithmetic, commonsense, and symbolic reasoning, covering a wide range of task complexities, input formats, and answer spaces (e.g., open-domain GSM8K and closed-domain AQuA). Our multi-faceted analysis yields several key mechanistic insights: \circ We propose a novel mechanism by which CoT may constrain the decoding space by leveraging answer templates. While previous work has hinted at the importance of prompt structure, we quantify this by demonstrating a strong correlation between reasoning structure adherence and performance. \diamond Our analysis demonstrates that CoT yields concentrated probability distributions, potentially minimizing prediction uncertainty, providing new evidence for CoT’s role in enhancing model confidence. \star We identify a task-dependent modulation of neuron engagement by CoT, which may vary across datasets: reducing activation in open-domain tasks, while unexpectedly increasing it in closed-domain tasks, a phenomenon not previously reported . These findings provide a more nuanced understanding of CoT’s mechanisms and offer a novel mechanistic interpretability and critical insights for designing more efficient and robust prompts. 2 Related Work This section reviews mechanistic interpretability for LLM analysis and CoT prompting with its influencing factors. We then position our work, which applies mechanistic interpretability to investigate CoT’s internal mechanisms. Mechanistic Interpretability Mechanistic interpretability endeavors to reverse-engineer neural network computations. Unlike approaches focused solely on input-output correlations, it constructs mappings from inputs via internal states to outputs through an analysis of the model’s internal structure ( Nanda et al. 2023 ) . Specific components have been investigated. Transformer feed-forward networks (FFNs), for instance, are interpreted as key-value memories linking textual patterns to output distributions ( Geva et al. 2021b ) . Studies also characterize neuron properties and functions, such as identifying universal neurons ( Gurnee et al. 2024 ) , analyzing activation patterns of reasoning neurons in FFNs ( Rai and Yao 2024 ) , and investigating activation sparsity ( Voita et al. 2024 ) . Other mechanistic interpretability approaches analyze the flow and representation of information through intermediate layers. Techniques include patching intermediate representations to localize computations ( Fierro et al. 2025 ) , tracking the trajectory of embeddings through input, concept, and output spaces ( Wendler et al. 2024 ) , and extending methods like the logit lens on vocabulary embeddings ( Cancedda 2024 ) . Collectively, these methods offer tools to dissect LLM internal processing and investigate mechanisms. Task Dataset (Answer Space) Example GSM8K / SVAMP (Open, Numerical) Question : If there are 3 cars in the parking lot and 2 more cars arrive, how many cars are in the parking? Answer : There are originally 3 cars. 2 more cars arrive. 3+2=5 . So the answer is 5 . Arithmetic AQuA (Closed, Options) Question : A person is traveling at 20 km/hr and reached his destiny in 2.5 hr then find the distance? Choices : (a) 53 km (b) 55 km (c) 52 km (d) 60 km (e) 50 km Answer : The distance that the person traveled would have been 20\,\text{km/hr}\times 2.5\,\text{hrs}=50\,\text{km} . So the answer is (e) . Bamboogle (Open, Text) Question : Who lived longer, Theodor Haecker or Harry Vaughan Watkins? Answer : Theodor Haecker was 65 years old when he died. Harry Vaughan Watkins was 69 years old when he died. So the answer is Harry Vaughan Watkins . Common sense StrategyQA / Sports (Binary, Yes/No) Question : Do hamsters provide food for any animals? Answer : Hamsters are prey animals. Prey are food for predators. Thus, hamsters provide food for some animals. So the answer is Yes . Date (Formatted) Question : The concert was scheduled to be on 06/01/1943, but was delayed by one day to today. What is the date 10 days ago in MM/DD/YYYY? Answer : One day after 06/01/1943 is 06/02/1943, so today is 06/02/1943. 10 days before today is 05/23/1943. So the answer is 05/23/1943 . Coin Flip (Binary, Yes/No) Question : A coin is heads up. Maybelle flips the coin. Shalonda does not flip the coin. Is the coin still heads up? Answer : The coin was flipped by Maybelle. So the coin was flipped 1 time, which is an odd number. The coin started heads up, so after an odd number of flips, it will be tails up. So the answer is No . Symbolic Last Letters (Open, Text) Question : Take the last letters of each words in “Lace y Nor a Debr a Ashleig h ” and concatenate them. Answer : The answer is yaah . Table 1: Part Dataset Examples (Full examples is in Appendix B ). Chain-of-Thought CoT prompting enhances LLM reasoning by generating intermediate steps before the final answer ( Wei et al. 2022 ) . This is effective across diverse reasoning tasks ( Tanneru et al. 2024 ) and leads to numerous