Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pretrained Priors
Hao Yang, Zhiyu Yang, Yunjie Zhang3, Shanyi Zhu4, Lin Yang · 2025
Read Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pretrained Priors by Hao Yang, Zhiyu Yang, Yunjie Zhang3, Shanyi Zhu4, Lin Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01236.txt.
Introduction
Chain-of-thought prompting, where a model writes out steps before answering, boosts reasoning, but why it works is still debated. Hao Yang, with four co-authors, examines the balance between two forces: what the model learned in pre-training, its priors, and what the prompt itself teaches in context. They analyze the wording of reasoning traces in fine detail.
By adding noisy examples step by step, the team watched how the model shifts between prior knowledge and prompt signals. Models picked up surface structure fast, yet also grasped deeper logic, while leaning hard on priors at first. Enough good examples tipped decisions toward the context. Yang and the co-authors also tested whether prompt design can induce slower, more careful thinking, mapping how prompting shapes reasoning.
Catalog record
| CREATOR | Hao Yang, Zhiyu Yang, Yunjie Zhang3, Shanyi Zhu4, Lin Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-01236
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pretrained Priors Hao Yang, Zhiyu Yang, Yunjie Zhang3, Shanyi Zhu4, Lin Yang Abstract Chain-of-Thought reasoning has emerged as a pivotal methodology for enhancing model inference capabilities. Despite growing interest in Chain-of-Thought reasoning, its underlying mechanisms remain unclear. This paper explores the working mechanisms of Chain-of-Thought reasoning from the perspective of the dual relationship between in-context learning and pretrained priors. We first conduct a fine-grained lexical-level analysis of rationales to examine the model’s reasoning behavior. Then, by incrementally introducing noisy exemplars, we examine how the model balances pretrained priors against erroneous in-context information. Finally, we investigate whether prompt engineering can induce slow thinking in large language models. Our extensive experiments reveal three key findings: (1) The model not only quickly learns the reasoning structure at the lexical level but also grasps deeper logical reasoning patterns, yet it heavily relies on pretrained priors. (2) Providing sufficient exemplars shifts the model’s decision-making from pretrained priors to in-context signals, while misleading prompts introduce instability. (3) Long Chain-of-Thought prompting can induce the model to generate longer reasoning chains, thereby improving its performance on downstream tasks. 1 Introduction As models scale up, large language models (LLMs) exhibit emergent I n- C ontext L earning (ICL) capabilities [ 2 ] . ICL enables LLMs to perform tasks by leveraging a few exemplars presented as demonstrations. Relying on ICL, C hain- o f- T hought (CoT) prompting incorporates rationales into exemplars, guiding models to solve problems step by step and significantly improving performance across various downstream tasks [ 30 ] . The reasoning process in CoT prompting can be formalized as: \text{Input: }X=\{(q_{i},r_{i},a_{i})\}_{i=1}^{N},\;q Figure 1: Framework highlighting the synergy between Pretrained Priors and In-Context Learning in powering CoT reasoning. where (q_{i},r_{i},a_{i}) represents the i -th exemplar consisting of a question q_{i} , a rationale r_{i} , and an answer a_{i} ; N denotes the number of exemplars; and q is the target question. The model generates a response (r,a) by conditioning on the exemplars and the question: P(r,a\mid X,q)=P(r\mid X,q)\cdot P(a\mid X,q,r) CoT prompting has become an essential technique for enhancing model reasoning capabilities. Previous studies have extensively explored how CoT affects reasoning performance, focusing on factors such as difficulty, step length, the number of exemplars, and order [ 4 , 8 , 30 , 32 ] . However, the underlying mechanism of CoT prompting remains inconclusive. Inspired by [ 14 , 20 ] , this paper explores the dual relationship between ICL and pretrained priors in CoT reasoning. In studies on the mechanism of ICL, [ 18 ] maintained that models primarily relied on pretrained priors during inference without acquiring new task-specific knowledge. This view was reinforced by [ 23 , 10 ] , who argued that LLMs struggle to overcome their pretrained preferences. In contrast, [ 20 , 31 ] demonstrated that model scaling enables the learning of novel input-label mappings, although this capability emerged only in larger models (exceeding 66B parameters) rather than in smaller ones. Regarding CoT mechanisms, current research presents divergent perspectives: [ 22 ] proposed that covariates in the prompts other than logical reasoning may be responsible for the performance improvements, while [ 16 ] attributed its effectiveness to task simplification achieved by decomposing complex problems. Furthermore, [ 15 ] suggested that models primarily imitated the CoT format through pattern recognition from exemplars, and [ 17 ] provided theoretical analyses of how CoT amplifies models’ computational capabilities. Although these hypotheses require further empirical validation, they collectively advance our understanding of CoT’s fundamental mechanisms and offer valuable directions for future research. As shown in Fig. 1 , we investigate the interaction between ICL and pretrained priors in CoT reasoning. Our study addresses three key questions: (1) What do LLMs learn from CoT exemplars through ICL? (2) Can LLMs override pretrained priors through ICL? (3) Can prompt engineering leverage both ICL and pretrained knowledge to elicit slow thinking? By disentangling the roles of ICL and pretrained priors, our findings shed light on the mechanisms behind reasoning emergence in LLMs, offering both theoretical insights into model cognition and practical implications for designing more effective prompts. To answer these questions, we designed a series of experiments analyzing how LLMs integrate CoT exemplars and pretrained priors in decision-making. First, we provided models with both task-specific and task-agnostic exemplars to examine how variations in input influenced reasoning behavior. Through a fine-grained input-output analysis at the lexicon level, we quantified the reliance of LLMs on ICL signals versus pretrained priors. To assess robustness, we introduced controlled noise into exemplars and progressively increased the number of noisy samples to observe model behavior under misleading information. These previous experimental results lead to the third research question: given both ICL signals and pretrained priors, can prompt engineering guide models to generate longer CoT rationales? To test this, we applied prompts that encouraged slow thinking, aiming to generate extended CoT rationales and improve downstream performance. Our findings revealed three key insights: • LLMs learn reasoning patterns from in-context exemplars, while CoT reasoning still remains influenced by pretrained semantic priors. • Smaller models (8B) can map inputs to outputs via CoT prompting, with an increased number of exemplars shifting reliance from pretrained priors to ICL signals. Additionally, low-quality exemplars increase instability. • Prompt engineering can induce longer CoT outputs, improving downstream performance and suggesting a path toward model self-evolution. 2 Experimental Setup 2.1 Evaluation Tasks Given CoT’s pronounced impact on reasoning tasks, we focus on arithmetic, commonsense, and symbolic reasoning, following [ 4 , 8 , 16 , 30 ] . For arithmetic reasoning, we employed GSM8K [ 5 ] and MATH-500 [ 13 ] , two widely adopted benchmarks in mathematical reasoning. For commonsense reasoning, we used Date Understanding [ 1 ] , which features compositional questions designed to assess commonsense. For the symbolic reasoning task, we considered two tasks: the Coin Flip task [ 30 ] and the Last Letters Concatenation (four words) dataset [ 9 ] . The Coin Flip task assesses the ability to reason about probabilistic outcomes, while the Last Letters Concatenation dataset evaluates the capacity to manipulate and reason about symbolic representations. 2.2 Models & Prompts Models. We conducted experiments using publicly available pretrained models of various series and sizes, including Gemma2-9B, Gemma2-27B [ 25 ] , LLaMA3.1-8B [ 26 ] , and Qwen2.5-32B [ 27 ] . Following [ 30 ] , we applied greedy decoding. Due to space constraints, only a subset of the results is presented in the main text. Prompts. We employed standard and few-shot CoT exemplars from [ 30 ] and a zero-shot CoT instruction (“ Let’s think step by step. ”) from [ 9 ] . Each dataset is associated with a task-specific CoT prompt. When a CoT prompt designed for one task is applied to a different task, it is referred to as a task-agnostic prompt. For example, in Section 3.1 , we utilized task-agnostic prompts (e.g., Sports-CoT, Date-CoT, etc.) on the GSM8K dataset. 3 Experiment 3.1 RQ1: What do LLMs Learn from CoT Exemplars Through ICL? This study compares CoT reasoning acquired through pretraining with that learned from exemplars through ICL, revealing what models extract from exemplars and the characteristics of