Decoding the Critique Mechanism in Large Reasoning Models Hoang Phan, Quang H. Nguyen, Hung T. Q. Le, Xiusi Chen, Heng Ji, Khoa D. Doan Abstract Large Reasoning Models (LRMs) exhibit backtracking and self-verification mechanisms that enable them to revise intermediate steps and reach correct solutions, yielding strong performance on complex logical benchmarks. We hypothesize that such behaviors are beneficial only when the model has sufficiently strong “critique” ability to detect its own mistakes. This work systematically investigates how current LRMs recover from errors by inserting arithmetic mistakes in their intermediate reasoning steps. Notably, we discover a peculiar yet important phenomenon: despite the error propagating through the chain-of-thought (CoT), resulting in an incorrect intermediate conclusion, the model still reaches the correct final answer. This recovery implies that the model must possess an internal mechanism to detect errors and trigger self-correction, which we refer to as the hidden critique ability . Building on feature space analysis, we identify a highly interpretable critique vector representing this behavior. Extensive experiments across multiple model scales and families demonstrate that steering latent representations with this vector improves the model’s error detection capability and enhances the performance of test-time scaling at no extra training cost. Our findings provide a valuable understanding of LRMs’ critique behavior, suggesting a promising direction to control and improve their self-verification mechanism. Our code is available at mail-research/lrm-critique-vectors . Figure 1 : Hidden self-correction despite erroneous reasoning in R1-32B. Left: original correct reasoning. Right: injected error (3 + 4 = 6 ) propagates to incorrect thinking conclusion ( $20 ), yet the model recovers to the correct final answer ( $18 ). We hypothesize this indicates implicit error detection beyond the observable CoT. Full generation details are in Appendix E.1 . 1 Introduction The recent wave of large reasoning models (LRMs)—including the o1 series ( 26 ) , DeepSeek-R1 ( 11 ) , and Qwen3 ( 41 ) —has substantially raised the performance ceiling on complex, multi-step reasoning benchmarks. Recent studies ( 11 ; 43 ) show that these models exhibit human-like “aha moments”, including backtracking and self-verification (e.g., “Wait, I made a mistake!”), resulting in non-linear traces rather than traditional linear CoT and leading to strong performance on complex logical benchmarks. Yet, many “aha moments” and backtracking steps are redundant or illusory, leading to overthinking: models produce excessive tokens, enter self-doubt loops, and often degrade performance due to a lack of critical thinking ( 6 ; 9 ; 49 ) . This suggests that while the model possesses some “critique” ability to detect its own mistakes, this ability is still intrinsically weak, underscoring the need for methods that enable the models to reliably detect genuine errors, execute precise self-correction, and eliminate decorative reasoning steps. Throughout this paper, we will use the terms critique , self-critique , and self-correction interchangeably to refer the ability of LRMs to detect and rectify errors. Recent work on test-time scaling increases inference compute to enhance reasoning performance, rather than scaling to larger parameters. For instance, s1 ( 24 ) shows that minimal fine-tuning with budget forcing, appending “Wait” tokens during inference, yields significant gains, while compute-optimal strategies enable smaller models to outperform larger ones ( 30 ) . However, test-time scaling does not consistently improve performance ( 10 ; 46 ) , as its effectiveness depends on the model’s intrinsic self-verification capacity ( 29 ; 46 ) . A key bottleneck is: LLMs struggle to detect errors in long CoTs ( 12 ) , fail to self-correct without external guidance ( 14 ) , and cannot reliably identify their own mistakes ( 35 ) . While training-based methods improve verification capabilities ( 38 ; 15 ; 4 ; 20 ) , they require substantial training costs, architectural complexity, or external tools, reducing practicality. We hypothesize that the budget forcing approach relies heavily on the error detection and correction ability, and investigate whether this ability can be identified and controlled without additional training. Understanding how LRMs handle errors internally requires samples where the model makes mistakes and performs self-correction. However, we observe that such instances are rare in standard generation.When the model answers correctly, CoTs are typically all correct; when it is wrong, errors persist throughout with no clear recovery. Moreover, labeling every step is prohibitively expensive. Instead, we deliberately inject arithmetic errors (e.g., 3+4=6 ) at intermediate steps within the reasoning trace and let the model continue without terminating its thinking process (Figure 1 ). We then compare the thinking conclusions against the ground-truth answers to assess its recovery, yielding a scalable analysis framework without per-step supervision. Surprisingly, we observe that despite the corrupted reasoning from the injected errors, the model often recovers and produces correct final answers (Figure 2 ). We call it the hidden critique ability , where LRMs hiddenly rectify errors without verbalization in chain of thoughts. To understand this recovery, we analyze the feature space and observe a strong separability between the error-recovery (with intervention) runs and the standard (without intervention) runs, although both yield correct final answers. We hypothesize that this separability captures an internal critique mechanism, i.e., the model’s hidden ability to detect and correct errors without explicit verbalization. This hidden capability can be represented as a single linear direction in the activation space – which is exactly the difference between the two mean vectors of activations of the intervened and standard runs. We call it the critique vector and validate its semantic meaning with logit lens analysis across model families. To further evaluate our hypothesis and test the utility of the critique vector, we use it to steer the LRMs and evaluate the performance on several error-detection tasks. Experimental results show that positive steering nudges LRMs’ behavior to become more “critique” and improves its mistake-identification performance, while negative steering worsens it. Finally, we show that controlling LRMs with this critique vector during test-time scaling enables the model to detect its mistakes better and produce more correct solutions. In summary, our contributions are as follows: • We are the first that uncover the hidden self-correction phenomenon in LRMs: injecting a simple arithmetic error into the CoT corrupts reasoning and intermediate conclusions, yet the model implicitly recovers and produces the correct final response. • Through feature space analysis, we show that activations are highly separable between the error-recovery runs and the standard runs. Leveraging this separation, we extract a critique vector and rigorously validate its error-detection capability through logit lens analysis and steering experiments. • Finally, we demonstrate that the critique vector influences test-time scaling: increasing it helps the model better recognize its mistakes and improve its final accuracy, while decreasing it degrades performance. 2 Related Work Representation Engineering in LLMs. Representation engineering extracts vectors from model activations to steer behavior without fine-tuning. Early work ( 28 ; 33 ) computed steering vectors from contrastive prompt pairs, enabling applications in style control ( 16 ) and contrastive steering ( 28 ) , later extended to refusal control ( 19 ; 2 ) and improved instruction following ( 31 ) . Recent advances address polysemanticity through feature-level steering ( 42 ) and apply these techniques to large reasoning models (LRMs), enabling control over thinking budgets ( 21 ) , trace editing ( 32 ) , and reasoning calibration ( 5 ) . Unlike prior work steering style, length, or consistency, we are the first that discover the hidden critique ability and identify a critique vector that represents latent error detection, providing direct mechanistic access to the internal processes underlying successful test-time scaling. Faithfulness of Chain-of-Thought Reasoning. Recent studies reveal that reasoning traces in LRMs are often unfaithful to the model’s actual internal computations. For example, models may fail to verbalize their true reasoning processes ( 7 ; 34 ) , generate misleading explanations ( 1 ) , or produce decorative “aha moments” that do not reflect genuine problem-solving ( 49 ) . Our work observes a related phenomenon, where models generate correct final answers despite corrupted reasoning, further evidencing the mismatch between verbalized and internal computation. However, we go beyond observation by locating this capability in latent space, interpreting it as a critique vector, and using it to control error detection and recovery behavior. Error Detection and Correction. Prior work reveals mixed capabilities in LLM error detection and self-correction. While self-verification improves reasoning ( 38 ) , fundamental limitations persist: LLMs struggle to detect errors in long CoT traces ( 12 ) , cannot reliably correct mistakes without external guidance ( 14 ) , and only succeed when error locations are provided ( 35 ) . Recent approaches address these gaps through key condition verification ( 40 ) , decomposition into confidence and critique components ( 45 ) , or reinforcement learning ( 17 ) . In reasoning models, self-verification emerges as a critical ability during training ( 11 ) , yet the underlying mechanisms remain underexplored. 18 examined its geometry in task-specific models, while 47 labeled reasoning steps using commercial models, then trained probes showing that models implicitly encode correctness information. In contrast, we focus on extracting the model’s intrinsic critique ability without external supervision, which enables mechanistic control over error detection across diverse logical reasoning tasks. Test-Time Scaling. Test-time scaling shifts focus from parameter scaling to allocating more computation during inference. Simple finetuning on reasoning traces and appending “Wait” tokens already yield notable gains ( 24 ) , while compute-optimal strategies allow smaller models to outperform much larger ones ( 30 ) . Follow-up work incorporates compute-optimal allocation ( 23 ) , self-verification ( 4 ) , tool-assisted verification ( 15 ) , and reward models to assess the reasoning quality ( 37 ; 48 ; 22 ) . However, recent studies reveal limitations to this potential, as extended thinking alone saturates and fails to scale beyond a certain point ( 10 ) . Additionally, LRMs show limited scaling capabilities without proper verification ( 46 ) , and scaling without verification or RL remains suboptimal ( 29 ) . We hypothesize that critique is the core mechanism underlying successful test-time scaling and demonstrate this by extracting critique vectors that enable direct control over the scaling process. 3 Uncovering Hidden Self-Correction 3.1 Intervention Framework LRMs typically operate in two phases: given the prompt q , they first generate a reasoning trace c between a special start-of-think token \langle\texttt{think}\rangle and an end-of-think token \langle\texttt{/think}\rangle , and then produce the final answer a . Formally: f_{\text{LRM}}:(q,\langle\texttt{think}\rangle)\mapsto(c,\langle\texttt{/think}\rangle,a). (1) Previous work has studied the model behavior under CoT interventions via prefilling ( 44 ; 39 ) , where a variable content e is integrated immediately after the thinking-start token. This generation process can be expressed as: f_{\text{LRM}}:(q,\langle\texttt{think}\rangle,e)\mapsto(\tilde{c},\langle\texttt{/think}\rangle,\tilde{a}). (2) Here, \tilde{c} represents the model’s reasoning continuation after intervention, and the complete CoT is formed by concatenating e+\tilde{c} . The intervention e is flexible and task-dependent; for instance, it may be an instruction, a prompt, or another mechanism designed to analyze specific model behaviors. A key question in our work is whether LRMs can detect and recover from errors during their internal reasoning. To investigate this, we inject an arithmetic error e (e.g., 3+4=6 ) into the reasoning chain and examine whether the continuation \tilde{c} recovers from it. We generate this error using GPT-5 with a carefully engineered prompt \mathcal{P}_{\text{err}} (detailed in Appendix A.1 ): f_{\texttt{GPT-5}}:(q,\mathcal{P}_{\text{err}})\mapsto e. (3) Using this approach, we construct GSM8K-Error and MATH500-Error by generating one error e per question q , creating a consistent benchmark across all models. For evaluation, we extract the thinking process \tilde{c} and the final answer a , then compare each against the ground truth y_{q} to determine whether the model can recover from the injected errors. Unlike step-by-step supervision methods ( 47 ) that require labeling each reasoning step, our approach is far more efficient and scalable. Full details on error generation process and correctness evaluation are in Appendix E.1 and Section 5.1 , respectively. 3.2 LRMs Hiddenly Recover in the Final Answer Figure 2 : Distribution of reasoning-result alignment across R1 and Qwen3 model variants. The proportions represent four distinct outcomes based on the correctness of the internal “thinking” process versus the correctness of the final “answer” on the GSM8K-Error and MATH500-Error benchmarks. In this paper, we analyze the self-correction mechanisms represented by the light blue segments: ‘ \times Think \checkmark Answer‘. We evaluate the error-recovery performance of four open-source models – DeepSeek-R1 (8B, 14B, 32B) and Qwen3 (4B), on GSM8K-Error and MATH500-Error. Details of these models are provided in Section 5.1 . Figure 2 outlines the proportions of model performance outcomes across various LRMs, categorized by the correctness between their internal reasoning processes (“Thinking”) and final output (“Answer”) for intervened datasets. Surprisingly, beyond the dominant cases whose reasoning and final answers align (i.e., either both are correct or both are incorrect), we observe a significant fraction of cases where the model’s visible reasoning remains corrupted, yet the final answer is correct. We hypothesize that in these cases, LRMs implicitly correct the injected mistake without verbalizing in the CoT. Specifically, on GSM8K-Error, R1 models consistently exhibit a hidden self-correction ability. In 50\% to 70\% of cases, they produce correct final answers, yet the injected errors remain intact in their reasoning trace. This implicit recovery persists across model scales and on the more challenging MATH500-Error ( 41–58\% ), while explicit corrections remain rare ( <6\% ). Unlike R1 models, the Qwen3-4B model demonstrates powerful explicit backtracking, recovering from the injected error in 37\%–41\% of GSM8K-Error cases and 78\% of MATH500-Error cases. However, there are still a moderate number of samples, 19\%–25\% on GSM8K-Error and 6\%–8\% on MATH500-Error, where Qwen3 models return the correct final answer without correcting the mistake in CoTs. This suggests that the hidden self-correction mechanism exists consistently across different model families. 4 The Geometry of Critique Ability Section 3 shows that LRMs possess a hidden critique ability to recover from injected errors. However, this behavior disappears in the original samples with natural CoTs. In this section, we compare these two cases to extract and analyze the critique ability using mechanistic interpretability tools. We observe linear separability in the latent space between the intervened (incorrect-CoT) and baseline (correct-CoT) runs in Section 4.2 . This linear separability suggests that the hidden capability can be decoded as a single direction in the activation space. We validate this hypothesis using Logit Lens ( 25 ) and extract steering vectors that capture the recovery mechanism. Table 1 : The distribution of the correctness of thinking processes and final answers on GSM8K and MATH500 datasets. Cases GSM8K MATH500 Qwen3-4B R1-8B R1-14B R1-32B Qwen3-4B R1-8B R1-14B R1-32B \times Thinking, \times Answer 3.9 % 15.6 % 6.8 % 5.0 % 1.5 % 5.0 % 4.1 % 2.4 % \checkmark Thinking, \checkmark Answer 95.8 % 81.8 % 92.4 % 94.0 % 98.3 % 95.0 % 95.5 % 97.4 % \checkmark Thinking, \times Answer 0.0 % 0.7 % 0.2 % 0.5 % 0.2 % 0.0 % 0.0 % 0.0 % \times Thinking, \checkmark Answer 0.2 % 1.9 % 0.5 % 0.5 % 0.0 % 0.0 % 0.4 % 0.2 % 4.1 The Behavior on Original Samples Section 3 shows that LRMs secretly detect the injected error and recover in the final answer. We investigate the behavior of LRMs on original samples and measure the accuracy of the thinking and the final answer separately. Table 1 demonstrates that the CoT and the final answer are usually consistent. When the thinking part is wrong, the model hardly fix it and thus returns the incorrect final answer. The recovery rate is only 1.9\% for R1-8B on GSM8k and less than 1\% in every other case. These results suggest that LRMs behave differently and do not invoke the critique behavior without the injected error. 4.2 The Linear Separability of Internal Thoughts To further investigate, we compare the latent features of intervened samples in Section 3 and original samples. Let c\sim y_{q} denote the answer in c , which could be the CoT or the final answer, aligns with the ground-truth y_{q} . We construct pairs of runs with identical prompts for 1,000 GSM8K training questions \mathcal{Q}_{\text{train}} : a baseline run with correct reasoning ( c\sim y_{q} ) and an intervened run with incorrect reasoning ( e+\tilde{c}\not\sim y_{q} ), both of which yield correct final answers ( a\sim y_{q},\tilde{a}\sim y_{q} ). As discussed above, in the final answer the model performs hidden critique ability to recover from mistakes. Therefore, we inspect the hidden states of the final answers and extract the average feature vectors across answer tokens at each layer \ell : \displaystyle h_{\text{baseline}}^{(\ell)}(q) \displaystyle=\frac{1}{|a|}\sum_{i\in a}h_{i}^{(\ell)}(q,c,a), (4) \displaystyle h_{\text{intervened}}^{(\ell)}(q) \displaystyle=\frac{1}{|\tilde{a}|}\sum_{i\in\tilde{a}}h_{i}^{(\ell)}(q,e+\tilde{c},\tilde{a}), where c\sim y_{q},\;a\sim y_{q},\;e+\tilde{c}\not\sim y_{q},\;\tilde{a}\sim y_{q} . For each layer \ell , we train a linear probe f_{\ell}(h)=W_{\ell}h+b_{\ell} with W_{\ell}\in\mathbb{R}^{2\times d} , where d is the size of feature vectors, to distinguish the baseline from intervened activations. The probe is trained with AdamW (learning rate 10^{-3} ) for 100 epochs, with a cosine schedule. We evaluate on in-distribution (ID) GSM8K-Error and out-of-distribution (OOD) MATH500-Error, reporting layer-wise Area Under the Receiver Operating Characteristic Curve (AUROC) and Expected Calibration Error (ECE). In Table 2 , we report the linear probing results across models, selecting for each model the layer with the highest ID AUROC and lowest ID ECE. Results for all layers and models are provided in the Appendix D . We observe consistently high probing performance of all models on both datasets with almost perfect AUROC, indicating that the underlying activations remain linearly separable across all evaluated models despite the matching final answers. The linear separability suggests that the hidden self-correction ability discovered in Section 3 corresponds to a single linear direction in the latent space. Table 2 : Linear probing results. Comparison of AUROC and ECE evaluated on GSM8K (ID) and MATH500 (OOD) across four different models. The reported results are selected for the best layer. Detailed results can be found in Appendix D . Model GSM8K MATH500 Layer AUROC ECE AUROC ECE Qwen3-4B 1.000 0.002 1.000 0.026 21 R1-8B 0.998 0.019 0.996 0.022 31 R1-14B 1.000 0.003 0.998 0.012 28 R1-32B 1.000 0.002 0.997 0.008 38 4.3 Extracting the Critique Vector Following previous works ( 33 ; 28 ) , we identify the critique vector as the difference-in-means between contrastive pairs of runs constructed in Eqn 4 . We consider the intervened runs in which the model performs hidden self-correction as positive examples, and the baseline runs in which the model merely summarizes the CoTs without invoking recovery as negative examples. Formally, we compute the critique vector at layer \ell as the mean difference between intervened and baseline activations over the training dataset \mathcal{Q}_{\text{train}} : v_{\ell}=\frac{1}{|\mathcal{Q}_{\text{train}}|}\sum_{q\in\mathcal{Q}_{\text{train}}}\Bigl(h_{\text{intervened}}^{(\ell)}(q)-h_{\text{baseline}}^{(\ell)}(q)\Bigr). (5) 4.4 Steering with the Critique Vector We can use the critique vector in Eqn 5 to causally control and modulate the model’s hidden critique/recovery ability. Following the standard steering approach ( 36 ; 32 ; 2 ; 5 ; 42 ) , we add a scaled steering vector to the residual stream immediately after the MLP layer: h_{\text{steered}}^{(\ell)}=h^{(\ell)}+\alpha\,v_{\ell}, (6) where h^{(\ell)} is the post-MLP residual stream activation at layer \ell , v_{\ell} is the steering vector at layer \ell , and \alpha\in\mathbb{R} is a scalar steering coefficient. The modified activations h_{\text{steered}}^{(\ell)} are then propagated through subsequent layers, allowing the steering direction to shape the model’s downstream generation. Intuitively, this steering vector represents the critique ability of LRMs, which helps them detect and recover from the injected mistakes. Positive values of \alpha amplify the model’s self-critique ability, whereas negative values suppress this behavior. Further details on how we select the layer \ell for steering are provided in Appendix C . 4.5 Logit Lens Interpretation Table 3 : Comparison of the top 10 predicted tokens and their associated logit scores for the Qwen3-4B and R1-32B models. Rank Qwen3-4B R1-32B Token Logit Token Logit 1 wrongly 1.78 Wait 2.68 2 Nope 1.72 但实际上 (But actually) 2.65 3 而且还 (And also) 1.63 却 (But/Yet) 2.64 4 really 1.61 however 2.60 5 _that 1.60 but 2.58 6 Actually 1.60 不过 (However) 2.57 7 但却 (But yet) 1.59 Wait 2.55 8 ?)\n\n 1.57 ( 2.49 9 oque 1.57 wait 2.41 10 actually 1.56 However 2.41 In this section, we investigate the meaning of v_{\ell} using Logit Lens ( 25 ) , which estimates the model’s belief of its internal states. Specifically, we project v_{\ell} into the vocabulary space using the unembedding matrix W_{U}\in\mathbb{R}^{d\times|V|} , where |V| denotes the size of the vocabulary: \text{logits}_{\ell}=v_{\ell}\,W_{U}^{\top}\in\mathbb{R}^{|V|}. (7) We interpret v_{\ell} by extracting tokens corresponding to the highest logit values. Table 3 lists the top 10 tokens for Qwen3-4B (layer 29) and R1-32B (layer 57), ranked by their projected \text{logits}_{\ell} . Notably, some of the highest-ranked tokens include ‘Nope’ or ‘Wait’ , which typically occur when the model tries to reassess its reasoning and answers. In addition to the English tokens, the steering vectors v_{\ell} for both Qwen3-4B and R1-32B are closely linked to several Chinese adversative markers such as: ‘ 但却’ (but yet), ‘ 但实际上’ (but actually), or ‘ 却’ (but/yet). This suggests that the identified steering vectors v_{\ell} effectively encode the critique behavior. Additional results for multiple layers can be found in Appendix B . 5 Experiments In this section, we confirm the role of the critique vector found in Section 4.3 . As discussed above, the critique ability allows LRMs to detect errors in CoTs and correct these errors in the final answer. We causally intervene on the feature space with the critique vector to evaluate the detection ability in Section 5.2 . We demonstrate the importance of the correction ability for test-time scaling in Section 5.3 . Figure 3 : Effect of different steering coefficients \alpha evaluated on ProcessBench . Left to right : Error detection accuracy, correct solution accuracy, and F1 score. Increasing \alpha generally improves error detection and F1 scores while leading to a decline in correct solution accuracy. The dashed vertical line indicates the baseline performance at \alpha=0 . Figure 4 : Effect of different steering coefficients \alpha evaluated on BIG-Bench Mistake . Left to right : Error detection accuracy, correct solution accuracy, and F1 score. The results illustrate similar effects to Figure 3 , where there is a monotonic relationship between \alpha and error detection. The dashed vertical line indicates the baseline performance at \alpha=0 . 5.1 Experimental Details Datasets. We intervene on two widely-used mathematics benchmarks, GSM8K ( 8 ) and MATH500 ( 13 ) , to observe model behavior under injected errors. The GSM8K training split is used to train linear probes and derive steering vectors. For error detection evaluation, we use two human-annotated benchmarks: ProcessBench ( 50 ) for Olympiad-level mathematics and BIG-Bench-Mistake ( 35 ) for diverse logical reasoning tasks. For test-time scaling and self-correction analysis, we use our constructed GSM8K-Error and MATH500-Error datasets alongside BIG-Bench-Mistake, which supports correction from erroneous chains-of-thought. Models. We evaluate two open-source reasoning model families: Qwen3 ( 41 ) and DeepSeek-R1 ( 11 ) . We test models across different scales: Qwen3-4B , DeepSeek-R1-Distill-Llama-8B , DeepSeek-R1-Distill-Qwen-14B , and DeepSeek-R1-Distill-Qwen-32B , covering both Llama and Qwen architectures within R1. Throughout this paper, we refer to these models as Qwen3-4B, R1-8B, R1-14B, and R1-32B respectively. Evaluation. We use xVerify-0.5B-I to verify both reasoning traces and final answers, as it provides state-of-the-art math evaluation without requiring human annotation and avoids template mismatch ( 3 ) . For feature analysis and steering experiments, we use greedy decoding (temperature 0.0) to study deterministic internal behavior. For steering experiments, we set the temperature to 0.6 following official guidelines to prevent reasoning loops ( 27 ) , where the model gets stuck in repetitive thinking patterns, a common issue in reasoning models that we also observed with greedy decoding on challenging tasks. We fix all random seeds to 0 to ensure complete reproducibility across runs. All experiments use two H100 GPUs. Problem Statement (BIG-Bench Arithmetic) Expression: (((7+2-4*-5)*(4+-3*-2-4))*((4+-2*8*3)*(8+-3*1*5)))= Target: 53592 Question: Which calculation is incorrect? The equation can be written as “(A * B) * (C * D)”… Let’s calculate A = (7+2-4*-5)=(7+2-(-20))=(7+2+20)=30 . Let’s calculate B = 6. C = -44. D = -7. Then, the final equation is (30*6)*(-44*-7)=180*308=55440 . Baseline Model (Coefficient 0.0) Calculating A = (7+2-4*-5) . The reasoning here is to handle multiplication first. So, 4*-5 is -20 . Then, substituting back: 7+2-(-20) . Subtracting a negative is like adding, so 7+2+20 equals 30 . That looks correct. … [Checks B, C, and D] … Wait, is there any step that might have an error? Let me go through each calculation again. A: 7+2-4*-5 . 4*-5 is -20 . So, 7+2 is 9 , minus -20 is 9+20=30 . Correct. Outcome: The model fails to spot the arithmetic error ( 9+20\neq 30 ). Positive Steering (Coefficient +1.0) Calculating A = (7+2-4*-5) … 4*-5=-20 . Substituting back… 7+2+20=30 . That looks correct. … [Checks B, C, and D] … Wait, let me double-check… Rechecking A: 7+2 is 9 , minus (-20) is 9+20 . Wait, hold on! 7+2 is 9 , and subtracting -20 is adding 20 , so 9+20 is 29 , not 30 . Oh! Conclusion: The calculation of A as 30 is wrong. Outcome: LRM identifies & fixes the error. Figure 5 : Qualitative example of positive steering on BIG-Bench arithmetic task with R1-32B. There is a subtle arithmetic error introduced in the prompt ( 7+2+20=30 , whereas it should be 29 ). While the baseline model hallucinates that the math is correct during verification, positive steering enables the model to successfully catch the addition error. Problem Statement (BIG-Bench Arithmetic) Expression: ((((-6--3)-(-4+4))*((7*0)-(0--7)))-(((5+8)*(9+6))-((5*-4)*(2--1))))= Target: -234 Question: Which calculation is incorrect? The equation can be written as “A * B - C”… Let’s calculate A = -3. Let’s calculate B = ((7*0)-(0--7))=((0)-(-7))=(0+7)=7 . Let’s calculate C = 255. Then, the final equation is (-3*7)-255=-276 . Baseline Model (Coefficient 0.0) The expression for B is ((7*0)-(0--7)) . First, 7*0=0 . Then, 0-(-7)=7 . So, it becomes 0-7 , which is -7 . In Thought 2, they computed it as 7 . Therefore, Thought 2 is incorrect. Conclusion: The error is in Thought 2. Outcome: The model correctly identifies the arithmetic error ( B=-7 ). Negative Steering (Coefficient -1.0) Calculating B = ((7*0)-(0--7)) . So that’s 0-(-7) , which is 0+7=7 . So B is 7. That seems correct. Wait, let me double-check… 0-(-7) is adding 7. So B is 7. Conclusion: All steps seem to check out. No incorrect step found. Outcome: LRM fails to spot the error. Figure 6 : Qualitative example of negative steering on BIG-Bench arithmetic task with R1-32B. The problem contains a sign error in the calculation of term B ( 0-(0--7)=-7 , not 7 ). While the baseline model correctly critiques the reasoning and identifies the error, negative steering (Coefficient -1.0) makes the model hallucinate that the incorrect calculation is valid. 5.2 Steering for Error Detection Setup. We conduct experiments on ProcessBench and BIG-Bench Mistake, which contains pairs of questions and corresponding CoT style traces. There is either one or no incorrect thought in those CoTs. We prompt LRMs to return the index of incorrect thoughts or return -1 if there is no mistake (details in Appendix A.2 ). We compare LRMs’ answers with the ground-truth labels and report the accuracy on CoTs with mistake (Error Accuracy), CoTs without mistake (Correct Accuracy), and F1 Score on all samples. We report these metrics consistently with the original implementation of these two datasets. Critique vector controls the detection ability. We steer LRMs with the critique vector using multiple scales ranging from -1.0 to 1.0 (Figures 3 and 4 ). Overall, there is a positive correlation between the steering coefficients and the detection performance, confirming that the critique vector causally controls error detection. Additionally, Qwen3-4B yields better performance than R1-distilled models finetuned from Qwen2.5 and Llama3. On ProcessBench, Figure 3 shows that positive steering with coefficient 1.0 improves the error detection accuracy by 5\% on all models. In contrast, negative steering consistently decreases these figures. Similarly, Figure 4 illustrates the effectiveness of the critique vector on BIG-Bench Mistake, raising the error accuracy of Qwen3-4B by 4%: from 52\% to 56\% . Qualitative examples are shown in Figure 5 and 6 . We notice that in some scenarios, positive steering could harm the correct accuracy, such as Qwen3-4B. We provide qualitative samples in Appendix . We hypothesize that this intervention forces the model to revise the solution, potentially misunderstanding the correct thought and inadvertently recognizing them as incorrect. However, the drop in correct accuracy is insignificant, and thus, we can still observe the positive correlation between F1 Score and steering coefficients. Figure 7 : Impact of steering on model accuracy for test-time scaling. Left to right: Accuracy across three benchmarks: GSM8K-Error, MATH500-Error, and BIG-Bench Mistake. For increasing iterations of test-time scaling, positive steering consistently enhances the performance of all models. 5.3 Test-Time Scaling with Critique Steering Setup. We evaluate the controllability of error recovery performance on GSM8K-Error, MATH500-Error, and BIG-Bench Mistake. For GSM8K-Error and MATH500-Error, we let the model generate from the injected error until it finishes the final answer. For BIG-Bench Mistake, we put the given CoT in the thinking part and let the model produce the final answer. More details of the test-time scaling instruction prompt can be found at Appendix A.3 . We compare the final answer with the ground-truth label to compute the accuracy. To check if test-time scaling help LRMs fix the injected errors, we adopt the budget forcing approach ( 24 ) and append the “Wait” token when the model wants to terminate the thinking process. We report the performance at different numbers of iterations corresponding to the number of appended “Wait” tokens. For the effect of the critique vector, we perform positive steering with a coefficient 1.0 and negative steering with a coefficient -1.0 . Critique vector steers the recovery performance. Figure 7 depicts that steering with the critique vector successfully controls the ability to recover from injected errors. Particularly, the original Qwen3-4B yields 77\% accuracy on GSM8K-Error, while positive steering boosts that value to almost 90\% . In contrast, negative steering degrades the performance to slightly above 50\% . Similar trends can be observed on other models (R1-distilled series) and datasets (MATH500-Error and BIG-Bench Mistake). This confirms that the critique ability plays a vital role in error correction and is represented by our identified critique vector. Test-time scaling corrects the injected mistake. Besides the hidden critique ability, we expect that explicit reflection by manually appending “Wait” can also recover from injected errors. Indeed, Figure 7 shows that more budget forcing reliably improves the accuracy on three datasets across all models. This observation aligns with previous studies ( 24 ; 30 ) exibiting the effectiveness of test-time scaling. Critique vector further enhance test-time scaling. However, test-time scaling alone does not reliably repair the incorrect CoT. We observe that positive steering consistently boosts the accuracy of test-time scaling on all forcing iterations. These results highlight the impact of the hidden critique ability, suggesting that it is complementary to and cannot be completely captured by explicit reflection. On the other hand, negative steering also reduces the accuracy on all iterations, indicating that test-time scaling might not be successful without the critique ability. 6 Conclusion In this paper, we discover the novel hidden critique behavior where LRMs recover from injected mistakes in CoTs and produce correct final answers, but do not verbalize that error-detection and error-correction process. Through feature analysis, we show that LRMs internally distinguish the latent features of intervened samples with hidden critique behavior and those of original samples without hidden critique behavior. We identify the critique vector representing this hidden critique mechanism and validate it through logit lens analysis. Our empirical evidence shows that steering with this vector successfully controls the error detection performance on several benchmarks across many models. In addition, we point out that test-time scaling relies on the critique ability and can be combined with critique steering to further enhance the correction performance. Our findings shed light on the role of the critique ability in current LRMs and open many interesting questions. Future extension should investigate if this behavior is the result of pre-training or post-training and how it emerges. Since the hidden critique ability makes LRMs correct errors in the final answer but not verbalize in CoTs, we can analyze its impact on the faithfulness of CoTs. Furthermore, it is interesting to study the influence of this ability on other properties, for example, alignment. Finally, we can leverage this ability to improve other test-time scaling methods, such as verifier-based approach, self-consistency, or self-correction prompting. We leave them to future works. Impact Statement Error-correction and error-detection are beneficial to guarantee the quality of responses provided by language models, which is important for high-stakes applications such as medical domains or scientific discovery. Our analysis reveals the mechanism of this behavior, and our proposed methods allow controlling error-recovery performance, advancing the development of reliable and trustworthy AI systems. Appendix A System Prompt A.1 GPT-5 Instruction Prompt The following system prompt \mathcal{P}_{\text{err}} was used to generate incorrect reasoning steps: GPT-5 Instruction Prompt Given a question and its ground-truth answer, generate a natural reasoning trace that contains a calculation error. PURPOSE: This trace is a deliberate error variant used to test a reasoning model’s ability to recover from arithmetic mistakes. INSTRUCTIONS: - Write the trace as a sequence of short, natural reasoning steps, phrased as if someone is thinking aloud. - Use natural, anthropomorphic openings such as “First, I need to…”, “Okay, so…”, “Let me…”, or other common starter phrases typically found in the thinking process of reasoning models. - Separate steps with two newlines ( \backslash n \backslash n). - Do NOT mention or hint that an error occurred (no phrases like “mistake”, “should be”, or “wrong”). - Introduce an arithmetic error at the earliest plausible calculation step. - Stop the reasoning immediately after the mistaken calculation step; do not continue solving or produce an answer. The goal is to isolate the reasoning up to the error only. - The trace must be self-consistent and match the question up to the step before the error. - Keep the reasoning coherent, natural, and plausible. EXAMPLE: Question: Olivia buys 3 packs of markers, each pack containing 12 markers. She gives 7 markers to her friend. How many markers does Olivia have left? Ground-truth answer: 29 Trace: “Okay, so I need to figure out how many markers Olivia got in total. She bought 3 packs, each with 12 markers. \backslash n \backslash nNext, I’ll multiply 3\times 12=38 .” Return only the interventions using the configured JSON schema. Question: {question} Ground-truth answer: {ground_truth} A.2 Error Detection Prompt We evaluate reasoning error detection using two benchmarks: ProcessBench and BIG-Bench-Mistake, each with its own prompting format. For ProcessBench, we follow the original critique-based prompting scheme exactly as implemented in the official repository. 1 1 1 https://github.com/QwenLM/ProcessBench The model is provided with a math problem and a paragraph-structured solution trace, and is instructed to identify the earliest paragraph containing an error, or return -1 if no error is present. For BIG-Bench-Mistake, we follow the original task-specific templates provided by the authors. 2 2 2 https://github.com/WHGTyen/BIG-Bench-Mistake We observe that strong reasoning models often solve the problem directly rather than performing error detection. To maintain the original task structure while enforcing focused evaluation, we prepend a minimal instruction requiring the model to identify incorrect steps only. ProcessBench Error Detection Prompt The following is a math problem and a solution (split into paragraphs, enclosed with tags and indexed from 0): [Math Problem] {problem} [Solution] {tagged_response} Your task is to review and critique the solution paragraph by paragraph. Once you identify an error in a paragraph, return the index of the paragraph where the earliest error occurs. Otherwise, return the index of -1 . Please put your final answer in \boxed{} . BIG-Bench-Mistake Task Template Given a problem and its solution steps below, identify whether there is any incorrect reasoning step. Do NOT solve the problem. Task: {task_description} Problem: {input} Reasoning: {steps} Return \boxed{k} where k is the step number of the incorrect step, or \boxed{-1} if all steps are correct. A.3 Test-time Scaling Prompt We perform test-time scaling by injecting a corrupted chain-of-thought (CoT) into the model’s intermediate thinking process and allowing the model to continue generating. This setup simulates partial erroneous reasoning and evaluates whether extended inference can recover from the injected mistake. Concretely, each generation follows the structure: Test-time Scaling Generation Format {question} \langle\texttt{think}\rangle {injected_erroneous_CoT} {continued_reasoning} \langle /think \rangle {final_answer} After the model completes its reasoning, we optionally append the token “Wait” multiple times to extend inference depth. This simple control mechanism encourages backtracking and revision of earlier steps, following prior work (e.g., ( 24 ; 46 ; 10 ) ). Appendix B Additional Results of Logit Lens Interpretation To validate the robustness of the identified steering vectors v_{\ell} , we analyze the trajectory of the top projected tokens with Logit Lens across five consecutive layers. Tables 4 and 5 present the evolution of the top 10 tokens and their associated logit scores for Qwen3-4B (Layers 29–33) and R1-32B (Layers 55–58), respectively. Comparing the two models reveals some qualitative differences in their self-critique mechanisms. While R1-32B (Table 5 tends toward the interruptive tokens such as ‘Wait’ or ‘ 但实际上’ (but actually), Qwen3-4B (Table 4 ) exhibits a preference for direct negation and correction tokens, such as ‘wrongly’ , ‘Nope’ , and ‘Actually’ throughout Layers 29–33. We hypothesize that this distinction may suggest that the extracted vector in Qwen3-4B encodes a more immediate “fact-checking” signal, whereas the vector in R1-32B captures a specialized backtracking behavior. This may also explain the significant difference in proportions of ‘ \times Think \checkmark Answer’ and ‘ \checkmark Think \checkmark Answer’ in Figure 2 between the two models. Table 4 : Evolution of top 10 predicted tokens and logit scores across five layers (Layer 29 to Layer 33) for the Qwen3-4B model. Rank Layer 29 Layer 30 Layer 31 Layer 32 Layer 33 Token Logit Token Logit Token Logit Token Logit Token Logit 1 wrongly 1.78 而且还 (And also) 2.28 really 2.49 wrongly 2.88 !)\n 3.10 2 Nope 1.72 Nope 2.21 wrongly 2.45 !)\n 2.69 wrongly 2.91 3 而且还 (And also) 1.63 wrongly 2.17 而且还 (And also) 2.42 —but 2.65 !)\n\n 2.80 4 really 1.61 really 2.11 Nope 2.38 而且还 (And also) 2.63 )!\n 2.79 5 _that 1.60 !)\n 2.02 Actually 2.22 rethink 2.62 !!\n\n 2.77 6 Actually 1.60 只不过 (Just) 2.01 !)\n 2.18 !)\n\n 2.62 !!\n 2.75 7 但却 (But yet) 1.59 Actually 2.00 只不过 (Just) 2.18 ?!\n\n 2.55 !!!\n 2.74 8 ?)\n\n 1.57 !!!\n 1.99 actually 2.17 !!!\n 2.52 really 2.74 9 oque 1.57 !)\n\n 1.96 !!!\n 2.15 !!\n\n 2.51 —but 2.74 10 actually 1.56 choc 1.94 somebody 2.15 —we 2.49 actually 2.74 Table 5 : Evolution of top 10 predicted tokens and logit scores across five layers (Layer 55 to Layer 58) for the R1-32B model. Rank Layer 55 Layer 56 Layer 57 Layer 58 Token Logit Token Logit Token Logit Token Logit 1 但实际上 (But actually) 2.83 但实际上 (But actually) 2.66 Wait 2.68 Wait 2.95 2 but 2.54 Wait 2.60 但实际上 (But actually) 2.65 Wait 2.78 3 Wait 2.43 Wait 2.46 却 (But/Yet) 2.64 however 2.78 4 但由于 (But due to) 2.42 however 2.46 however 2.60 ( 2.74 5 但现在 (But now) 2.38 却 (But/Yet) 2.45 but 2.58 却 (But/Yet) 2.70 6 却被 (But was) 2.37 but 2.36 不过 (However) 2.57 but 2.68 7 却 (But/Yet) 2.30 wait 2.31 Wait 2.55 wait 2.65 8 但从 (But from) 2.28 但由于 (But due to) 2.30 ( 2.49 但实际上 (But actually) 2.62 9 Wait 2.18 却被 (But was) 2.28 wait 2.41 不过 (However) 2.58 10 不过 (However) 2.18 However 2.28 However 2.41 However 2.50 Appendix C How to Select the Optimal Layer for Steering? Following previous works ( 49 ; 5 ; 36 ; 32 ) , we only apply the steering vector to the LRM for one selected layer. In this section, we describe the procedure for identifying the optimal layer that is the most representative of the self-critique mechanism. Consistent with 32 , for each LRM, we apply the steering vector as defined in Eq. 6 to each layer independently using scaling factors \alpha\in\{-1.0,1.0\} . All experiments for layer-wise steering are conducted on the GSM8K training split (Figure 8 ). We define the optimal layer as the one which yields the greatest accuracy differential between positive (1.0) steering and negative (-1.0) steering. At this layer, the effect of self-correction enhancement or suppression is the most pronounced, suggesting that this layer acts as the primary locus for the model’s internal self-critique processes. As demonstrated in Table 6 and Figure 8 , the differential between positive and negative steering ( \Delta ) is the most prominent for middle layers, with \Delta even exceeding 38\% for Layer 21 of Qwen3-4B. Figure 8 : Layer-wise steering results. The separation between the performance of positive and negative steering is minimal for early and late layers. For middle layers, this performance gap is the most noticeable. Table 6 : Results of the optimal layer. Overall accuracy (%) of the optimal layer evaluated on the training split of GSM8K for baseline ( \alpha=0.0 ), positive steering ( \alpha=1.0 ), negative steering ( \alpha=-1.0 ), and the difference between positive and negative steering ( \Delta ). Model \alpha=0.0 \alpha=-1.0 \alpha=1.0 \Delta Layer Qwen3-4B 42.36 25.00 63.72 38.72 21 R1-8B 57.03 53.36 73.48 20.13 13 R1-14B 80.99 73.71 85.64 11.93 28 R1-32B 79.42 69.74 86.25 16.51 44 Appendix D Additional Results of Linear Separability of Internal Thoughts In Section 4.2 , we have provided the results of the best layer for the linear probing experiments. In this section, we outline the results (AUROC and ECE) for all layers across four LRMs: Qwen3-4B, R1-8B, R1-14B, and R1-32B. As shown in the AUROC plots (left column), linear separability saturates near 1.0 for the majority of the network depth across all models. The GSM8K dataset (in-distribution, orange) consistently exhibits high and stable separability almost immediately from the initial layers. In addition, we observe that lower layers tend to be poorly calibrated (high ECE), with calibration improving significantly in the middle layers. Comparing the two datasets, GSM8K demonstrates consistently lower Expected Calibration Error and smoother layer-wise progression compared to MATH500. The MATH500 dataset exhibits considerably higher variance and erratic spikes in ECE. However, for most layers, the performance of the linear prober for GSM8K and MATH500 remains close, suggesting this separability has a high level of generalization. Figure 9 : Layer-wise linear probing results. Left to right: AUROC and ECE for GSM8K (in-distribution) and MATH500 (out-of-distribution). For both settings, the linear prober can achieve near-perfect AUROC and low ECE at several layers. Appendix E Qualitative Examples E.1 Full Examples of Original and Intervened Generation We provide the complete responses of R1-32B model as shown in Figure 1 from the GSM8K dataset (Figure 10 ). Response Comparison - R1-32B Original Response Prompt: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers’ market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers’ market? ⟨think⟩ First, I need to determine the total number of eggs Janet’s ducks lay each day. According to the problem, the ducks lay 16 eggs per day. Next, I’ll calculate how many eggs Janet uses each day. She eats 3 eggs for breakfast and uses 4 eggs for baking muffins, totaling 7 eggs consumed daily. To find out how many eggs are left to sell, I’ll subtract the eggs used from the total eggs laid: 16 eggs minus 7 eggs equals 9 eggs available for sale. Finally, I’ll calculate Janet’s daily earnings from selling the eggs. Each egg is sold for $2, so multiplying the number of eggs sold by the price per egg gives 9 eggs multiplied by $2, which equals $18 per day. ⟨/think⟩ Final Answer: Let’s break down the problem step by step to find out how much Janet makes every day at the farmers’ market. 1. Total Eggs Laid Per Day: 16 eggs/day 2. Eggs Consumed Per Day: • For Breakfast: 3 eggs/day • For Baking Muffins: 4 eggs/day Total Eggs Consumed: 3+4=7 eggs/day 3. Eggs Available for Sale: 16-7=9 eggs/day 4. Earnings from Selling Eggs: 9\text{ eggs}\times\$2/\text{egg}=\$18/\text{day} Final Answer: \boxed{18} Intervened Response Prompt: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers’ market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers’ market? ⟨think⟩ First, I want to find how many eggs Janet uses before selling any. She eats 3 each morning and uses 4 more for muffins. So she uses 3+4=7 eggs per day. Next, I’ll subtract the eggs she uses from the total she collects. She collects 16 eggs each day. So, 16-6=10 eggs are left to sell. Finally, I’ll calculate her earnings by multiplying the number of eggs she sells by the price per egg. She sells 10 eggs at $2 each. So, 10\times\$2=\$20 . Therefore, Janet makes $20 every day at the farmers’ market. ⟨/think⟩ Final Answer: Let’s break down the problem step by step to find out how much Janet makes every day at the farmers’ market. 1. Total Eggs Laid Per Day: Janet’s ducks lay a total of: 16\text{ eggs/day} 2. Eggs Used by Janet: • For Breakfast: Janet eats 3 eggs every morning. • For Muffins: She uses 4 eggs daily. Total Eggs Used: 3+4=7\text{ eggs/day} 3. Eggs Available for Sale: 16-7=9\text{ eggs/day} 4. Earnings from Selling Eggs: 9\text{ eggs}\times\$2/\text{egg}=\$18 Final Answer: \boxed{18} Figure 10 : Complete example shown in Figure 1 generated by DeepSeek-R1-Distill-Qwen-32B (R1-32B). As laid out in Section 3.1 , we generate the arithmetic error with GPT-5 (highlighted by a box in the right column) and insert this error immediately after the \langle think \rangle token, which causes the model to make subsequent mistakes (highlighted in red ) in the thinking stage. However, for the final answers, the model is able to recover from those thinking mistakes and derive the correct final solution. In this example, correct thinking steps and solutions are highlighted in green . E.2 Effects of Positive and Negative Steering on LRMs In this section, we present a qualitative analysis of R1-32B model’s reasoning behaviors under the influence of steering vectors derived from the critique mechanism. We examine three distinct configurations: the Baseline Model (Coefficient 0.0), Positive Steering (Coefficient +1.0), and Negative Steering (Coefficient -1.0). The examples, selected from GSM8K samples of ProcessBench and GSM8K-Error, illustrate how steering modulates the model’s capacity for error detection, semantic interpretation, and arithmetic verification. For this section, ellipsis is used with “…” to truncate model responses for brevity and clarity. E.2.1 Error Detection We first examine how steering affects error detection capability, drawing on examples from ProcessBench-GSM8K and GSM8K-Error. These cases reveal two complementary failure modes that positive and negative steering address in opposite directions: missing genuine errors and hallucinating errors in correct solutions. As shown in Figures 11 and 12 , positive steering enhances the model’s ability to detect subtle reasoning errors while avoiding false positives. In contrast, Figure 13 demonstrates that negative steering suppresses critical verification, causing the model to accept erroneous premises without scrutiny. Problem Statement (ProcessBench - GSM8K) Problem: Gerald works at a daycare that pays him $30 every day. He worked for an entire week and spent a total of $100. How much does he have left? Target: $110 Question: Identify any errors in the solution step below. Proposed Solution Step: First, calculate the weekly earnings. Gerald earns $30 each day. Assuming a standard workweek of 5 days : \text{Weekly earnings}=5\times\$30=\$150 Baseline Model (Coefficient 0.0) The solution starts by calculating his weekly earnings. It mentions a standard workweek of 5 days. So, they multiply 5 by $30, which gives $150. That seems correct because 5 days times $30 is indeed $150. … [Checks subtraction of expenses] … Conclusion: The calculation steps seem accurate, and there doesn’t appear to be any errors in the reasoning or arithmetic. Outcome: The model overlooks the “entire week” wording and accepts the 5-day assumption. Positive Steering (Coefficient +1.0) First, the solution starts by calculating his weekly earnings… 5 by $30 gives $150. But wait, the problem says he worked for an entire week . Typically, a week has 7 days. If he worked 7 days, his earnings should be 7\times\$30=\$210 . The initial solution assumes a 5-day workweek, but the problem implies the full week. Conclusion: The error is in assuming 5 days instead of 7. Outcome: Successfully identifies the reasoning error. Figure 11 : Qualitative example of positive steering on a GSM8K question from ProcessBench with R1-32B model. The proposed solution makes an unjustified assumption about the work schedule (5 days vs 7 days). The Baseline model accepts this assumption as “standard”, while the steered model critically analyzes the specific phrasing “entire week” and identifies the discrepancy. Problem Statement (GSM8K) Problem: Tom bought a CD for $4, and a headphone set. In total, he paid $48. How many more CDs would Tom have been able to buy if he had decided not to buy the headphone set? Proposed Solution: … To find the cost of the headphone set, we subtract the cost of the CD from the total amount he paid: \$48-\$4=\$44 . So, the headphone set costs $44. If Tom didn’t buy the headphone set, he would have $44 more to spend on CDs. Since each CD costs $4, we can divide $44 by $4 to find out how many more CDs he could buy: \$44\div\$4=11 . Baseline Model (Coefficient 0.0) Let’s check paragraph 2. ”If Tom didn’t buy the headphone set, he would have $44 more…” Wait, the total amount he paid was $48. If he didn’t buy the headphones, he would have had the full $48 available to spend on CDs. The solution claims he would have $44 more, but he should have the total $48. So 48 / 4 = 12 CDs. Since he already bought one, it’s 11 more? No, the calculation of available funds is wrong. Conclusion: The solution is incorrect because the available amount should be $48. Outcome: The model hallucinates an error in a correct solution (False Positive). Positive Steering (Coefficient +1.0) Checking paragraph 2… The problem states he bought a CD ($4) and a headphone set. The total paid is $48. This implies the headphone set cost \$48-\$4=\$44 . If he decides not to buy the headphone set, he saves exactly the cost of the headphones: $44 . He can then use this $44 to buy more CDs: \$44/\$4=11 . This matches the solution. Conclusion: The logic in paragraph 2 is sound. He saves $44, not $48, because the CD purchase is fixed. Outcome: Correctly validates the solution (Avoids Hallucination). Figure 12 : Qualitative example on GSM8K task with Qwen3-4B model. The baseline model hallucinates a logic error, incorrectly arguing that the total budget ($48) should be used for the counterfactual, ignoring that the CD was still purchased. The positive steering helps the model successfully verify the correct logic: money saved equals the cost of the unpurchased item ($44). Problem Statement (ProcessBench-GSM8K) Problem: A caterer prepared hot dogs for 36 guests. He prepared enough for half of the guests to have two hotdogs (implying the other half has one). However, 40 guests showed up, and everyone wanted a second hotdog. Question: How many guests did not get a second hotdog? Proposed Solution Step 1: The caterer planned for half of 36 guests (18) to eat two hotdogs. So, the total number of hotdogs prepared is 18\times 2=36 hotdogs. (Note: Ignores the single hotdogs for the other 18 guests). Baseline Model (Coefficient 0.0) Let’s figure out how many hotdogs he prepared. Half of 36 is 18, so 18 guests have two hotdogs ( 18\times 2=36 ). Wait, hold on a second. If half are having two, the other half would have one each. So the total should be 36+18=54 hotdogs. But the solution says he prepared 36 hotdogs. That doesn’t seem right. The solution only accounted for the 36… their initial calculation is wrong. Outcome: Correctly identifies the error (54 vs 36 prepared). Negative Steering (Coefficient -1.0) Half of 36 is 18, so 18 guests would have two hotdogs each. Multiplying 18 by 2 gives 36 hotdogs. So, the caterer prepared 36 hotdogs in total. …If the caterer only prepared 36 hotdogs, and 40 guests wanted a second… 4 guests couldn’t get a second. The solution seems accurate… Therefore, the solution is correct, and there’s no error. Outcome: Fails to spot the error (Maintains the Error). Figure 13 : Qualitative example of negative steering on ProcessBench-GSM8K with R1-32B model. The prompt contains a logical error in the first step (calculating total food prepared). The baseline model successfully debugs the setup ( 54 hotdogs vs 36 ), whereas negative steering suppresses this critical check and accepts the erroneous premise. E.2.2 Test-Time Scaling We next examine how steering interacts with the model’s internal self-verification behavior with test-time scaling methods. Figures 14 and 15 illustrate that positive steering enables successful error recovery during the “Wait” phase, whereas negative steering causes the model to rationalize and reinforce injected errors rather than correct them. Problem Statement (GSM8K-Error) Question: Bailey starts with a certain amount of money. Then she receives a weekly allowance of $5 for 8 weeks. At the end of the 8 weeks, if she has a total of $100, how much money did Bailey start with? Target: 60 Injected Error: First, I need to find the total amount she received from allowance: \$5 each week for 8 weeks. So I’ll multiply 5×8=30 and subtract it from the final total ( \$100 ). Baseline Model (Coefficient 0.0) Wait, that doesn’t make sense… But just to double-check, let me add $70 and $30 again… Yes, 70+30=100 . So, my answer is correct. Outcome: The model fails to spot the arithmetic error ( 5\times 8\neq 30 ). Positive Steering (Coefficient +1.0) Wait, but let me double-check. If she started with $70 and received $5 each week for 8 weeks… Wait, 5\times 8 is actually 40, not 30. So initial money should be 100-40=60 . Yes, that makes sense. If she started with $60 and received $5 for 8 weeks, that’s 60+40=100 . Conclusion: The initial calculation of allowance was wrong. Outcome: Identifies & Fixes Error ( 30\rightarrow 40 ). Figure 14 : Qualitative example of positive steering on GSM8K-Error with R1-32B model. The baseline model uses a multiplication error ( 5\times 8=30 ) and confirms it during self-verification ( 70+30=100 ). Positive steering initially makes the same error but successfully catches it during the “Wait” phase, correcting the allowance to 40 and the final answer to 60 . Problem Statement (GSM8K-Error) Question: Juan and Carlos sell lemonade. Costs/gallon: $3 lemons, $2 sugar. Sell: 20 glasses at $0.50. Profit $25. How much spent on lemons? Injected Error: The costs are $3 for lemons and $2 for sugar per gallon, so the total cost per gallon is $6 . Revenue per gallon is $10. Profit per gallon is $4. If total profit is $25, gallons sold is 6.25. Total spent on lemons is 6.25 * $3 = $18.75. Baseline Model (Coefficient 0.0) Wait, but the problem asks for the amount spent on lemons… So, 6.25 gallons * $3 = $18.75. Wait, but let me double-check my steps to make sure I didn’t make a mistake. Total cost per gallon: $3 (lemons) + $2 (sugar) = $5? Wait, no, $3 + $2 is $5, so total cost per gallon is $5, not $6. … Therefore, the amount spent on lemons is 5 gallons * $3 per gallon = $15. Outcome: The model catches the addition error ($3+$2=$5) and corrects the answer. Negative Steering (Coefficient -1.0) Wait, but the problem asks for the amount spent on lemons… So, using the number of gallons sold, which is 6.25, multiplied by the cost per gallon for lemons, which is $3, gives the total amount spent on lemons. So, 6.25 gallons * $3 per gallon = $18.75. But wait, the problem might expect an exact answer… 25/4 * 3 = 75/4, which is $18.75. Yes, that makes sense. Outcome: Maintains the Error. Figure 15 : Qualitative example of negative steering on ProcessBench-GSM8K with R1-32B model. The prompt context introduces an arithmetic error ($3+$2=$6 instead of $5). The baseline model initially adopts the error but triggers a self-correction after generating “Wait,” re-evaluating the costs. The negatively steered model accepts the erroneous premise and justifies the incorrect result.