Reasoning Models Know When They’re Right: Probing Hidden States for Self-Verification
Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, He He · 2025
Read Reasoning Models Know When They’re Right: Probing Hidden States for Self-Verification by Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, He He (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05419.txt.
Introduction
Anqi Zhang and co-authors probe whether reasoning models know when their own answers are right, in a 2025 paper. Such models often keep thinking long after they have hit the right answer, which wastes time and cost.
A probe trained on the model's inner states can grade half done answers with high accuracy and well matched confidence. The inner states even hint at whether a future answer will be right before it is fully formed. Used as a stop rule, the probe cut token use by about a quarter with no loss of score. The models hold the news, the authors note, but fail to use it.
Catalog record
| CREATOR | Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, He He |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-05419
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reasoning Models Know When They’re Right: Probing Hidden States for Self-Verification Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, He He Abstract Reasoning models have achieved remarkable performance on tasks like math and logical reasoning thanks to their ability to search during reasoning. However, they still suffer from overthinking , often performing unnecessary reasoning steps even after reaching the correct answer. This raises the question: can models evaluate the correctness of their intermediate answers during reasoning? In this work, we study whether reasoning models encode information about answer correctness through probing the model’s hidden states. The resulting probe can verify intermediate answers with high accuracy and produces highly calibrated scores. Additionally, we find models’ hidden states encode correctness of future answers, enabling early prediction of the correctness before the intermediate answer is fully formulated. We then use the probe as a verifier to decide whether to exit reasoning at intermediate answers during inference, reducing the number of inference tokens by 24% without compromising performance. These findings confirm that reasoning models do encode a notion of correctness yet fail to exploit it, revealing substantial untapped potential to enhance their efficiency. 1 Introduction Recent advances in reasoning models, such as OpenAI’s o1 ( OpenAI, 2024 ) and DeepSeek-R1 ( DeepSeek-AI et al., 2025 ) , have demonstrated significant progress in complex reasoning capabilities, particularly in domains such as mathematical problem solving ( DeepMind, 2024 ; Zhou et al., 2023 ) and logical reasoning ( Feng et al., 2023 ; Liu et al., 2025 ; Lam et al., 2024 ) . A key advantage of reasoning models lies in their ability to search: they often explore multiple reasoning paths leading to different intermediate answers to the original problem before arriving at a final solution (Figure 1 , left). While this search-based reasoning is beneficial, it also introduces inefficiencies. Previous studies ( Chen et al., 2025 ; Sui et al., 2025 ) show that reasoning models tend to overthink by exploring additional reasoning paths even after reaching a correct answer. This observation prompts the question: to what extent can models evaluate the correctness of their intermediate answers during reasoning? The answer to this question is also crucial to preventing overthinking, either through a more targeted design of the training strategy or a better elicitation method. We investigate this question by probing the model’s hidden states for answer correctness. Specifically, we segment the long Chain-of-Thought (CoT) into chunks containing intermediate answers, and train a binary classifier to predict answer correctness from the model’s hidden states at the answer positions (Figure 1 ). We find that information about answer correctness is readily encoded in the model’s internal representations. A simple probe can reliably extract this information, performing accurately on both in-distribution and out-of-distribution examples. Moreover, the probe is highly calibrated, with an expected calibration error (ECE) below 0.1. Our analysis also reveals that the model’s hidden states contain ”look-ahead” information: correctness can be predicted even before the intermediate answer is fully articulated. Notably, when applying the same probing method to traditional short CoT models, we observe a significant degradation in performance, suggesting that the encoded correctness information is likely acquired during long CoT training. We also investigate whether reasoning models effectively use this information on answer correctness during inference. Because the trained probe is well-calibrated, we use the output score to measure the model’s confidence in the current intermediate answer. Ideally, the model should reason at an optimal length if it is taking advantage of the well-encoded correctness information, i.e. it should stop reasoning when the confidence about an intermediate answer is high enough. We adopt the probe as a verifier and implement a confidence-based early-exit strategy by thresholding confidence scores from the probe. The strategy achieves up to 24% reduction in inference tokens without compromising accuracy. The improvement in efficiency with our verifier reveals that while reasoning models encode information about answer correctness, they do not efficiently use this internal knowledge during inference. Figure 1 : An illustration of the probing method. On the left side, long CoT is parsed into multiple chunks, each corresponding to a reasoning path and contains an intermediate answer as termination. On the right side, representations for each chunk are obtained and probe is used to predict the probability of answer being correct. 2 Related work Uncertainty estimation in LLMs. Black-box techniques for estimating LLM uncertainty over their response have primarily focused on prompting the model to verbalize its confidence directly, often aggregating self-reported confidence scores over multiple samples ( Lin et al., 2022 ; Tian et al., 2023 ) . However, Xiong et al. (2024) ; Kapoor et al. (2024) find that white-box methods, including those that depend on internal model representations ( Mielke et al., 2022 ) , tend to perform better than black-box methods on confidence estimation. For instance, Azaria & Mitchell (2023) ; Burns et al. (2024) show that an LLM’s representation after processing a statement is highly predictive of the statement’s correctness; moreover, linear probes trained on these representations can classify correctness, even without ground-truth labels. We extend this work to long CoT generated by reasoning models, demonstrating that the representations at intermediate stages of the CoT also capture key information about the correctness of each intermediate stage. Efficient reasoning during inference. Reasoning models demonstrate improved performance on many tasks thanks to their ability to search while generating reasoning chains, which often demand additional test-time compute in comparison to standard CoT ( DeepSeek-AI et al., 2025 ) . Additionally, reasoning models often suffer from repeated and unnecessary reasoning steps—or “overthinking”—even after a correct answer has been reached ( Chen et al., 2025 ) . Recent work has explored training methods to make reasoning more concise or to reduce the frequency of overthinking ( Chen et al., 2025 ; Munkhbat et al., 2025 ) . Other inference-time techniques focus on curtailing generations that are unlikely to be successful ( Zhao et al., 2025 ; Manvi et al., 2024 ; Li et al., 2024 ) or dynamically adjusting the test-time compute budget based on input difficulty or other properties of the prompt ( Damani et al., 2024 ; Wang et al., 2025 ; Xu et al., 2024 ; Fu et al., 2024 ) . We find that while the model encodes information about answer correctness, it fails to use it efficiently, which may contribute to overthinking. We leverage this to perform threshold-based early-exiting at inference time, reducing test-time compute while preserving performance. Learned verifiers. The ability to verify intermediate answers is also related to the line of works on verifiers, which is an important technique used to regulate test-time scaling. Previous work has focused on training verifiers to classify the correctness of a model-generated solution or select which of two model-generated responses is preferred ( Bai et al., 2022 ; Cobbe et al., 2021 ; Lightman et al., 2023 ; Zhang et al., 2025 ; Zheng et al., 2023 ; Creswell & Shanahan, 2022 ; Paul et al., 2024 ) . However, recent improvements in reasoning capabilities have enabled LLMs to critique and refine their outputs without the aid of external verifiers, often using natural language prompt templates to guide self-critique of model-generated