Thought calibration: Efficient and confident test-time scaling

Menghua Wu, Cai Zhou, Stephen Bates, Tommi Jaakkola · 2025

Read Thought calibration: Efficient and confident test-time scaling by Menghua Wu, Cai Zhou, Stephen Bates, Tommi Jaakkola (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18404.txt.

Introduction

Wu and co-authors present a 2025 paper on stopping a reasoning model's thinking at the right moment. Long chains of thought lift accuracy but burn computing time. Easy problems waste the most, since one rule for all fits none. Their method, thought calibration, watches the model's inner states and calls a stop when fresh reasoning levels off.

Small probes read the model's hidden signals to judge structure and consistency of the chain. Tests on three reasoning models and four data sets kept accuracy intact. Thinking tokens fell by up to sixty percent on familiar data and twenty percent on new data. The work offers a cheap dial for test-time budgets.

Catalog record

CREATORMenghua Wu, Cai Zhou, Stephen Bates, Tommi Jaakkola
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18404

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Thought calibration: Efficient and confident test-time scaling Menghua Wu, Cai Zhou, Stephen Bates, Tommi Jaakkola Abstract Reasoning large language models achieve impressive test-time scaling by thinking for longer, but this performance gain comes at significant compute cost. Directly limiting test-time budget hurts overall performance, but not all problems are equally difficult. We propose thought calibration to decide dynamically when thinking can be terminated. To calibrate our decision rule, we view a language model’s growing body of thoughts as a nested sequence of reasoning trees, where the goal is to identify the point at which novel reasoning plateaus. We realize this framework through lightweight probes that operate on top of the language model’s hidden representations, which are informative of both the reasoning structure and overall consistency of response. Based on three reasoning language models and four datasets, thought calibration preserves model performance with up to a 60% reduction in thinking tokens on in-distribution data, and up to 20% in out-of-distribution data. 1 Introduction Test-time scaling presents a new paradigm for improving language model reasoning by expending large amounts of compute during inference ( Kaplan et al., 2020 ; Wei et al., 2022 ) . Though the strategies for eliciting reasoning vary – from large-scale reinforcement learning ( Guo et al., 2025a ) to explicit tree search ( Zhang et al., 2024b ; Zhang et al., 2024a ) – a common effect is that language models improve by sampling substantially more tokens. This may result in wasted compute on easy problems ( Chen et al., 2024 ; Sui et al., 2025 ) , but naively limiting the generation length leads to pronounced drops in accuracy ( Muennighoff et al., 2025 ) . This motivates early stopping strategies that reduce the inference budget without significantly degrading performance, and control the extent of impact, if performance must be compromised. Numerous methods have been proposed for teaching language models to be economical with their token budgets ( Han et al., 2024 ; Arora and Zanette, 2025 ; Sui et al., 2025 ) , or for identifying opportune stopping points ( Yang et al., 2025 ; Zhang et al., 2025 ) . While these methods demonstrate strong empirical performance, they lack strict statistical guarantees about when they could fail. In an orthogonal direction, conformal prediction has been adapted to equip language models with calibrated confidences about the quality or consistency of their generations ( Mohri and Hashimoto, 2024 ; Quach et al., 2024 ; Rubin-Toles et al., 2025a ; Rubin-Toles et al., 2025b ; Cherian et al., 2024 ) . However, most of these algorithms operate through post-hoc filtering and require external LLM-based validation for scoring intermediate steps – rendering them unsuitable for actively terminating generation. In this work, we jointly pursue an effective and calibrated decision rule to determine when a language model can stop “thinking.” To do so, we introduce the notion of a reasoning tree, where at each step of sampling, a language model either adds a new leaf, walks along the tree, or backtracks to a previous step. Notably, identifying when the thoughts have converged is equivalent to detecting when this reasoning tree stops growing. Inspired by this concept, we approach early stopping as multiple hypothesis testing problem. At each generation step, we test whether the current tree is expected to change, based on the predictions of lightweight probes over the language model’s hidden representations. Our algorithm is based on the Learn then Test framework ( Angelopoulos et al., 2021 ) , which provides finite-sample, distribution-free guarantees for controlling the risk of our decisions. We evaluate this strategy, thought calibration , based on its ability to guide efficient reasoning, and whether its decisions are well-calibrated. Our experiments consider two empirical settings: we may or may not have access to training and calibration data from the true test distribution. In the first setting, we train variants of thought calibration using three reasoning lanugage models (DeepSeek-R1 distilled Qwen 32B and Llama 70B ( Guo et al., 2025a ; Yang et al., 2024 ; Grattafiori et al., 2024 ) , QwQ 32B ( Team, 2025 ) ), evaluated on a helf-out split of s1K-1.1 ( Muennighoff et al., 2025 ) . Here, we are able to halve the number of thinking tokens across accuracy levels, with a maximum reduction of 60% . Then we evaluate Qwen 32B-based thought calibration on three test datasets: AIME 24, GPQA Diamond ( Rein et al., 2024 ) , MATH-500 ( Lightman et al., 2023 ) ). Though these datasets vary in format and difficult, thought calibration is still able to reach up to a 20% reduction in thinking tokens , and in the worst case, is always as efficient as naive budget constraints . In summary, this work has three main contributions. 1. We interpret LLM reasoning through the lens of an abstract reasoning tree, where the problem of early exiting is equivalent to identifying when this tree stops growing. 2. This view allows us to calibrate the decision rule for actively terminating generation. 3. Based on multiple language models and reasoning benchmarks, we provide empirical evidence that thought calibration is effective for efficient test-time scaling. 2 Background 2.1 Test-time scaling Efficient inference Since current reasoning models are post-trained through reinforcement learning ( Guo et al., 2025a ) , a number of works address the overthinking problem ( Sui et al., 2025 ) as part of the reinforcement learning process ( Han et al., 2024 ; Arora and Zanette, 2025 ; Hou et al., 2025 ) . Other works focus on the inference-time problem of predicting when a language model should stop generating ( Yang et al., 2025 ; Zhang et al., 2025 ; Ma et al., 2025 ) . This papers falls under the latter category, which on a whole, is compatible with methods that reduce a language model’s verbosity during post-training. Finally, another option is to achieve efficiency in terms of model architecture. Some works dynamically adapt compute cost ( Lei, 2021 ; Leviathan et al., 2023 ) , while others employ only a subset of all modules during sampling ( Kim and Cho, 2021 ; Liu et al., 2022 ; Schuster et al., 2022 ) . While these strategies operate over the Transformer stack, rather than the generation sequence length, many high-level ideas are broadly applicable to early exiting in our situation. Self-consistency Self-consistency has been widely used to provide a self-supervised form of confidence during the sampling process ( Wang et al., 2022 ) . These methods aim to improve the quality of generated samples, often in situations where multiple samples may be sequentially generated ( Mitchell et al., 2022 ; Madaan et al., 2023 ; Shi et al., 2023 ; Weng et al., 2023 ; Guo et al., 2025b ) . Consistency can also provide feedback for reasoning-focused reinforcement learning ( Wang et al., 2024b ) . Several recent works have observed that confidence scores can be probed and calibrated from internal representations, to prioritize reasoning trajectories for subsequent runs ( Li et al., 2024 ; Huang et al., 2025 ; Xie et al., 2024 ) or for early exiting, similar to this work ( Zhang et al., 2025 ) . Our key departure

Read the complete text