Deep Think with Confidence

Yichao Fu, Xuewei Wang, Yuandong Tian, Jiawei Zhao · 2025

Read Deep Think with Confidence by Yichao Fu, Xuewei Wang, Yuandong Tian, Jiawei Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15260.txt.

Introduction

Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao added a confidence filter to the way language models think in parallel, in a 2025 paper. A common trick samples many reasoning paths and takes a majority vote, but gains fade and costs soar. Their method reads the model's own confidence signals to weed out weak paths.

The method, DeepConf, can filter during or after generation and needs no retraining or tuning knobs. It drops into existing serving setups with ease. The team tested it on varied reasoning tasks with recent open models. On a hard 2025 math contest set, the top setting reached near-perfect accuracy. It also cut generated tokens by up to about 85 percent against full parallel thinking. The authors call it a simple lever for cheaper, sharper test-time reasoning.

Catalog record

CREATORYichao Fu, Xuewei Wang, Yuandong Tian, Jiawei Zhao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-15260

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Deep Think with Confidence Yichao Fu, Xuewei Wang, Yuandong Tian, Jiawei Zhao Abstract Large Language Models (LLMs) have shown great potential in reasoning tasks through test-time scaling methods like self-consistency with majority voting. However, this approach often leads to diminishing returns in accuracy and high computational overhead. To address these challenges, we introduce Deep Think with Confidence (DeepConf) , a simple yet powerful method that enhances both reasoning efficiency and performance at test time. DeepConf leverages model-internal confidence signals to dynamically filter out low-quality reasoning traces during or after generation. It requires no additional model training or hyperparameter tuning and can be seamlessly integrated into existing serving frameworks. We evaluate DeepConf across a variety of reasoning tasks and the latest open-source models, including Qwen 3 and GPT-OSS series. Notably, on challenging benchmarks such as AIME 2025, DeepConf@512 achieves up to 99.9% accuracy and reduces generated tokens by up to 84.7% compared to full parallel thinking. 1 1 footnotetext: Email: [email protected] , [email protected] Figure 1: Up: DeepConf on AIME 2025. Down: Parallel thinking using DeepConf. 1 Introduction Large Language Models (LLMs) have demonstrated exceptional reasoning capabilities, particularly when equipped with methods that enhance their performance during test-time inference. A prominent technique is self-consistency , which samples multiple reasoning paths and aggregates final answers through majority voting ( Wang et al. 2023 ) . This type of approach, also known as parallel thinking , significantly improves reasoning accuracy but incurs substantial computational overhead: generating numerous reasoning traces per query scales inference overhead linearly, limiting practical deployment ( Xue et al. 2023 ) . For example, improving pass@1 accuracy from 68% to 82% using standard majority voting on AIME 2025 requires 511 additional reasoning traces per question using Qwen3-8B, consuming 100 million additional tokens. Moreover, parallel thinking with majority voting exhibits diminishing returns —performance often saturates or degrades as the number of traces increase ( Chen et al. 2024a ) . A key limitation is that standard majority voting treats all reasoning traces equally, ignoring quality variations ( Pal et al. 2024 ; Wang et al. 2025 ) . This can lead to suboptimal performance when low-quality traces dominate the voting process. Recent work has leveraged next-token distribution statistics to assess reasoning trace quality ( Geng et al. 2024 ; Fadeeva et al. 2024 ; Kang et al. 2025 ) . Higher prediction confidence typically correlates with lower entropy and reduced uncertainty. By aggregating token-level statistics such as entropy and confidence scores, existing methods compute global confidence measures across an entire trace to identify and filter low-quality traces to improve majority voting performance ( Kang et al. 2025 ) . However, global confidence measures present several limitations in practice. First, they may obscure confidence fluctuations at local reasoning steps, which can provide sufficient signals for estimating trace quality. Averaging across entire tokens in a trace can mask critical reasoning breakdowns that occur at specific intermediate steps. Second, global confidence measures require generating complete reasoning traces before they can be calculated, which prevents early stopping of low-quality traces. We introduce Deep Think with Confidence (DeepConf) , a simple yet effective test-time method that combines parallel thinking with confidence-aware filtering, based on local confidence measurements. DeepConf operates in both offline and online modes, identifying and discarding low-confidence reasoning traces either during or after generation. This approach reduces unnecessary token generation while maintaining or improving final answer accuracy. We evaluate DeepConf across multiple reasoning benchmarks (AIME 2024/2025, HMMT 2025, BRUMO25, GPQA-Diamond) and models (DeepSeek-8B, Qwen3-8B/32B, GPT-OSS-20B/120B). Through extensive experiments averaged across 64 repetitions per setting, we demonstrate that DeepConf achieves superior reasoning performance while requiring significantly fewer generated tokens compared to standard majority voting. In offline mode with access to all reasoning traces, DeepConf@512 achieves 99.9% accuracy on AIME 2025 using GPT-OSS-120B (no tools), saturating this benchmark compared to 97.0% for cons@512 (majority voting) and 91.8% for pass@1. In online mode with real-time generation control, DeepConf reduces token generation by up to 84.7% compared to standard parallel thinking while maintaining or exceeding accuracy. Fig. 1 highlights our key results. 2 Confidence as an Indicator of Reasoning Quality Recent work has demonstrated that reasoning trace quality can be effectively estimated using metrics derived from the model’s internal token distributions Kang et al. 2025 . These metrics provide model-intrinsic signals for distinguishing high-quality reasoning trajectories from erroneous ones without requiring external supervision. Token Entropy. Given a language model’s predicted token distribution P_{i} at position i , the token entropy is defined as: H_{i}=-\sum_{j}P_{i}(j)\log P_{i}(j), (1) where P_{i}(j) represents the probability of the j -th vocabulary token. Low entropy indicates a peaked distribution with high model certainty, while high entropy reflects uncertainty in the prediction. Token Confidence. We define token confidence C_{i} as the negative average log-probability of the top- k tokens at position i : C_{i}=-\frac{1}{k}\sum_{j=1}^{k}\log P_{i}(j), (2) where k denotes the number of top tokens considered. High confidence corresponds to peaked distributions and greater model certainty, while low confidence indicates uncertainty in token prediction. Average Trace Confidence. Token-level metrics require aggregation to assess entire reasoning traces. Following Kang et al. 2025 , we employ average trace confidence (also termed self-certainty) as a trace-level quality measure: C_{\text{avg}}=\frac{1}{N}\sum_{i=1}^{N}C_{i}, (3) where N is the total number of generated tokens. As demonstrated in Fig. 2 , average trace confidence effectively distinguishes between correct and incorrect reasoning paths, with higher values indicating greater likelihood of correctness. Despite its effectiveness, average trace confidence has notable limitations. First, global aggregation obscures intermediate reasoning failures: a few high-confidence tokens can mask numerous low-confidence segments, potentially hiding critical errors. Second, this approach requires complete traces for quality assessment, preventing early termination of low-quality generations and resulting in computational inefficiency. Figure 2: Confidence distributions for correct vs. incorrect reasoning traces across different metrics. Data from HMMT25: 30 problems, 4096 traces each. 3 Deep Think with Confidence In this section, we present how to leverage confidence metrics more effectively to improve both reasoning performance and thinking efficiency. We target two primary scenarios: offline and online thinking. Offline thinking leverages confidence to enhance reasoning performance by evaluating and aggregating information from completed reasoning traces. Online thinking incorporates confidence during token generation to improve reasoning performance and/or computational efficiency in real-time. 3.1 Confidence Measurements To address the limitations of global confidence measures like self-certainty, we introduce several alternative confidence measurements that capture local intermediate step quality and provide more fine-grained assessment of reasoning traces. Group Confidence. We quantify the confidence of intermediate reasoning steps using group confidence . Group confidence provides a more localized and smoother signal by averaging token confidence over overlapping spans of the reasoning trace. Each token is associated with a sliding window group G_{i} consisting of n previous tokens (e.g., n=1024 or 2048 ) with overlapping adjacent windows. For each group G_{i} , group confidence is defined as: C_{G_{i}}=\frac{1}{|G_{i}|}\sum_{t\in G_{i}}C_{t}, (4) where |G_{i}| is the number of tokens in group G_{i} . Estimating reasoning trace quality requires aggregating signals from group confidence. We observe that intermediate steps with extremely low confidence in a trace can significantly affect final solution correctness. For instance, when confidence drops sharply during reasoning with repeated low-confidence tokens like "wait", "however", and "think again", it disrupts reasoning flow and leads to subsequent

Read the complete text