Inference Time Optimization with Confidence Dynamics

Yu Wang, Minghao Liu, Jiayun Wang, Jinrui Huang, Ankit Shah, Wei Wei · 2026

Read Inference Time Optimization with Confidence Dynamics by Yu Wang, Minghao Liu, Jiayun Wang, Jinrui Huang, Ankit Shah, Wei Wei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25244.txt.

Introduction

Yu Wang and colleagues track how a language model's confidence moves as it reasons, and find a telling split. Correct solution paths tend to gain confidence step by step, while wrong ones flatten or fade. Their confidence-dynamic voting method weighs whole answers by that trajectory rather than by final confidence alone. Across four open model families and several contest benchmarks, the method beats standard repeated-sampling baselines, offering a sturdy signal for picking among candidate answers.

Catalog record

CREATORYu Wang, Minghao Liu, Jiayun Wang, Jinrui Huang, Ankit Shah, Wei Wei
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70405
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.25244

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Inference Time Optimization with Confidence Dynamics Yu Wang, Minghao Liu, Jiayun Wang, Jinrui Huang, Ankit Shah, Wei Wei Abstract Inference time optimization techniques, such as repeated sampling, have significantly advanced the reasoning capabilities of Large Language Models (LLMs). However, the critical role of model uncertainty remains largely underexplored in these optimization strategies. In this paper, we investigate the dynamics of confidence along reasoning trajectories and for first time reveal a surprising and unique pattern: correct answer traces tend to exhibit confidence improvement over time (positive confidence gain), while incorrect traces show attenuated or declining confidence as reasoning proceeds. Based on this observation, we propose Confidence Dynamic Gain (CDG) based voting, which incorporates how the confidence trajectory of the response evolves along the reasoning chain. Experiments across four open-source architectures (DeepSeek-R1, gpt-oss, Gemma-3, Qwen-QwQ) on the AIME24/25, HMMT25, and BRUMO25 benchmarks demonstrate that CDG yields a significant performance boost over baselines. These results demonstrate that our method provides a robust discriminative signal for improving answer selection in LLM reasoning. We also provide theoretical insights for this phenomenon. Code will be released at https://github.com/Accenture/CDG.git . 1 Introduction Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities across mathematical problem-solving, code generation, and scientific reasoning tasks ( 6 ; 30 ; 1 ; 8 ; 16 ; 33 ) . One of the key strategies for improving inference time reasoning accuracy is Best-of-N sampling, where multiple reasoning traces are generated for each given question, and an aggregation strategy selects the final answer ( 28 ; 21 ) . The simplest and most widely adopted approach is majority voting ( 28 ) , which selects the answer appearing most frequently among traces. Despite its simplicity and effectiveness, majority voting treats all traces equally regardless of their quality and uncertainty. Recent state-of-the-art work has explored the incorporation of model confidence into the voting process, using metrics such as sequence-level perplexity or average token probability to weight votes ( 13 ; 7 ) . However, these methods rely on static confidence measures that aggregate information across an entire trace, potentially obscuring valuable signals about how the model’s certainty evolves within reasoning chains. In this work, we investigate the dynamics of model confidence along the reasoning trajectory. Through extensive experiments across four mainstream LLM architectures—OpenAI gpt-oss ( 1 ) , DeepSeek-R1 ( 6 ) , Google Gemma ( 8 ) , and Qwen-QwQ ( 16 ) —we uncover a striking empirical pattern: correct reasoning traces exhibit systematically higher confidence gains from the beginning to the end of the trace compared to incorrect traces . Specifically, when we partition each trace into position-normalized bins and compare confidence levels between the initial (head) and final (tail) portions, correct traces show positive averaged confidence improvement at the end of the reasoning in comparison to the reasoning start, while incorrect traces display attenuated or negative end-start confidence difference. This observation leads us to ponder whether such a signal provides additional information to the existing confidence measures for sampling and why. To test our hypothesis, we propose Confidence Dynamic Gain (CDG) based Voting . Rather than relying solely on average confidence or answer frequency, CDG explicitly incorporates the confidence dynamic gain —the difference between answer’s tail and head confidence—as a discriminative side information for answer selection. Inspired by existing voting method ( 7 ; 13 ) , our answer voting function elegantly integrates three signals: (1) the count of traces for each answer, (2) the mean confidence across traces, and (3) confidence dynamic gain. Surprisingly, we observe that by simply amplifying such a confidence dynamic signal, CDG achieves strong performance boost compared to the state-of-the-art inference time answer selection approaches. While we are open to other legitimate explanations, we hypothesize that the training dynamics of Group Relative Policy Optimization (GRPO) ( 19 ) , a well-adopted algorithm to train LLMs, could potentially be the reason behind. GRPO has reinforced tokens proportional to their frequency within the group. In correct traces, while the initial reasoning paths are diverse (low frequency), the final answers strictly converge to the same ground truth (high frequency). In contrast, incorrect traces are subject to negative confidence suppression owing to diluted reasoning paths. With mild assumptions, we show that this concentration at the tail for correct traces drives a distinctively sharp rise in confidence—creating the head-to-tail gap that our method exploits at inference time. We hypothesize that the insights here generalize to any advantage-weighted policy gradient with verifiable reward. Our main contributions are: • We identify an empirical phenomenon: correct reasoning traces exhibit systematically higher confidence gains along the reasoning trajectories than incorrect traces across diverse LLM architectures. • We propose CDG, a novel voting method that leverages such confidence dynamics for improved answer selection, generalizing existing confidence-based approaches and majority voting. • We provide theoretical analysis explaining why GRPO-trained models exhibit such discriminative confidence gradients, connecting CDG to the verifiable reward and training policy behind. • We demonstrate empirical improvements on competitive mathematics benchmarks, showing that CDG outperforms state-of-the-art baselines. 2 Related Work Inference time scaling. Current LLMs increasingly succeed by allocating very large amounts of reasoning at inference, a paradigm we call inference time scaling ( 21 ) . Two major directions include: a) Increasing reasoning trajectory: previous work like Chain-of-Thought ( 29 ) proposes to scale reasoning steps by lengthening thinking steps in a single reasoning trajectory. Later models, such as DeepSeek-R1 ( 9 ) , Grok-4 ( 30 ) and Qwen3 ( 33 ) , rely on RL to facilitate this form of inference time scaling. b) Best-of-N setting (this paper’s focus): Parallel generation is scaled by increasing the number of trajectories and aggregating them. Self-Consistency ( 28 ) and Best-of-N ( 21 ; 13 ) sample multiple candidates and select via voting or a score. Some rely on reward models such as outcome reward models ( 5 ) and process reward models ( 25 ) . More efficient variants like Dynamic Voting ( 32 ) and ranked voting ( 27 ) , reduce the required sample count while preserving accuracy. Our proposed CDG aligns with this line of research, aiming to improve inference time accuracy through optimized answer selection. Position plays a role in LLM responses . Transformers do not inherently model order, so it is critical that positional information is injected through positional encodings (e.g., RoPE ( 22 ) and other absolute/relative schemes), which can directly shape attention allocation and, consequently, generation behavior ( 22 ; 20 ) . For example, these schemes often lead to position related structural biases like the “Attention Sink” phenomenon ( 31 ; 12 ) , where massive attention scores are disproportionately allocated to the first token regardless of relevance. Previous work treat these positional artifacts as mechanical features to be managed through architectural improvements ( 23 ; 15 ) . Beyond attention-level effects, token position also shapes how models utilize their context: 14 reports a “lost in the middle” accuracy curve in long-context question answering, with performance degrading sharply when relevant information is placed in the middle of the input, indicating that position systematically biases what LLMs attend to and reproduce. Others also study the position biases in multi-modality settings ( 24 ; 34 ) . Still,

Read the complete text