Influential Training Data Retrieval for Explaining Verbalized Confidence of LLMs

Yuxi Xia, Loris Schoenegger, Benjamin Roth · 2026

Read Influential Training Data Retrieval for Explaining Verbalized Confidence of LLMs by Yuxi Xia, Loris Schoenegger, Benjamin Roth (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-10645.txt.

Introduction

Yuxi Xia, Loris Schoenegger, and Benjamin Roth study the training data behind a language model's stated confidence. Models often say how sure they are in words. Users may trust those words, yet models tend to sound more sure than they should be. Stated confidence often fails to match real accuracy.

The authors retrieve the training examples that most shaped a model's verbalized confidence. Finding those examples helps explain why a model sounds sure or unsure on a given answer. That kind of audit can guide fixes for overconfidence. It matters for any setting where users act on what a model claims to know.

Catalog record

CREATORYuxi Xia, Loris Schoenegger, Benjamin Roth
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-10645

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Influential Training Data Retrieval for Explaining Verbalized Confidence of LLMs Yuxi Xia, Loris Schoenegger, Benjamin Roth Abstract Large language models (LLMs) can increase users’ perceived trust by verbalizing confidence in their outputs. However, prior work has shown that LLMs are often overconfident, making their stated confidence unreliable since it does not consistently align with factual accuracy. To better understand the sources of this verbalized confidence, we introduce TracVC ( Trac ing V erbalized C onfidence), a method that builds on information retrieval and influence estimation to trace generated confidence expressions back to the training data. We evaluate TracVC on OLMo and Llama models in a question answering setting, proposing a new metric, content groundness, which measures the extent to which an LLM grounds its confidence in content-related training examples (relevant to the question and answer) versus in generic examples of confidence verbalization. Our analysis reveals that OLMo2-13B is frequently influenced by confidence-related data that is lexically unrelated to the query, suggesting that it may mimic superficial linguistic expressions of certainty rather than rely on genuine content grounding. These findings point to a fundamental limitation in current training regimes: LLMs may learn how to sound confident without learning when confidence is justified. Our analysis provides a foundation for improving LLMs’ trustworthiness in expressing more reliable confidence. 1 1 1 Source code is available at https://github.com/Yuuxii/training˙data˙confidence/ . 1 Introduction Verbalized confidence in large language models (LLMs) is increasingly used to estimate the certainty of their generated outputs to improve transparency and user trust [ 5 , 22 , 43 ] . However, recent studies have shown that LLMs frequently exhibit overconfidence, which is not aligned with factual accuracy, resulting in the poor reliability of their expressed confidence [ 34 , 47 , 48 , 53 ] . This finding leads to a foundational question: what drives confidence in LLMs, and do LLMs understand the intended meaning of expressing confidence? Figure 1 : The workflow of TracVC. We first retrieve the top 10 relevant samples for content (question ( q ) and answer ( a )) and confidence (prompt ( p ) and confidence ( c )) segment in the test sample. Then, we compute and compare the influence score regarding the confidence generation for content-related and confidence-related data samples. In this paper, we investigate the origins of verbalized confidence in LLMs by examining the role of training data. Specifically, we ask: Do LLMs ground their confidence in lexically relevant, content-related training examples, or are they more influenced by superficial, confidence-related cues? To address this question, we propose TracVC (shown in Fig. 1 ), a method for tracing verbalized confidence back to its influential training data. For each test instance, which consists of a content segment (question and LLM answer) and a verbalized confidence segment, TracVC retrieves two sets of top 10 relevant training examples: one that is lexically similar to the content and another that is similar to the confidence expression. Then, TracVC estimates the influence of the retrieved examples on the model’s confidence generation by adapting the gradient-based attribution method TracIn [ 37 ] . After comparing the influence of these two sets, TracVC reveals whether content-related or confidence-related training data plays a greater role in shaping the model’s verbalized confidence. We apply TracVC to 11 open-source LLMs with publicly available training corpora, including OLMo [ 36 ] , Llama [ 13 ] instruction models, and their corresponding checkpoints trained with different post-training techniques (e.g., direct preference optimization [ 38 ] ). Those models are evaluated on five question answering benchmarks. To quantify the extent to which content-related training data outweighs confidence-related data in shaping verbalized confidence, we introduce content groundness as a new metric, defined as the proportion of cases where content-related examples dominate over confidence-related ones. Our results demonstrate that: 1. LLMs, especially the OLMo2-13B model, can be influenced by confidence-related training examples that are lexically unrelated to the question, often latching onto keywords like “confidence” regardless of context. This suggests that they do not fully ground their confidence in content-related information, but instead learn to mimic linguistic markers of certainty. 2. Larger LLMs do not demonstrate higher content groundness than smaller ones. We hypothesize that larger models, due to their higher capacity, may be more sensitive to stylistic or superficial patterns in training data. 3. Content groundness is higher when tested on samples that are all correctly answered by the evaluated LLM. We hypothesize that LLMs are likely to ground more on content-related examples if they have seen those examples during training. 4. Post-training techniques can impact content groundness in the opposite direction on different LLMs. These findings highlight a fundamental limitation in current training regimes: LLMs may learn to sound confident without understanding when confidence is warranted. Our work introduces a data-driven perspective of model confidence, offering insights that can guide future training approaches toward improving the trustworthiness of model confidence. 2 Related Work Trustworthiness of Verbalized Confidence in LLMs. Verbalized confidence offers a convenient way for users to gauge the trustworthiness of LLM outputs [ 12 , 43 , 48 ] . Empirical evidence suggests that moderate confidence expressions can foster higher user trust, satisfaction, and task performance than either high or low confidence levels [ 49 ] . However, recent studies show that LLMs often exhibit overconfidence in their answers [ 16 , 47 , 48 , 53 ] , a mismatch between expressed certainty and factual correctness that can undermine user trust. Explaining verbalized confidence is therefore crucial. Retrieving Influential Training Data for Explanation. Gradient-based influence estimation methods [ 3 , 7 , 20 , 2 , 25 ] provide an alternative to leave-one-out retraining for studying the effect of individual training examples on model behavior. Those methods can broadly be grouped into two categories: (1) First-order approaches, such as TracIn [ 37 ] , which measure gradient similarity between training examples and test predictions; and (2) Hessian-based approaches, such as DataInf [ 28 ] , are potentially more precise, however, while several adaptations make such methods more scalable [ 4 , 17 , 40 ] , they are still computationally prohibitive for large-scale LLMs. To address scalability, information retrieval-based influence estimation methods [ 11 , 31 ] , such as OLMoTRACE [ 31 ] , retrieve lexically similar training examples under the assumption that they are likely to influence predictions. While efficient, such methods rely on surface-level token overlap and do not leverage internal model states as gradient-based influence estimation methods. Other approaches [ 8 , 23 ] train models explicitly to report their influential data, but these require additional training and are unsuitable for our setting. Our proposed method, TracVC, utilizes both information retrieval techniques [ 11 ] and the gradient-similarity-based method TracIn [ 37 ] to analyze how training data influences LLMs’ confidence expression, which combines the strengths of both methods and thus is efficient to apply while considering internal model states. Explaining Verbalized Confidence of LLMs. Existing work has primarily examined verbalized confidence through the lens of its alignment with internal model probabilities [ 26 , 34 ] or with human confidence [ 42 ] ; others [ 48 , 50 , 52 ] investigate prompting strategies or calibration techniques

Read the complete text