Closing the Confidence-Faithfulness Gap in Large Language Models
Miranda Muqing Miao, Lyle Ungar · 2026
Read Closing the Confidence-Faithfulness Gap in Large Language Models by Miranda Muqing Miao, Lyle Ungar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25052.txt.
Introduction
Miao and Ungar probe why a language model's stated confidence often misses its true accuracy. Probes and steering tests show two signals live inside the model. One tracks how likely an answer is right. The other drives the confidence the model says out loud. The two run at right angles, so they barely interact. Asking the model to reason while it scores itself warps the spoken signal and worsens the mismatch. Their two-stage steering reads the inner estimate and bends the spoken score to match it.
Catalog record
| CREATOR | Miranda Muqing Miao, Lyle Ungar |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41544 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.25052
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Closing the Confidence-Faithfulness Gap in Large Language Models Miranda Muqing Miao, Lyle Ungar Abstract Large language models (LLMs) tend to verbalize confidence scores that are largely detached from their actual accuracy, yet the geometric relationship governing this behavior remain poorly understood. In this work, we present a mechanistic interpretability analysis of verbalized confidence, using linear probes and contrastive activation addition (CAA) steering to show that calibration and verbalized confidence signals are encoded linearly but are orthogonal to one another — a finding consistent across three open-weight models and four datasets. Interestingly, when models are prompted to simultaneously reason through a problem and verbalize a confidence score, the reasoning process disrupts the verbalized confidence direction, exacerbating miscalibration. We term this the ”Reasoning Contamination Effect.” Leveraging this insight, we introduce a two-stage adaptive steering pipeline that reads the model’s internal accuracy estimate and steers verbalized output to match it, substantially improving calibration alignment across all evaluated models. 1 Introduction Large language models have shown to be systematically overconfident. This miscalibration primarily takes two forms: at the token level, where output probabilities are poorly calibrated despite high accuracy ( 8 ; 5 ) , and at the verbalized level , where models cluster their verbal confidence scores near the top of the range regardless of actual performance ( 18 ; 13 ; 28 ) . Instruction tuning and RLHF exacerbate the problem, compressing verbalized confidence even further toward high certainty ( 25 ; 15 ) . Of these two failure modes, verbalized confidence is particularly consequential for safe deployment. It is the primary natural language channel through which the average user receives uncertainty information. When a model tells a physician “I am 95% confident” about a diagnosis it answers correctly only 40% of the time, the downstream consequences can be catastrophic. We argue that verbalized miscalibration is not caused by a lack of internal knowledge but by a failure to read out signals that are already present. The information needed for faithful confidence statements exists in the residual stream; the generation process simply fails to use it. This understanding shifts the question from “how do we teach models to be calibrated?” to “how do we correct the readout?” A growing body of mechanistic-interpretability research has shown that high-level semantic and behavioral properties are encoded as linear directions in the residual stream. Linear probes recover truth and falsehood from internal activations ( 4 ; 20 ; 2 ) , and steering vectors along these directions causally shift model behavior at inference time for truthfulness ( 16 ) , broad behavioral traits ( 30 ; 27 ; 22 ) , and refusal ( 1 ) . Simultaneous work has begun extending this lens to verbalized confidence. 14 show that verbal confidence is cached at answer-adjacent positions and retrieved later. 23 identify “answer-independence” as a driver of overconfidence and propose a fine-tuning fix. These studies establish that verbalized confidence has a nontrivial internal presence, yet a core question remains unanswered: what is the geometric relationship between the model’s internal accuracy signal and its verbalized confidence, and can that relationship be leveraged to improve calibration? Existing methods for improving verbalized-confidence calibration treat the model as a black box. Prompt-engineering strategies elicit better-calibrated scores by asking models to consider top- K alternatives ( 25 ) or by aggregating across multiple response samples ( 28 ) . The most closely related prompting work, SteerConf ( 29 ) , shifts verbalized confidence through a range of cautious-to-confident prompt framings and aggregates the resulting scores. Training-based approaches fine-tune models to express calibrated scores using proper scoring rules ( 17 ) or RL reward shaping ( 3 ) . All of these methods manipulate the input or retrain the model without leveraging existing signals at the representational level. By contrast, our two-stage pipeline reads the model’s internal accuracy estimate and steers the output to match it, achieving substantially lower calibration error than both unsteered verbalized confidence and SteerConf across all evaluated models. Our main contributions are: • Geometric dissociation. Models encode well-calibrated accuracy information in a linearly accessible direction, but verbalized confidence occupies a separate, nearly orthogonal direction (cosine similarity <0.04 ). The model “knows” when it is likely wrong, but the generation process fails to surface this signal. • Reasoning contamination. When the model solves a problem and rates its confidence jointly, the confidence and accuracy directions shift from weakly aligned to sharply opposed (cosine similarity dropping from +0.26 to -0.63 ), meaning joint prompting actively inverts the relationship between what the model knows and what it says. • Steering-based calibration. Contrastive activation addition produces causally controlled shifts in verbalized confidence that generalize across datasets and transfer from base to instruction-tuned models. We introduce a two-stage adaptive steering pipeline that reads the model’s internal accuracy estimate and steers verbalized output to match, improving calibration by 4 – 7\times . 2 Method This section describes the two methodological tools that underpin our analysis. We first introduce gold calibration linear probing, which tests whether accuracy information is linearly accessible in the residual stream. We then describe contrastive activation steering, which constructs steering vectors that causally shift verbalized confidence at inference time. The remaining paragraphs detail the datasets, models, activation extraction procedure, and prompt design used throughout. 2.1 Gold Calibration Linear Probing To test whether calibration information is linearly accessible in model activations, we train ridge regression probes on extracted activation vectors. For gold calibration probing , we use activations from the pure correctness prompt and regress against binary correctness labels or binned empirical accuracy (the fraction of times the model answers a question correctly across 50 samples with different random seeds). We sweep over a broad range of \ell_{2} regularization strengths and select the value that maximizes validation performance. 2.2 Contrastive Activation Steering To move beyond correlation and establish a causal link between activation directions and verbalized confidence, we apply contrastive activation addition (CAA) ( 26 ) . We elicit the same set of questions under K=11 prompt framings that span a wide range of instructed confidence levels, collecting hidden-state activations \mathbf{h}_{q,k}^{(\ell)}\in\mathbb{R}^{d} at layer \ell for question q under framing k . Each instance is paired with its parsed verbalized confidence c_{q,k}\in[0,1] . The exact K prompts used are shown in the Appendix. We partition instances into a high-confidence set \mathcal{H}_{q}=\{k:c_{q,k}>\tau_{\mathrm{hi}}\} and a low-confidence set \mathcal{L}_{q}=\{k:c_{q,k}<\tau_{\mathrm{lo}}\} , with \tau_{\mathrm{hi}}=0.75 and \tau_{\mathrm{lo}}=0.25 . For each question q that contains at least one instance in both sets, we compute a per-question contrast: \boldsymbol{\delta}_{q}^{(\ell)}=\frac{1}{|\mathcal{H}_{q}|}\sum_{k\in\mathcal{H}_{q}}\mathbf{h}_{q,k}^{(\ell)}-\frac{1}{|\mathcal{L}_{q}|}\sum_{k\in\mathcal{L}_{q}}\mathbf{h}_{q,k}^{(\ell)}. (1) The steering vector is then obtained by averaging over all qualifying questions \mathcal{Q} : \mathbf{v}^{(\ell)}=\frac{1}{|\mathcal{Q}|}\sum_{q\in\mathcal{Q}}\boldsymbol{\delta}_{q}^{(\ell)}. (2) Because each \boldsymbol{\delta}_{q} is computed within a single question, this design controls for confounds such as question difficulty, topic, and prompt framing, isolating the component of the activation that varies specifically with expressed confidence. At inference time, we inject the steering vector into the residual stream during autoregressive generation. Let \mathbf{h}_{t}^{(\ell)} denote the hidden state at layer \ell and generation step t . The steered activation is: \tilde{\mathbf{h}}_{t}^{(\ell)}=\mathbf{h}_{t}^{(\ell)}+\alpha\,\hat{\mathbf{v}}^{(\ell)}, (3) where \hat{\mathbf{v}}^{(\ell)}=\frac{\mathbf{v}^{(\ell)}}{\|\mathbf{v}^{(\ell)}\|}\cdot\bar{n}^{(\ell)} is the steering vector normalized to unit length and rescaled by the mean activation norm \bar{n}^{(\ell)} at layer \ell , and \alpha\in\mathbb{R} controls steering strength. We evaluate three injection sites: the last prompt token