Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation
Bhaktipriya Radharapu, Eshika Saxena, Kenneth Li, Chenxi Whitehouse, Adina Williams, Nicola Cancedda · 2025
Read Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation by Bhaktipriya Radharapu, Eshika Saxena, Kenneth Li, Chenxi Whitehouse, Adina Williams, Nicola Cancedda (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-22245.txt.
Introduction
Bhaktipriya Radharapu and five co-authors make AI judges report how unsure they are, accurately and cheaply. Language models increasingly grade other models' work, but their stated confidence is often poorly calibrated, and sampling many answers to estimate uncertainty costs a lot of computation. The team's fix is a linear probe, a small readout trained on the judge's hidden states with a Brier-score loss.
Across objective tasks like reasoning, mathematics, factuality, and coding, plus subjective human preference judgments, the probes calibrated better than verbalized confidence or multi-generation methods at about a tenth of the compute. They generalized to unseen domains and were more accurate on their high-confidence calls. Their estimates run conservative, suiting safety-critical settings where false positives hurt most.
Catalog record
| CREATOR | Bhaktipriya Radharapu, Eshika Saxena, Kenneth Li, Chenxi Whitehouse, Adina Williams, Nicola Cancedda |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-22245
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation Bhaktipriya Radharapu, Eshika Saxena, Kenneth Li, Chenxi Whitehouse, Adina Williams, Nicola Cancedda Abstract As LLM-based judges become integral to industry applications, obtaining well-calibrated uncertainty estimates efficiently has become critical for production deployment. However, existing techniques, such as verbalized confidence and multi-generation methods, are often either poorly calibrated or computationally expensive. We introduce linear probes trained with a Brier score-based loss to provide calibrated uncertainty estimates from reasoning judges’ hidden states, requiring no additional model training. We evaluate our approach on both objective tasks (reasoning, mathematics, factuality, coding) and subjective human preference judgments. Our results demonstrate that probes achieve superior calibration compared to existing methods with ≈ 10 \approx 10 x computational savings, generalize robustly to unseen evaluation domains, and deliver higher accuracy on high-confidence predictions. However, probes produce conservative estimates that underperform on easier datasets but may benefit safety-critical deployments prioritizing low false-positive rates. Overall, our work demonstrates that interpretability-based uncertainty estimation provides a practical and scalable plug-and-play solution for LLM judges in production. 1 Introduction Figure 1: Calibration of models across model architectures, datasets, and uncertainty estimation methods as measured by the Kuiper metric. We compare four approaches across dense prompted judges (LLaMA 8B/70B, Qwen 32B), dense fine-tuned judges (J1 family), and MoE prompted judges (GPT-OSS 20B, LLaMA Scout). Our probe-based method outperforms baseline approaches across all architectures and training paradigms. Results averaged across all evaluation datasets. Lower values indicate better calibration. LLM-as-judge paradigm ( 74 ; 16 ; 62 ) has become ubiquitous in modern AI development, providing reward signals for alignment training (RLAIF) ( 6 ; 33 ; 29 ) and ranking models for deployment decisions ( 12 ) . With the rapid advancement of reasoning-capable models ( 47 ; 15 ) , LLM judges that generate reasoning traces before rendering verdicts are becoming increasingly prevalent due to their higher accuracy and transparency ( 67 ; 11 ) . Yet, current practice treats all judgments as equally reliable. Without calibrated confidence estimates, we cannot distinguish high-confidence judgments from cases where the LLM is essentially guessing. Moreover, LLM judges are known to be overconfident, systematically expressing higher confidence than their empirical accuracy supports ( 61 ; 24 ) . This leaves practitioners to either blindly trust all judgments or manually review everything (defeating the purpose of automation). This lack of uncertainty awareness is particularly problematic across judgment types: for correctness evaluation, we cannot identify objectively wrong judgments to exclude ( 76 ; 55 ; 64 ) ; for preference evaluation, we cannot detect genuinely ambiguous cases where humans may disagree or multiple valid answers exist ( 3 ; 50 ; 45 ; 57 ; 53 ) . Calibrated LLM judges, whose expressed confidence matches their empirical accuracy, enable systems to route straightforward cases to efficient models while reserving expensive judges or human reviewers for uncertain decisions ( 24 ; 10 ) , dramatically reducing computational and labor costs. Training processes become more efficient by down-weighting uncertain judgments, preventing noisy labels from causing reward hacking ( 19 ) and model collapse ( 72 ) . Across these applications, calibration serves complementary purposes: flagging likely errors in correctness tasks while preserving valid diversity in preference tasks. Calibration is thus essential for building reliable and trustworthy AI systems with LLM judges. In this work, our main contributions are: (1) A Brier-score–trained linear probe that produces calibrated uncertainty estimates from reasoning judges’ hidden states, requiring no additional model training or multi-sample generation. (2) Our probe achieves substantially better calibration than verbalized and multi-generation baselines across multiple model families (dense and MoE) and judging styles (prompted and fine-tuned), while requiring an order of magnitude less compute. (3) Strong out-of-distribution generalization to unseen benchmarks, with analysis of key trade-offs in probe’s performance. 2 Related Work 2.1 LLM Judges and Reasoning-Based Evaluation Large language model judges have become ubiquitous for evaluating model outputs across tasks ranging from pairwise preference ranking for RLHF ( 13 ; 49 ) to judging for correctness in verifiable tasks. These judges span a spectrum from prompted general-purpose models like GPT-4 ( 74 ; 16 ) to specialized fine-tuned judges ( 67 ; 77 ; 27 ; 28 ; 34 ) that reason before giving their verdict. While these judges achieve high accuracy, LLM judges are known to suffer from systematic overconfidence ( 24 ; 61 ; 70 ) . 2.2 Uncertainty Estimation in Large Language Models Uncertainty estimation methods for LLMs fall into several categories, each with distinct trade-offs: Logit-Based Methods. Token-level uncertainty estimation methods like Perplexity and Temperature Scaling ( 21 ) assume uniform calibration across tokens, ignoring the semantic and contextual nuances crucial for reasoning tasks ( 68 ) . Maximum Softmax Probability (MSP) ( 51 ) has been shown to be consistently miscalibrated in reasoning and multiple-choice QA tasks. Additionally, approaches such as contextual calibration ( 73 ) and batch calibration ( 75 ) operate on single-token output logits, making them inapplicable to reasoning judges that generate multi-token responses. Verbalized Confidence. Asking models directly for confidence scores ( 60 ; 25 ; 39 ) is straightforward, and has been argued to be better than logit-based methods. However, verbalized confidence produces overconfident estimates ( 61 ; 70 ; 59 ; 42 ) . Consistency-Based Methods. Self-consistency ( 65 ; 43 ) , semantic entropy ( 31 ) , and related approaches ( 9 ) achieve strong calibration by aggregating uncertainty across multiple generations. Methods such as prompt ensembles or model ensembles 22 ; 24 ; 61 involve perturbing prompts or aggregating uncertainty from different models. However, they incur substantial computational costs (typically 10–20 \times inference overhead), limiting practical deployment. Training-Based Approaches. Methods like fine-tuning the model for improved verbalized confidence ( 14 ; 37 ; 38 ) or specialized architectures ( 22 ; 26 ; 68 ) require significant computational overhead through additional training or architectural modifications. Interpretability-Based Approaches. Recent work has extracted uncertainty signals from internal representations ( 5 ; 78 ; 35 ; 30 ; 8 ; 56 ) , but focuses predominantly on hallucination detection in factual question-answering where models recall memorized knowledge—a fundamentally different setting from reasoning. Other methods 71 ; 7 ; 66 ; 23 ; 69 targetted at reasoning, store hidden states per layer and per intermediate reasoning step/token during inference, incurring significant computational overhead with longer reasoning chains. Moreover, these methods target uncertainty-based selective classification (distinguishing correct from incorrect predictions) rather than uncertainty calibration (aligning predicted uncertainty with actual correctness). These tasks are complementary, and performance on one does not imply the other ( 59 ) . Our work addresses these limitations with a plug-and-play approach: linear probes that achieve strong calibration for reasoning judges without requiring additional model training, multi-sample generation, or per-token state storage during inference, offering a practical and computationally efficient solution for deployment at scale. 3 Method Method LLAMA 8B LLAMA 70B QWEN 32B J1 QWEN 32B J1 LLAMA 8B J1 LLAMA 70B PPE Correctness Verbalized 0.182 / 0.174 0.135 / 0.131 0.107 / 0.108 0.103 / 0.105 0.219 / 0.213 0.076 / 0.074 Consistency 0.159 / 0.170 0.211 / 0.208 0.132 / 0.145 0.098 / 0.130 0.278 / 0.260 0.134 / 0.144 Majority 0.159 /