Calibrate, Don’t Curate: Label-Efficient Estimation from Noisy LLM Judges

Yanran Li · 2026

Read Calibrate, Don’t Curate: Label-Efficient Estimation from Noisy LLM Judges by Yanran Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09702.txt.

Introduction

Li challenged the usual habit of keeping only the best judges when scoring language models. Multi-judge panels, including LLM judges and reward models, are normally curated down to the most accurate members. Li compared that accuracy-ranked selection with keeping the full panel, holding the scoring and calibration steps fixed, on four labeled pairwise-evaluation benchmarks.

The full panel won consistently for calibrated probabilistic scoring, which asks how sure the verdict should be, not just which answer wins. On RewardBench2, the full panel reached a negative log-likelihood of 0.006 against 0.013 for the top-five panel, roughly halving the calibration error. Even below-chance judges helped when their biases could be learned and their signals added something new. Keep weak judges, Li argues, if they can be calibrated.

Catalog record

CREATORYanran Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.09702

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Calibrate, Don’t Curate: Label-Efficient Estimation from Noisy LLM Judges Yanran Li Abstract Multi-judge evaluation is increasingly used to assess LLMs and reward models, and the prevailing heuristic is to curate: keep the most accurate judges and discard weaker ones. We show that this heuristic can reverse when the target is not point accuracy, but calibrated probabilistic evaluation from a labeled calibration set. Holding the aggregation and calibration procedures fixed, we compare accuracy-ranked top- k k judge selection with using the full judge panel. Across four labeled pairwise-evaluation benchmarks spanning LLM-as-judge and reward-model settings, the calibrated full panel consistently outperforms accuracy-based selection. On RewardBench2, retaining all judges achieves negative log-likelihood (NLL) of 0.006 0.006 versus 0.013 0.013 under top-5 selection, halving the calibration error. This advantage persists after judge-family deduplication and against stronger same-pipeline subset search. We explain this reversal with oracle analyses showing that the optimal calibrated risk under proper scoring rules cannot increase when additional judge signals are made available, and that even below-chance judges can be useful when their biases are learnable and their signals are non-redundant. The resulting operating principle is simple: in multi-judge evaluation with labeled calibration data, do not discard weak judges by accuracy alone; keep them when they are parseable, non-redundant, and calibratable. 1 Introduction Pairwise comparison by multiple judges is the dominant paradigm for evaluating large language models ( 9 ; 47 ) , training reward models ( 27 ) , and aggregating crowdsourced annotations ( 7 ) . These judges vary substantially in quality: LLM judges exhibit position and verbosity biases ( 47 ; 43 ) , and reward models range from near-random to near-expert ( 22 ) . The standard response is to curate : select a panel of the best judges and discard the rest. This is sensible when the goal is point accuracy ( 21 ; 23 ) . But accuracy is not the only goal. Increasingly, evaluation systems must answer not just which response is better? but how confident should we be? Many evaluation settings require reliable confidence estimates, not just accurate winners. In this regime, pruning weak judges is not always the right choice: after calibration, keeping more judges can outperform selecting only the most accurate ones. We study calibrated aggregation for labeled pairwise-comparison benchmarks, where held-out ground truth is available for calibration. These benchmarks, including JudgeBench ( 34 ) , RewardBench ( 22 ) , RewardBench2 ( 25 ) , and LLMBar ( 45 ) , raise a practical question: should we curate a small panel of high-accuracy judges, or calibrate the full panel? Across four benchmarks, calibrating the full panel usually outperforms top- k accuracy-based pruning and remains competitive with stronger pruning baselines. The lesson is not to keep every judge blindly, but to avoid discarding judges solely because their point accuracy looks weak. Why can weak judges help once calibration is available? The relevant question is not only whether a judge is accurate, but whether its votes carry learnable signal about the correct answer. A weak judge may still help estimate confidence: its disagreements can mark uncertain items, and a stable bias can be corrected or reversed. Section 3 studies this idea in an idealized setting where judge reliability is known; our experiments test whether the same principle remains useful when reliability must be estimated from limited calibration data. This interpretation also explains when inclusion helps most. Accuracy-based pruning removes judges that look weak as point predictors, but their disagreements can still help identify uncertain items. Full-panel calibration keeps this information and learns how much to trust it. The gain is largest when the added judges contain learnable signal, and smallest when the panel is already redundant. 1.1 Contributions 1. We present the first systematic evidence of a calibration reversal: full-panel calibration usually beats accuracy-based judge pruning in multi-judge LLM evaluation. 2. We explain why weak judges can help: point accuracy is not the only criterion; learnable bias and disagreement can improve probability estimation. 3. We stress-test inclusion against stronger curators, judge-family deduplication, and same-pipeline subset search; even the strongest curator does not improve on full-panel calibration. 4. We identify post-hoc calibration as the main practical lever and characterize the boundary cases where inclusion can fail. 2 Setup and Methods We consider K judges evaluating N labeled pairwise comparisons. Each judge k\in[K] produces a verdict y_{tk}\in\{0,1,-1\} for comparison t , where -1 denotes a missing verdict. A labeled calibration split provides ground-truth labels z_{t}\in\{0,1\} for fitting the aggregation and calibration maps; a disjoint evaluation split is used only for reporting. Our goal is to estimate a calibrated predictive probability \hat{p}_{t}=\hat{P}(z_{t}=1\mid\{y_{tk}\}_{k=1}^{K}) such that \mathbb{P}(z_{t}=1\mid\hat{p}_{t}=p)\approx p , and optionally to construct valid prediction sets with coverage at least 1-\alpha . The big picture is a four-step statistical pipeline: (1) choose a judge panel and aggregate its votes into a raw probability; (2) correct residual bias; (3) apply a distribution-level calibration map; and (4) optionally wrap the result in conformal prediction sets. Figure 1 summarizes the comparison between accuracy-based pruning and full-panel inclusion. The inclusion-vs-selection experiments hold the aggregation and calibration families fixed across the two arms, so the only changed object is the judge panel. The nested model space \mathcal{M}_{\omega}\subset\mathcal{M}_{\omega,\delta}\subset\mathcal{M}_{\omega,\delta,G} keeps this progression explicit: each step expands the post-hoc model class without increasing calibrated log-loss asymptotically (Theorem 2 ). Judge pool native + API/GPU Top- k pruning Keep all judges Aggregator Aggregator Calibrator Calibrator Calibrated probabilities Calibrated probabilities Held-out labels Evaluation NLL / Brier / Accuracy Selective prediction Curate Include Figure 1: Overview of the curation and full-panel calibration pipelines. The two arms use the same aggregation and calibration families; only the judge-panel choice changes. 2.1 Step 1: Heterogeneity-Aware Judge Aggregation ( \mathcal{M}_{\omega} ) Step 1 maps a selected panel’s verdict vector to a raw probability. Our default aggregator is an integrated Bayesian one-coin model, a closed-form implementation of the log-odds weighting principle in Proposition 2 . From calibration data, judge k receives a_{k}\sim\mathrm{Beta}(c_{k}+1,n_{k}-c_{k}+1) , where c_{k} is its number of correct calibration verdicts and n_{k} its number of non-missing calibration verdicts. Integrating over a_{k} gives P(z_{t}=1\mid\mathbf{y}_{t})=\sigma\!\left(\sum_{k\in J_{t}}\log\frac{\mathbb{E}[a_{k}^{y_{tk}}(1-a_{k})^{1-y_{tk}}]}{\mathbb{E}[a_{k}^{1-y_{tk}}(1-a_{k})^{y_{tk}}]}\right), (1) where J_{t} contains the non-missing judges for item t . Equivalently, judge k contributes \log(\alpha_{k}/\beta_{k}) for a positive verdict and \log(\beta_{k}/\alpha_{k}) for a negative verdict, with \alpha_{k}=c_{k}+1 and \beta_{k}=n_{k}-c_{k}+1 . Judges with uncertain accuracy have \alpha_{k}\approx\beta_{k} and therefore near-zero weight. Dawid–Skene expectation-maximization (EM) ( 11 ) fits the same role and behaves similarly after post-hoc calibration (Section 4.4 ; Appendix H.9 ). Curation happens before aggregation: top- k ranks judges on the calibration split by posterior mean accuracy \alpha_{k}/(\alpha_{k}+\beta_{k}) , keeps the first k , and then uses the same aggregator and calibrator as the all-judges arm. Full inclusion sets S=[K] . 2.2 Step 2: Residual Bias Correction ( \mathcal{M}_{\omega,\delta} ) Step 2 captures systematic bias left by the weighted aggregator with a low-capacity post-hoc map from the Step 1 score to the empirical label frequency: \text{logit}(\hat{p}_{t}^{\text{corr}})=a\cdot\text{logit}(\hat{p}_{t})+b, (2) where \hat{p}_{t} is the Step 1 posterior and (a,b) are fit by logistic regression on calibration labels. This Platt-style correction ( 28 ) captures global scale and shift bias. When item features \mathbf{x}_{t} are available, such as topic or judge agreement level, we allow the residual map to vary

Read the complete text