LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy
Hamed Karimi, Vaishali Meyappan, Reza Samavi · 2026
Read LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy by Hamed Karimi, Vaishali Meyappan, Reza Samavi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04295.txt.
Introduction
Hamed Karimi, Vaishali Meyappan, and Reza Samavi propose a way to measure how unsure a large language model is when it answers a question. Models can sound confident while inventing facts, which is risky in safety-critical uses, and many uncertainty scores ignore that different wordings can mean the same thing.
Their score groups several sampled answers by meaning and measures how widely those meanings spread, adjusting for features of each group. A calibration step then turns the score into an accept-or-abstain rule with a guaranteed bound on the error rate among accepted answers, checked in extensive experiments.
Catalog record
| CREATOR | Hamed Karimi, Vaishali Meyappan, Reza Samavi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.04295
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy Hamed Karimi, Vaishali Meyappan, Reza Samavi Abstract LLMs’ overconfidence, particularly when hallucinating, poses a significant challenge for the deployment of the models in safety-critical settings and makes a reliable estimation of uncertainty necessary. Existing approaches for uncertainty quantification typically prioritize lexical or probabilistic measures; however, these techniques often ignore the semantic variance of different responses with similar meaning. In this paper, we propose Adaptive Conformal Semantic Entropy (ACSE), a method for estimating prompt-level uncertainty by adaptively measuring semantic dispersion in LLMs outputs. Our uncertainty scoring function is based on clustering semantic entropy of multiple diverse responses to the same prompt. The function adaptively adjusts the uncertainty score based on semantic features of each cluster. To ensure statistical reliability of our score, we use conformal calibration to apply a decision rule to accept/abstain the prompts, providing a finite-sample, distribution-free guarantee such that the error rate among the accepted responses remains bounded by a user-specified tolerance. Our extensive experimental evaluations using different LLMs and datasets, demonstrate that our approach consistently outperforms state-of-the-art uncertainty quantification baselines using discriminative performance, conformal guarantees, and probabilistic calibration indicators. As a highlight, for TriviaQA dataset, AUROC of our approach is 0.88 0.88 compared to 0.65 0.65 produced by the token entropy approach. 1 Introduction Despite the wide range application of Large Language Models (LLMs), they remain uncertain in their predictions and often generate incorrect or misleading outputs with high confidence 30 . This overconfidence limits their safe deployment in high-stakes domains such as healthcare 34 , law 33 , and scientific research 25 . Reliable uncertainty quantification (UQ), defined as the process of estimating the model’s confidence in its own predictions, is therefore essential for improving model reliability, safety, and trustworthiness 6 . Related Work. Most existing methods for uncertainty estimation in LLMs rely on token-level signals, such as entropy over next-token distributions 7 ; 9 ; 41 or average sequence log-likelihood 40 ; 15 . While computationally efficient, these approaches capture only surface-level lexical variability rather than uncertainty over meaning 31 . In autoregressive generation, LLMs often assign high probability to multiple near-synonymous tokens, producing outputs that are probabilistically coherent yet semantically ambiguous or incorrect. As a result, token-level entropy cannot distinguish superficial phrasing variation from genuine semantic ambiguity. In particular, entropy may remain low even when probability mass is spread across responses with mutually inconsistent meanings, creating a misleading impression of confidence 4 ; 14 . More recent work has shifted toward semantic-level uncertainty estimation, notably Semantic Entropy (SE) 21 , which aggregates uncertainty by clustering semantically equivalent responses using Natural Language Inference. However, SE relies on hard cluster assignments, forcing each response into a single cluster and ignoring overlapping semantic regions, which can yield unstable uncertainty estimates. To improve reliability, conformalized approaches 20 employ dynamic clustering with finite-sample guarantees, while the Conformalized Abstention Policy 32 integrates reinforcement learning to learn instance-specific risk thresholds. Despite their guarantees, these methods treat uncertainty as a scalar signal and remain insensitive to cluster brittleness. As a result, they are vulnerable to the wrong-consensus trap, in which lexically consistent but factually incorrect response clusters satisfy conformal thresholds, masking underlying semantic error. Figure 1 : ACSE Pipeline. (a) To calibrate a pretrained LLM, for each prompt x\in\mathcal{D}_{cal} , we use an Adaptive Uncertainty Engine (AUE) to compute an adjusted uncertainty score, \hat{u}(x) . The \hat{u}(x) is then used to compute a cutoff for a conformalized decision rule (prompt-level) and prediction set in response-level for the user defined upper bound error guarantee of \alpha . (b) In inference time, for a new prompt, the same AUE is used to compute \hat{u}(x) to compare with the calibration cutoff. (c) AUE starts with sampling n responses, embeds them, and soft-clusters the embeddings, leading to the computation of base semantic entropy u(x) . Proposal. To address these limitations, we propose Adaptive Conformal Semantic Entropy (ACSE), a model-independent method that quantifies uncertainty in pretrained LLMs based on dispersion in meaning. Rather than relying on token-level signals, ACSE estimates prompt-level uncertainty in a continuous semantic vector space, capturing dispersion in response meaning. For a finite set of representative prompts (e.g., 1,000 calibration prompts), we sample multiple responses per prompt, embed each response using a sentence encoder whose cosine similarity reflects semantic proximity 29 , and cluster the embeddings into coherent semantic groups. We then estimate probability mass for each group via soft assignments and derive a base uncertainty score from the normalized entropy of the resulting distribution. This base semantic uncertainty reflects the degree of meaning dispersion: higher dispersion–corresponding to multiple semantically distinct responses–indicates greater uncertainty, whereas semantically consistent responses imply lower uncertainty 39 . We subsequently adjust the uncertainty score when cluster structure indicates brittleness, such as weak support for the dominant cluster, high internal diversity, or other risk patterns that empirically correlate with errors under distribution shift. Finally, we calibrate the adjusted uncertainty score using conformal prediction on held-out calibration prompts 37 to learn a cutoff that serves two purposes: (1) making prompt-level answer/abstain decisions and (2) generating response-level prediction sets to support user decision-making. Figure 1 provides an overview of the approach, which is illustrated by a toy example. Toy Example. Consider a user querying a pretrained geographical LLM with the prompt What is the capital of Australia? . The model returns Sydney with probability 70\% (high confidence) despite its semantic incorrectness, and Canberra with probability 30\% (low confidence). Selecting the most likely sequence yields a wrong consensus , a common failure mode in open-ended generation. Assume the user has calibrated the model on a held-out set of geographical prompts and identified 0.58 as a conformity cutoff corresponding to 90\% confidence under the assumption that new prompts are drawn from the same distribution 37 . For a new prompt, ACSE samples 10 responses using fixed decoding settings: 9 contain Sydney and 1 contains Canberra . These responses are embedded, clustered by meaning, and assigned soft semantic memberships via normalized cosine similarity to cluster centroids. Since assignments are probabilistic, clusters are non-disjoint. Averaging memberships across responses yields mean cluster weights of 0.87 for Sydney and 0.13 for Canberra . The resulting base semantic uncertainty score, given by the normalized entropy of these weights, is 0.55 . Because this score falls below the abstention threshold ( 0.55<0.58 ), the model would return the dominant Sydney cluster. However, semantic entropy alone can be overly optimistic when clusters are internally brittle, e.g., when cluster membership is unstable or weakly representative. Our approach accounts for such brittleness by adjusting the uncertainty score. In this example, incorporating cluster quality factors (Section 3.1 ) increases the score to 0.62 . Since 0.62>0.58 , the model abstains from returning the incorrect answer. While sampling and post-processing incur additional cost, this overhead is justified in safety-critical applications such as using LLMs for mental health support. Contributions. We make the following contributions. First, we propose ACSE, a method for estimating entropy-based uncertainty at the semantic level for a given prompt. Second, we introduce an adjusted uncertainty score that leverages cluster robustness features to adaptively inflate semantic uncertainty, explicitly penalizing ambiguous response semantics and mitigating hallucinations. Third, we incorporate a post-hoc conformal calibration phase on held-out