Adaptive Conformal Prediction for Improving Factuality of Generations by Large Language Models

Aleksandr Rubashevskii, Dzianis Piatrashyn, Preslav Nakov & Maxim Panov · 2026

Read Adaptive Conformal Prediction for Improving Factuality of Generations by Large Language Models by Aleksandr Rubashevskii, Dzianis Piatrashyn, Preslav Nakov & Maxim Panov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13991.txt.

Introduction

Rubashevskii and colleagues make conformal prediction adapt to each prompt. Standard methods miss prompt-to-prompt shifts. They filter too little for some prompts and too much for others. The new approach tunes scores per prompt. It keeps total coverage and improves conditional coverage. It also supports picking which claims to trust downstream.

Catalog record

CREATORAleksandr Rubashevskii, Dzianis Piatrashyn, Preslav Nakov & Maxim Panov
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36338
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13991

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adaptive Conformal Prediction for Improving Factuality of Generations by Large Language Models Aleksandr Rubashevskii, Dzianis Piatrashyn, Preslav Nakov & Maxim Panov Abstract Large language models (LLMs) are prone to generating factually incorrect outputs. Recent work has applied conformal prediction to provide uncertainty estimates and statistical guarantees for the factuality of LLM generations. However, existing approaches are typically not prompt-adaptive, limiting their ability to capture input-dependent variability. As a result, they may filter out too few items (leading to over-coverage) or too many (under-coverage) for a given task or prompt. We propose an adaptive conformal prediction approach that extends conformal score transformation methods to LLMs, with applications to long-form generation and multiple-choice question answering. This enables prompt-dependent calibration, retaining marginal coverage guarantees while improving conditional coverage. In addition, the approach naturally supports selective prediction, allowing unreliable claims or answer choices to be filtered out in downstream applications. We evaluate our approach on multiple white-box models across diverse domains and show that it significantly outperforms existing baselines in terms of conditional coverage. 1 Introduction Large language models (LLMs) have demonstrated impressive performance across diverse applications ( 30 ; 18 ) . Despite this progress, they are still susceptible to hallucinations, producing fluent but factually incorrect outputs ( 12 ) . This limitation is especially concerning in high-risk domains such as medicine, where even a few errors within extended generations can lead to significant consequences ( 27 ) . To mitigate these risks, it is essential to develop methods with rigorous reliability guarantees. Conformal prediction offers a theoretically grounded approach to uncertainty quantification, providing distribution-free guarantees on error rates ( 28 ; 2 ) . Conformal prediction has recently been applied to large language models in tasks such as long-form question answering ( 19 ) and multi-choice QA ( 15 ) . In these settings, conformal methods are typically used to construct prediction sets or filtering rules based on uncertainty scores, enabling selective prediction: the model either returns only high-confidence outputs or abstains from uncertain ones. For example, in long-form generation, individual claims or spans can be filtered based on their estimated reliability, while in multiple-choice settings, conformal prediction produces a subset of candidate answers guaranteed to contain the correct one with high probability. However, existing conformal procedures for LLMs lack adaptivity: a single calibrated quantile is applied uniformly across all test prompts, regardless of their difficulty, ambiguity, or rarity. While this guarantees marginal coverage on average, it can lead to substantial miscalibration at the prompt level ( 3 ) . For certain inputs, the method may exhibit over-coverage (an overly conservative threshold), whereas for others it may result in under-coverage (an insufficiently strict threshold). We propose an adaptive conformal prediction approach for evaluating the factuality of large language models that accounts for the characteristics and difficulty of specific tasks. The proposed methodology is evaluated across multiple domains using various models and uncertainty quantification techniques. Figure 1 illustrates the main result of our work: standard conformal methods fail to achieve category-wise (conditional) coverage for heterogeneous prompts, whereas our adaptive approach improves conditional coverage while preserving marginal guarantees (see Section 3 ). Our contributions are as follows: 1. We propose a new conformal prediction approach for hallucination detection in LLMs that learns a prompt-adaptive correction to conformity scores via embedding-conditioned quantile regression. 2. We show that our method preserves the finite-sample marginal coverage guaranties of split conformal prediction while improving conditional coverage across heterogeneous prompts. 3. Experiments on long-form and multiple-choice question answering benchmarks across multiple LLMs show improved hallucination detection performance and more stable coverage compared to existing conformal methods. 2 Methodology 2.1 Background (a) (b) Figure 1: Long-form QA target vs. empirical coverage for (a) Conformal Factuality and (b) Adaptive Conformal (ours). The conformal procedure is calibrated jointly on two categories (“Inventions” and “Persons”). While both methods achieve the target coverage marginally, only the adaptive approach closely approximates conditional coverage. Conformal prediction assumes exchangeable data \{X_{i},Y_{i}\}_{i=1}^{N+1} with input features X_{i} and output labels Y_{i} , and a user-specified miscoverage level \alpha . Using calibration data \{X_{i},Y_{i}\}_{i=1}^{N} it constructs a prediction set C_{\alpha}(X) such that for a new test point \{X_{N+1},Y_{N+1}\} : \mathbb{P}\bigl(Y_{N+1}\in\mathcal{C}_{\alpha}(X_{N+1})\bigr)\geq 1-\alpha. (1) This guarantee is marginal coverage, meaning the coverage holds on average over the distribution of X_{N+1} . Usually, it assumed that some predictive model \widehat{f}(x) was constructed that models the dependence between x and y . Let V(x,y) be a nonconformity score function, where larger values indicate worse agreement between \widehat{f}(x) and y . Using a calibration set \{X_{i},Y_{i}\}_{i=1}^{N} , define the calibration scores v_{i}:=V(X_{i},Y_{i}),i=1,\ldots,N . Then the conformal prediction set is \mathcal{C}_{\alpha}(x)=\left\{y\in\mathcal{Y}\colon V(x,y)\leq Q_{1-\alpha}\left(\sum_{i=1}^{N}\frac{\delta_{v_{i}}}{N+1}+\frac{\delta_{\infty}}{N+1}\right)\right\}, (2) where Q_{1-\alpha} denotes the (1-\alpha) -quantile of a distribution, \delta_{v} is a Dirac mass at v . In the next section, we show how conformal prediction can be adapted to ensure factuality of LLM generations. 2.2 Conformal Prediction for LLMs LLMs typically generate free-form text rather than structured outputs. To enable fine-grained factuality assessment, a popular approach is to decompose generated responses into atomic, verifiable claims. For example, the response “Paris is the capital of France and was founded in the 3rd century BC” can be split into claims such as (i) “Paris is the capital of France” and (ii) “Paris was founded in the 3rd century BC”. Let an LLM for a long-form QA task produce a finite set of candidate claims from input x : L(x)=\{c_{1},\ldots,c_{m}\}\subset\mathcal{C} , where each c_{i} is a verifiable atomic claim. Given a claim-level score s\colon\mathcal{C}\to\mathbb{R} measuring uncertainty, define the filtered output at threshold t as F_{t}\bigl(x,L(x)\bigr):=\{c\in L(x)\colon s(c)\leq t\}. (3) Intuitively, F_{t} retains only sufficiently low-uncertainty (i.e., confident) claims. Accordingly, the filtered set of claims produced by a large language model can be interpreted as a conformal prediction set, such as C_{\alpha} in equation ( 2 ), as it restricts the output space to claims whose uncertainty scores do not exceed a calibrated threshold. Let w\colon\mathcal{C}\times\mathcal{Y}\to\mathbb{R} be a claim-level factuality function (e.g., based on a pre-trained Natural Language Inference (NLI) model) that evaluates whether a claim is supported by the reference. We distinguish this from an uncertainty score s(c) , which provides a model-based estimate of how likely a claim is to be incorrect and is used to rank and filter claims. In contrast, w(c,y) serves as an oracle that determines whether a claim is factually correct with respect to the ground-truth answer y . An illustrative example is provided in Appendix C.3 . For the long-form QA setting, we define the score V(x,y) as the largest uncertainty threshold such that all retained claims are factually correct: V(x,y)=\sup\left\{t\colon\forall c\in F_{t}(x,L(x)),\;w(c,y)\geq\beta\right\}, (4) where \beta is a fixed task-dependent factuality threshold defining claim correctness. We compute the conformal threshold as the (1-\alpha) -quantile of the scores \{v_{i}=V(x_{i},y_{i})\}_{i=1}^{N} on a calibration set. At test time, the final claim set is obtained by filtering according to the uncertainty scores: \bar{L}_{\alpha}(x)=\left\{c\in L(x)\colon s(c)\leq Q_{1-\alpha}\left(\sum_{i=1}^{N}\frac{\delta_{v_{i}}}{N+1}+\frac{\delta_{\infty}}{N+1}\right)\right\}. (5) This procedure ensures the following marginal coverage guarantee: \mathbb{P}\Bigl(c\in\bar{L}_{\alpha}(X_{N+1})\colon w(c,Y_{N+1})\geq\beta\Bigr)\geq 1-\alpha. (6) The condition w(c,Y)\geq\beta plays a role analogous to the membership test Y\in C_{\alpha}(X) in classical conformal prediction, defining whether a retained prediction is correct; see equation ( 1 ). 19 propose a related mechanism for long-form question answering using entailment-based sets. Multiple-Choice QA

Read the complete text