Towards Dependable Retrieval-Augmented Generation Using Factual Confidence Prediction

Florian Geissler, Francesco Carella, Laura Fieback, Jakob Spiegelberg · 2026

Read Towards Dependable Retrieval-Augmented Generation Using Factual Confidence Prediction by Florian Geissler, Francesco Carella, Laura Fieback, Jakob Spiegelberg (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05244.txt.

Introduction

Florian Geissler and three co-authors make retrieval-augmented generation safer by judging how factual its retrieved context and final answers really are. Retrieval can mislead the model with irrelevant passages, so users need a useful trust signal. Their two-stage approach first uses conformal prediction to keep only retrieved chunks likely to come from the correct source.

That filtering alone improved answer quality on some datasets, but its formal guarantees depend on the retriever setup, so the authors add checks for whether the setup fits. A second, attention-based scorer then rates whether the final answer is in line with the retrieved context. It caught mismatched answers with solid success in their tests.

Catalog record

CREATORFlorian Geissler, Francesco Carella, Laura Fieback, Jakob Spiegelberg
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.05244

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Dependable Retrieval-Augmented Generation Using Factual Confidence Prediction Florian Geissler, Francesco Carella, Laura Fieback, Jakob Spiegelberg Abstract Incorporating specific knowledge into large language models via retrieval-augmented generation (RAG) is a widespread technique that fuels many of today’s industry AI applications. A fundamental problem is to assess if the context retrieved by some similarity search provides indeed supporting facts, or instead misguides the generator with irrelevant information. It is critical to associate meaningful confidence measures about the factuality of the retrieval process with the generated answers. We present a new, two-staged approach to predict fact faithfulness of the output of retrieval-augmented generations. First, we employ conformal prediction to select only those retrieved chunks who have a high chance to come from the correct source. This approach in itself can improve answer quality by up to 6 % 6\% in some of the studied datasets, however, the associated statistical guarantees do not hold generally, since the assumption of sample exchangeability depends on the retriever setup. We present diagnostic metrics to assess whether a setup is suitable. Second, we quantify confidence in the consistency of a generated final answer with a given retrieved context, using an attention-based factuality classifier. This approach can detect inconsistent answers with a chance of up to 77 % 77\% . Our work helps to establish a novel type of certified RAG systems for a broad range of natural language industry applications. 1 Introduction Retrieval augmented generation (RAG) has become a de-facto standard to empower large language models (LLMs) with domain knowledge, such as a curated corpus of source documents. The leap to industry applications has long happened, as frameworks to set up RAG bots in a production environment have become simple and resource-efficient [ 1 , 19 ] . In modern agentic AI applications, RAG is commonly used as a subroutine [ 14 ] , which creates a dependency of the final outcome on the reliable operation of this mechanism. Certifiable guarantees of the RAG pipeline must be established, which serve as stepping stones in the safety assurance of the application. Figure 1 : Diagram of a generic RAG workflow, augmented by two certifications as proposed in this article. For the retrieval block 1 (blue), we ensure confidence in retrieving context from the original source. In the generation block 2 (grey), attention monitoring protects the LLM output against potential hallucinations based on certified context. Previous works on reliable RAG systems have focused on extending the basic RAG pipeline - see Fig. 1 - by various additional mechanisms to improve performance or robustness heuristically, or by analyzing and quantifying failure modes, see Sec. 2 . In this work, we pursue instead an architecture-agnostic certification protocol, that does not rely on modifications of individual components but is applicable to any RAG instantiation. By conformalizing the retrieval process, we construct certificates for the retrieval quality and analyze the ability of such certificates to generalize (see Sec. 3.2 ). Hallucinated generations from the certified chunks are identified by a detector trained on chunk attentions (Sec. 3.3 ). Our approach is not limited by the choice of retriever or open-source model, and can be combined with any modern, extended RAG workflow. In detail, we demonstrate \blacksquare a certifiable RAG pipeline featuring conformalized retrieval and chunk-based hallucination detection, \blacksquare an analysis of the pipeline’s performance on several data sets. Caveats for real world system designs are discussed, \blacksquare an analysis of the generalisation capability of the proposed method across various datasets, to validate empirical performance in real-world deployments. 2 Related work A major barrier for the adoption of LLMs in safety-critical applications is the possibility of factually incorrect answers, so-called hallucinations [ 4 , 18 ] . RAG models ground the available information in external sources [ 22 , 15 ] , but provide no factuality guarantees. Hallucinations originate typically from retrieved context, parametric knowledge, or conflicts between external context and internal knowledge [ 32 ] , for which different hallucination detection strategies have been introduced. These methods can be categorized into closed-book detection methods, which tackle parametric knowledge hallucinations, and open-book methods aiming to detect context hallucinations [ 32 ] . While closed-book detection methods rely on internal model representations [ 5 , 35 ] or consistency checks [ 26 ] , open-book methods measure the importance of context tokens during generation by leveraging attention maps [ 10 ] or comparing output distributions with and without external knowledge [ 21 ] . The factuality of RAG systems can further be improved via advanced processing pipelines, involving additional steps like self-verification [ 3 ] , knowledge consolidation, grading [ 33 , 16 , 12 ] , or uncertainty estimation [ 29 , 23 ] . Adaptive retrieval and risk-aware generation are strengthened by semantic entropy-guided retrieval [ 36 ] and counterfactual prompting to promote abstention under low confidence [ 9 ] . Recent works also explore how answer reliability can be boosted by agent consensus [ 8 ] . Lastly, security-focused evaluations highlight the need for pipeline-level defenses against adversarial attacks [ 24 ] . 3 Methodology 3.1 Concept A generic RAG workflow, as depicted in Fig. 1 , can be divided into two blocks: The first block retrieves relevant context from a database, typically a vector store. We conformalize scores and identify chunks with a statistically guarantee of relevance. The second block generates an answer for the user question, given the trusted context retrieved in the first block. To confide in the output generated by the LLM, it is key to understand whether the output was obtained based on the provided context and is free of hallucinations. We quantify this using intermediate attention values and a factuality classifier, as explained in Sec. 3.3 . 3.2 Retrieval confidence estimation Conformal prediction (CP) is a distribution-free uncertainty quantification method [ 31 , 6 , 2 ] based on the assumption of the exchangeability of data. In a first calibration step, a set of training data is reserved to calculate a non-conformity threshold with a user-given significance level. During inference, this threshold is used to form prediction sets that meet statistical guarantees. We here apply this principle to retrieve document chunks with guaranteed source fidelity. Calibration ground truth: Our datasets consist of triplets of questions (q) , reference answers (r) and document indices d(r) , where the respective answer is located. A retriever selects for a given question i the top K candidate chunks, where K is the retrieval depth. A retrieval score s is assigned to each retrieved chunk c , based on the estimated relevance of that chunk for the given query. This leads to a set of calibration scores S_{cal}=\{s_{i,j};\ i\in(1,\ldots N_{cal}),\ j\in(1,\ldots K)\} , with N_{cal} being the number of calibration samples. To facilitate thresholding, we perform a global normalization step to ensure s\in[0,1] . With n_{1}=\min(S_{cal}) and n_{2}=\max(S_{cal}) , we rescale S_{cal}\to(S_{cal}-n_{1})/(n_{2}-n_{1}) . To establish ground truth, we compare a chunk candidate c with the reference answer r for a given question, using a selected similarity metric h . The source document associated with the candidate chunk is d(c) . We first define a score threshold s\textsubscript{thres} as the \beta -percentile P_{\beta} of retrieval scores from those chunks that do not

Read the complete text