HALT-RAG: A Task-Adaptable Framework for Hallucination Detection with Calibrated NLI Ensembles and Abstention

Saumya Goswami, Siddharth Kurra · 2025

Read HALT-RAG: A Task-Adaptable Framework for Hallucination Detection with Calibrated NLI Ensembles and Abstention by Saumya Goswami, Siddharth Kurra (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-07475.txt.

Introduction

Language models sometimes state facts their source documents never supported, a failure called hallucination. Saumya Goswami and Siddharth Kurra present HALT-RAG in a 2025 research paper. HALT-RAG checks the outputs of retrieval-augmented generation systems, which answer questions using fetched documents. It flags statements that contradict or outrun the source text.

The system combines two ready-made inference models with light word-level signals. A small calibrated classifier, adapted per task, turns those signals into a hallucination probability. Training used a strict out-of-fold protocol to avoid leakage. On the HaluEval benchmark, scores were strong across summarization, question answering, and dialogue. Because its probabilities are well calibrated, the system can also abstain when unsure, a useful safety valve for deployed AI.

Catalog record

CREATORSaumya Goswami, Siddharth Kurra
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-07475

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HALT-RAG: A Task-Adaptable Framework for Hallucination Detection with Calibrated NLI Ensembles and Abstention Saumya Goswami, Siddharth Kurra Abstract Detecting content that contradicts or is unsupported by a given source text is a critical challenge for the safe deployment of generative language models. We introduce HALT-RAG , a post-hoc verification system designed to identify hallucinations in the outputs of Retrieval-Augmented Generation (RAG) pipelines. Our flexible and task-adaptable framework uses a universal feature set derived from an ensemble of two frozen, off-the-shelf Natural Language Inference (NLI) models and lightweight lexical signals. These features are used to train a simple, calibrated, and task-adapted meta-classifier. Using a rigorous 5-fold out-of-fold (OOF) training protocol to prevent data leakage and produce unbiased estimates, we evaluate our system on the HaluEval benchmark. By pairing our universal feature set with a lightweight, task-adapted classifier and a precision-constrained decision policy, HALT-RAG achieves strong OOF F 1 F_{1} -scores of 0.7756, 0.9786, and 0.7391 on the summarization, QA, and dialogue tasks, respectively. The system’s well-calibrated probabilities enable a practical abstention mechanism, providing a reliable tool for balancing model performance with safety requirements. 1 Introduction Modern generative NLP models often produce hallucinations : content that is not supported by the input source or external knowledge [ 2 , 3 ] . This problem is widespread; studies report that standard models for abstractive summarization on the XSum dataset produce factually consistent summaries only 20–30% of the time [ 9 ] . Detecting these inconsistencies is therefore critical for deploying safe and reliable NLP systems, especially for **Retrieval-Augmented Generation (RAG)** pipelines, where output quality depends entirely on the retrieved context. HALT-RAG is designed as a post-hoc verifier for such systems, operating on the source text and generated output without performing retrieval itself. However, existing metrics for factual consistency have clear limitations. Many are task-specific , such as NLI-based detectors like FactCC [ 6 ] and SummaC [ 7 ] for summarization. Others rely on heuristic or synthetically generated training data [ 6 , 8 ] , which can limit their applicability. Furthermore, many produce uncalibrated confidence scores that are hard to interpret [ 5 ] , making them a poor fit for applications that need a ”reject option” [ 1 ] . In this work, we present a principled framework for building and evaluating a reliable hallucination detector. Our system, HALT-RAG , is built on a universal feature set derived from a dual-NLI predictor ensemble and simple lexical signals. This combined signal is fed into a calibrated meta-classifier . By training with an out-of-fold (OOF) strategy and applying post-hoc calibration [ 10 , 13 ] , we obtain well-calibrated confidence estimates that enable precise, policy-driven decisions. Our main contributions are: 1. We demonstrate that an ensemble of frozen, off-the-shelf NLI models, combined with simple lexical statistics, forms a powerful and efficient universal feature set. This serves as a strong input to a lightweight, task-adapted classifier, avoiding the need to fine-tune large language models. 2. We introduce a principled evaluation protocol using out-of-fold prediction and post-hoc calibration to produce reliable, unbiased performance estimates suitable for deploying safety-critical systems. 3. We show that our framework, pairing a universal feature set with a task-adapted classifier and a precision-constrained thresholding policy (Precision \geq 0.70 ), achieves strong performance on the diverse tasks within the HaluEval benchmark. (a) Precision-Recall Curve (b) ROC Curve (c) Calibration Curve (ECE = 0.011) (d) Risk-Coverage Trade-off Figure 1 : Performance of HALT-RAG on the Summarization task. The markers on the PR/ROC curves show the operating point at the chosen threshold ( t^{*}=0.377 ). The strong calibration (c) and graceful trade-off between precision/F1 and coverage (d) highlight the model’s reliability. (a) Precision-Recall Curve (b) ROC Curve (c) Calibration Curve (ECE = 0.005) (d) Risk-Coverage Trade-off Figure 2 : Performance of HALT-RAG on the Question Answering task. The detector achieves near-perfect discrimination, with its chosen threshold ( t^{*}=0.395 ) operating at very high precision and recall. The extremely low ECE of 0.005 indicates excellent calibration. (a) Precision-Recall Curve (b) ROC Curve (c) Calibration Curve (ECE = 0.013) (d) Risk-Coverage Trade-off Figure 3 : Performance of HALT-RAG on the Dialogue task. This task is the most challenging due to conversational nuances, but the model still maintains strong, well-calibrated performance at its chosen threshold ( t^{*}=0.421 ). 2 Related Work 2.1 Hallucination in NLP Tasks Hallucination is a well-documented problem across several NLP domains. 2.1.1 Abstractive Summarization In abstractive summarization, models are highly prone to generating unfaithful content [ 9 , 2 ] . Maynez et al. [ 9 ] reported that on the XSum dataset, over 70% of single-sentence summaries contain hallucinations. Early work to detect these inconsistencies involved NLI-based detectors like FactCC [ 6 ] and SummaC [ 7 ] , which frame the problem as an entailment task. SummaC noted that prior models often failed because of a ”mismatch in input granularity” between sentence-level NLI datasets and document-level consistency checking [ 7 ] . At the same time, QA-based metrics like QAGS [ 11 ] and FEQA [ 2 ] have shown stronger correlation with human judgments than simple lexical overlap. 2.1.2 Knowledge-Grounded Dialogue In dialogue systems, a common failure is the production of ”unsupported utterances,” or hallucinations [ 3 ] . Dziri et al. found that many conversational datasets contain hallucinations and introduced FaithDial , a benchmark designed to mitigate this issue [ 3 ] . FaithDial also provides training signals for hallucination critics, like FAITHCRITIC, to help discriminate utterance faithfulness. Honovich et al. [ 4 ] proposed Q2 , a QA-inspired metric that compares answer spans using NLI instead of token matching, making it more robust to lexical variation. 2.1.3 Open-Domain Question Answering In QA, models need to know ”when to abstain from answering” to avoid providing incorrect information, especially under domain shift [ 5 ] . Kamath et al. [ 5 ] showed that policies based on simple softmax probabilities perform poorly because models are often overconfident on out-of-domain inputs , highlighting the need for robust calibration. 2.1.4 Hallucination Evaluation Benchmarks HaluEval [ 8 ] is a key recent benchmark that includes a ”large collection of generated and human-annotated hallucinated samples” for summarization, QA, and dialogue. Its multi-task, human-annotated nature makes it an ideal platform for evaluating general-purpose detectors like HALT-RAG. 2.2 Contemporary Approaches to Hallucination in RAG Recent work (late 2023-2024) has explored methods more deeply integrated into the RAG process itself. Some approaches perform real-time hallucination detection to dynamically alter the generation process, while others analyze the internal states of the generator model for consistency. Another line of research focuses on making RAG systems robust to the noisy or irrelevant context often returned by the retriever module. In contrast to these more integrated approaches, HALT-RAG is designed as a complementary, retrieval-agnostic, post-hoc verifier . Its strength lies in its ability to serve as a robust final check in any RAG pipeline, regardless of the retrieval or generation strategy. 2.3 Factual Verification and Selective Answering The ”reject option” allows a model to abstain when its confidence is low [ 5 ] . This idea, formalized by Chow (1970), converts potential misclassifications into rejections [ 1 ] . For this to work well, models must provide well-calibrated confidence estimates .

Read the complete text