FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMs

Albert Sawczyn, Jakub Binkowski, Denis Janiak, Bogdan Gabrys, Tomasz Kajdanowicz · 2025

Read FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMs by Albert Sawczyn, Jakub Binkowski, Denis Janiak, Bogdan Gabrys, Tomasz Kajdanowicz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17229.txt.

Introduction

Albert Sawczyn and four co-authors built a way to catch made-up facts in chatbot answers, fact by fact, in a 2025 paper. Most older checks work on whole sentences or passages, which can hide which single claim is false. Their method needs no outside source and no training data.

First, an answer is turned into a graph of simple facts, each a three-part claim. The model is then sampled many times, and facts that fail to recur earn a high falsehood score. The method matched top sentence-level checks while giving finer detail. Used to fix answers, it raised true content far more than a sentence-level rival. The team also shared a new test data set.

Catalog record

CREATORAlbert Sawczyn, Jakub Binkowski, Denis Janiak, Bogdan Gabrys, Tomasz Kajdanowicz
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-17229

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMs Albert Sawczyn, Jakub Binkowski, Denis Janiak, Bogdan Gabrys, Tomasz Kajdanowicz Abstract Large Language Models (LLMs) frequently generate hallucinated content, posing significant challenges for applications where factuality is crucial. While existing hallucination detection methods typically operate at the sentence level or passage level, we propose FactSelfCheck, a novel black-box sampling-based method that enables fine-grained fact-level detection. Our approach represents text as knowledge graphs consisting of facts in the form of triples. Through analyzing factual consistency across multiple LLM responses, we compute fine-grained hallucination scores without requiring external resources or training data. Our evaluation demonstrates that FactSelfCheck performs competitively with leading sentence-level sampling-based methods while providing more detailed insights. Most notably, our fact-level approach significantly improves hallucination correction, achieving a 35.5 % 35.5\% increase in factual content compared to the baseline, while sentence-level SelfCheckGPT yields only a 10.6 % 10.6\% improvement. The granular nature of our detection enables more precise identification and correction of hallucinated content. Additionally, we contribute a new dataset for evaluating sampling-based methods – FavaMultiSamples. 1 Introduction Large Language Models (LLMs) have gained significant attention from academia and industry recently. However, a major limitation of LLMs is their tendency to generate hallucinated information Farquhar et al. 2024 ; Huang et al. 2025 , posing significant challenges for applications where factual correctness is crucial, such as healthcare Sallam 2023 . Although numerous methods have been proposed to reduce hallucinations Zhang et al. 2023 , it is not possible to eliminate them, and LLMs will constantly hallucinate Lee 2023 ; Xu et al. 2024 . Therefore, there remains a critical need for reliable hallucination detection in LLM responses. Effective detection enables system interventions by either preventing the transmission of hallucinated content to users or facilitating its correction Zhang et al. 2023 . Previous approaches to hallucination detection have primarily focused on classifying hallucinations at either the passage or sentence level Huang et al. 2025 . While valuable, these approaches are limited in their granularity, as they do not provide detailed information about specific hallucinated facts. To address this limitation, we propose a novel method for hallucination detection that operates at the fact level, offering finer-grained analysis. In our approach, we define a fact as a triple consisting of a head, relation, and tail – a standard representation in knowledge graphs (e.g., ( Robert Smith , member of , The Cure )) Hamilton et al. 2017 . Our method provides more precise and actionable information by computing hallucination scores for individual facts than traditional passage-level or sentence-level classification approaches. Moreover, our method represents facts in a structured way through knowledge graphs (KGs), enabling more straightforward interpretation, processing, and verification of factual content Pan et al. 2024 . Figure 1: The pipeline of FactSelfCheck in two variants. For response p , entities \mathcal{E}_{p} and relations \mathcal{R}_{p} are extracted, followed by the construction of knowledge graphs KG_{p} , for which hallucination scores \mathcal{H}_{\text{fact}} are calculated. Samples’ entities \mathcal{E}_{S} and relations \mathcal{R}_{S} are created by merging \mathcal{E}_{p} and \mathcal{R}_{p} with entities and relations from KG_{p} . For each sample s , the knowledge graph KG_{s} is extracted. FactSelfCheck-KG assesses the consistency between a fact and all KG_{s} . FactSelfCheck-Text assesses the consistency between a fact and all s directly. To obtain sentence-level score \mathcal{H}_{\text{sentence}} , fact-level scores are aggregated, as indicated by dashed arrows . Our granular approach is motivated by two key observations. First, a single sentence can contain multiple facts, with the number of facts varying significantly across sentences, contexts, and domains. This variability makes it challenging to identify hallucinated aspects of generated output precisely when using sentence-level detection. Second, false information can be dispersed throughout a text, as a single fact may appear across multiple sentences. That can mislead the sentence-level detection, as the factuality of a sentence is dependent on the previous sentences Zhang et al. 2024a . These challenges become particularly acute when analyzing long texts generated by LLMs. Fine-grained fact-level detection provides a more precise understanding of text factuality than a sentence analysis. It enables better assessment of content reliability and, as we show later, more effective factuality correction. We propose FactSelfCheck, a black-box method for fact-level hallucination detection, meaning it does not require access to the model’s internal parameters. This design choice makes our approach universally applicable across any LLM, including closed models, like GPT OpenAI et al. 2024 . Following a sampling-based detection paradigm, introduced by Manakul et al. 2023 , our method utilizes multiple response generations and analyzes the factual consistency of extracted facts across these samples. This paradigm is based on the phenomenon that factual information remains largely consistent across different generations, while hallucinated content tends to vary or contradict itself between samples Manakul et al. 2023 ; Wang et al. 2023 . This way, we can effectively identify hallucinated facts without relying on external resources (zero-resource) or access to the model’s internal parameters (black-box). Moreover, our method is non-parametric, as it does not require any training, making it easy to implement in any domain without the need for training data. The FactSelfCheck pipeline consists of three main steps: knowledge graph extraction, which extracts sets of facts from the initial response and samples; fact-level hallucination scoring; and calculating sentence-level scores by aggregating fact-level scores. We evaluated our method using the WikiBio GPT-3 Hallucination Dataset Manakul et al. 2023 and the FavaMultiSamples dataset, a new dataset for evaluating sampling-based methods. We performed sentence-level evaluation by aggregating fact-level scores. Our approach achieves performance comparable to leading sampling-based methods while providing more detailed information about hallucinations. Additionally, we demonstrate that our fact-level approach significantly improves hallucination correction. Compared to a baseline, providing incorrect facts to the correction method leads to a 35.5\% increase in factual content, while passing incorrect sentences leads to an 10.6\% increase. Our key contributions are as follows: 1. The novel black-box sampling-based method for fact-level hallucination detection – FactSelfCheck. It enables fine-grained hallucination detection in LLM responses without requiring training data or external resources, as it is both non-parametric and zero-resource. We propose two effective approaches for measuring factual consistency across multiple samples: FactSelfCheck-KG using knowledge graph comparisons and FactSelfCheck-Text using direct text comparison. 2. The FavaMultiSamples dataset, a novel dataset for evaluating sampling-based methods. 3. Comprehensive evaluation of our method, which shows competitive performance with leading sampling-based methods while providing more detailed insights. 4. Demonstration that fact-level detection significantly improves hallucination correction compared to sentence-level approaches. Our code is available on GitHub 1 1 1 github.com/graphml-lab-pwr/FactSelfCheck license: CC BY-SA 4.0 . The FavaMultiSamples dataset is available on Hugging Face 2 2 2 huggingface.co/datasets/graphml-lab-pwr/FavaMultiSamples license: CC BY-SA 4.0 . We publish all the code and data, allowing for the reproduction of the results. 2 Related work Xu et al. 2024 have proven that hallucinations are inevitable in LLMs. As LLMs are powerful tools, many recent studies have been conducted regarding hallucination mitigation and detection Zhang et al. 2023 ; Huang et al. 2025 . The detection methods can be divided into two groups: white-box and black-box. White-box methods analyze LLMs’ internal states ( Farquhar et al. 2024 ; Azaria and Mitchell 2023 ) . While these methods are universal across all LLMs, they often require multiple generations,

Read the complete text