HalluEntity : Benchmarking and Understanding Entity-Level Hallucination Detection
Min-Hsuan Yeh [email protected], Max Kamachee [email protected], Seongheon Park [email protected], Yixuan Li [email protected] · 2025
Read HalluEntity : Benchmarking and Understanding Entity-Level Hallucination Detection by Min-Hsuan Yeh [email protected], Max Kamachee [email protected], Seongheon Park [email protected], Yixuan Li [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11948.txt.
Introduction
A language model can slip one invented name into an otherwise accurate page, and sentence-level detectors miss the forgery. Min-Hsuan Yeh and co-authors build a benchmark that hunts at the level of single entities. Their dataset, HalluEntity, marks which specific names and facts in a text are hallucinated. They then test uncertainty-based detection methods across seventeen modern language models.
Methods that judge single-token probabilities tend to cry wolf, flagging too much. Context-aware methods do better but remain far from reliable. A close qualitative study links hallucination habits to linguistic patterns in the output. The findings point toward detectors that weigh whole contexts, not isolated tokens. For long texts that mix truth and invention, entity-level checking is where the real problem lives.
Catalog record
| CREATOR | Min-Hsuan Yeh [email protected], Max Kamachee [email protected], Seongheon Park [email protected], Yixuan Li [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-11948
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HalluEntity : Benchmarking and Understanding Entity-Level Hallucination Detection Min-Hsuan Yeh [email protected], Max Kamachee [email protected], Seongheon Park [email protected], Yixuan Li [email protected] Abstract To mitigate the impact of hallucination nature of LLMs, many studies propose detecting hallucinated generation through uncertainty estimation. However, these approaches predominantly operate at the sentence or paragraph level, failing to pinpoint specific spans or entities responsible for hallucinated content. This lack of granularity is especially problematic for long-form outputs that mix accurate and fabricated information. To address this limitation, we explore entity-level hallucination detection . We propose a new data set, HalluEntity , which annotates hallucination at the entity level. Based on the dataset, we comprehensively evaluate uncertainty-based hallucination detection approaches across 17 modern LLMs. Our experimental results show that uncertainty estimation approaches focusing on individual token probabilities tend to over-predict hallucinations, while context-aware methods show better but still suboptimal performance. Through an in-depth qualitative study, we identify relationships between hallucination tendencies and linguistic properties and highlight important directions for future research. HalluEntity : https://huggingface.co/datasets/samuelyeh/HalluEntity Reviewed on OpenReview: https://openreview.net/forum?id=494k7e9R5D 1 Introduction How can we trust the facts generated by large language models (LLMs) when even a single hallucinated entity can distort an entire narrative? While LLMs have revolutionized text generation in various domains, from summarization to scientific writing ( Liang2024MappingTI ) , their tendency to produce hallucinations—factually incorrect or unsupported content—remains a critical challenge ( Xu2024HallucinationII ) . This issue is particularly concerning in high-stakes applications, such as medical diagnostics ( Chen2024EvaluatingLL ) , legal document drafting ( lin2024legaldocumentsdraftingfinetuned ) , and news generation ( odabaşı2025unravelingcapabilitieslanguagemodels ) , where inaccurate information can cause harm to individuals and erosion of public trust. Detecting hallucinations is therefore a critical step toward ensuring the responsible deployment of LLMs. Various approaches have been proposed to detect hallucinations ( Luo2024HallucinationDA ) , with uncertainty-based methods emerging as a promising direction ( Zhang2023EnhancingUH ) . However, current uncertainty-based hallucination detection approaches mainly operate at the sentence or paragraph level, classifying the entire generation as either hallucinated or correct. While this provides a coarse-grained assessment of factuality, it lacks the granularity needed to pinpoint which specific spans or entities contribute to hallucination. This limitation is particularly problematic for long-form text, where both accurate and hallucinated information frequently coexist. For example, a generated response about a historical event might correctly state the date but fabricate details about the individuals involved, necessitating finer-grained detection. To address these limitations, we present a first systematic exploration of entity-level hallucination detection by introducing a benchmark dataset, extensively evaluating uncertainty-based detection methods on this benchmark, and analyzing their strengths and limitations in identifying hallucinated entities. Specifically, we begin by constructing a benchmark for entity-level hallucination detection, HalluEntity , which fills in the critical blank for the field. Constructing such a dataset is challenging due to the labor-intensive nature of entity-level annotation, which requires annotators to segment meaningful entities and verify their factuality against reliable sources one by one. To overcome this, we develop a systematic pipeline that maps atomic facts from model-generated text to entity-level annotations, enabling structured hallucination detection at a finer granularity. HalluEntity encompasses 18,785 annotated entities, and provides a foundation for evaluating hallucination detection methods with greater interpretability and precision. Building on HalluEntity , we comprehensively evaluate the reliability of token-level uncertainty measurements in detecting hallucinated entities and their potential for localizing hallucinations within the generated text. Our evaluation broadly includes standard uncertainty estimators, such as token-level likelihood ( guerreiro-etal-2023-looking ) and entropy scores ( malinin2021uncertainty ) , as well as more advanced context-aware approaches that refine uncertainty estimation ( fadeeva-etal-2024-fact ; duan-etal-2024-shifting ; zhang-etal-2023-enhancing-uncertainty ) . By aggregating token-level uncertainty to the entity level, we assess whether these methods can accurately distinguish hallucinated entities from factual ones. We experiment with 17 modern LLMs across different model families and capacities. The results reveal that methods solely relying on individual token probabilities (e.g., likelihood and entropy) tend to over-predict hallucinations, making them less reliable. In contrast, context-aware approaches ( duan-etal-2024-shifting ; zhang-etal-2023-enhancing-uncertainty ) demonstrate better overall performance in entity-level hallucination detection. Additionally, model family and size have a limited impact on performance, compared to the choice of uncertainty estimation method, emphasizing the need for improved uncertainty modeling. Through in-depth qualitative analysis, we further identify relationships between hallucination tendencies and linguistic properties, such as sentence positions and entity types. We found that calibrating uncertainty score with contextual information in the best-performing method ( zhang-etal-2023-enhancing-uncertainty ) helps reduce over-confidence tendencies in later sentence positions but can unintentionally penalize non-hallucinated content. We also found that some uncertainty scores can frequently assign high uncertainty to informative content like named entities. These observations highlight critical areas for future research, including better modeling of contextual dependencies to maintain balanced precision-recall trade-offs. Our key contributions are summarized as follows: 1. We propose an entity-level hallucination detection dataset, HalluEntity , which contains 18,785 annotated entities for ChatGPT-generated biographies. 2. We comprehensively evaluate uncertainty-based hallucination detection approaches across 17 LLMs on our proposed dataset. 3. We conduct an in-depth analysis to identify the strengths and weaknesses of current uncertainty-based approaches, and provide insight to design better uncertainty scores. 2 Related Work Uncertainty-based hallucination detection methods. Various approaches have been proposed to detect hallucinated content in LLM generation. Unlike other methods that require external knowledge sources for fact-checking ( gou2024critic ; chen-etal-2024-complex ; min-etal-2023-factscore ; huo2023retrieving ) , uncertainty-based approaches are reference-free and rely only on LLM internal states or behaviors to determine hallucination ( 10.1145/3703155 ; park2025steer ) . For instance, sampling-based approaches generate multiple responses and measure the diversity in meaning among them ( fomicheva-etal-2020-unsupervised ; kuhn2023semantic ; lin2024generating ) , while density-based approaches approximate the training data distribution and provide probabilities or unnormalized scores to assess how likely a generated response belongs to the distribution ( yoo-etal-2022-detection ; ren2023outofdistribution ; vazhentsev-etal-2023-hybrid ) . In this paper, we focus on uncertainty quantification methods that rely on token-level likelihood or entropy ( guerreiro-etal-2023-looking ; malinin2021uncertainty ) . Recent works have explored refining likelihood estimation by incorporating semantic relationships or reweighting token importance. For instance, Claim-Conditioned Probability (CCP) ( fadeeva-etal-2024-fact ) was introduced to recalculate likelihood according to semantical equivalence; while zhang-etal-2023-enhancing-uncertainty and duan-etal-2024-shifting adjust token weights to better convey meaning in uncertainty aggregation. Although these approaches leverage token-level information, they are typically evaluated at the sentence level, raising questions about their reliability . To address this, we conduct a comprehensive analysis of entity-level hallucination detection for finer-grained performance insights. Fine-grained hallucination detection benchmark. Most hallucination detection benchmarks are at the sentence or paragraph level. For example, CoQA ( reddy-etal-2019-coqa ) , TriviaQA ( joshi-etal-2017-triviaqa ) , TruthfulQA ( lin-etal-2022-truthfulqa ) , and HaluEval ( li-etal-2023-halueval ) . These benchmarks classify each generated response as either hallucinated or correct. However, instance-level detection cannot pinpoint specific hallucinated content, which is crucial for correcting misinformation ( cattan2024localizingfactualinconsistenciesattributable ) . This limitation becomes particularly problematic in long-form text, where a single response often combines supported and unsupported information, making binary quality judgments inadequate ( min-etal-2023-factscore ) . To address these challenges, recent works have advanced benchmarks for more granular hallucination detection. For example, min-etal-2023-factscore introduced FActScore , which decomposes LLM-generated text into atomic facts—short sentences conveying a single piece of information—for more