HALoGEN : Fantastic LLM Hallucinations and Where to Find Them
Abhilasha Ravichander, Shrusti Ghela, David Wadden, Yejin Choi, [7pt] University of Washington, Google, NVIDIA[3pt] https://halogen-hallucinations.github.io [5pt] {aravicha, yejin}@cs.washington.edu{shrustighela1, dave.wadden}@gmail.com · 2025
Read HALoGEN : Fantastic LLM Hallucinations and Where to Find Them by Abhilasha Ravichander, Shrusti Ghela, David Wadden, Yejin Choi, [7pt] University of Washington, Google, NVIDIA[3pt] https://halogen-hallucinations.github.io [5pt] {aravicha, yejin}@cs.washington.edu{shrustighela1, dave.wadden}@gmail.com (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-08292.txt.
Introduction
Ravichander, Ghela, Wadden, and Choi release a benchmark for catching language model hallucinations in twenty twenty-five. Models often state falsehoods fluently, but checking output by hand is slow and costly. Their benchmark, HALoGEN, automates verification. It holds nearly eleven thousand prompts across nine domains.
Domains include programming, scientific attribution, and summarisation. Automatic verifiers split each answer into atomic facts. Each fact is checked against a trusted knowledge source. The team scored about one hundred fifty thousand generations from fourteen models. Even the best models hallucinated heavily, up to eighty-six percent of facts in some domains. Errors are classed as misremembering, bad training data, or pure fabrication. The authors hope the tool enables principled study of the problem.
Catalog record
| CREATOR | Abhilasha Ravichander, Shrusti Ghela, David Wadden, Yejin Choi, [7pt] University of Washington, Google, NVIDIA[3pt] https://halogen-hallucinations.github.io [5pt] {aravicha, yejin}@cs.washington.edu{shrustighela1, dave.wadden}@gmail.com |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 91 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-08292
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HALoGEN : Fantastic LLM Hallucinations and Where to Find Them Abhilasha Ravichander, Shrusti Ghela, David Wadden, Yejin Choi, [7pt] University of Washington, Google, NVIDIA[3pt] https://halogen-hallucinations.github.io [5pt] {aravicha, yejin}@cs.washington.edu{shrustighela1, dave.wadden}@gmail.com Abstract Despite their impressive ability to generate high-quality and fluent text, generative large language models (LLMs) also produce hallucinations: statements that are misaligned with established world knowledge or provided input context. However, measuring hallucination can be challenging, as having humans verify model generations on-the-fly is both expensive and time-consuming. In this work, we release HALoGEN , a comprehensive hallucination benchmark consisting of: (1) 10,923 prompts for generative models spanning nine domains including programming, scientific attribution, and summarization, and (2) automatic high-precision verifiers for each use case that decompose LLM generations into atomic units, and verify each unit against a high-quality knowledge source. We use this framework to evaluate ∼ \sim 150,000 generations from 14 language models, finding that even the best-performing models are riddled with hallucinations (sometimes up to 86% of generated atomic facts depending on the domain). We further define a novel error classification for LLM hallucinations based on whether they likely stem from incorrect recollection of training data (Type A errors) , or incorrect knowledge in training data (Type B errors) , or are fabrication (Type C errors) . We hope our framework provides a foundation to enable the principled study of why generative models hallucinate , and advances the development of trustworthy large language models. 1 Introduction A practical challenge to deploying commercial large language models (LLMs) is their propensity to produce hallucinated output : facts that are not aligned with world knowledge, or with the input context provided by the user. LLM hallucinations can cause potential downstream harms for real-world users ( NIST 2023 ) . Yet, the reasons behind why models hallucinate are unknown. Worse, it is difficult to even measure the extent to which models hallucinate, due to the open-ended nature of model generations, and the associated time, effort, and cost of human verification. In this work we address these challenges by (1) creating a comprehensive benchmark over diverse domains to measure hallucination behavior in language models at scale, and (2) using this diverse benchmark to investigate potential sources of language model hallucination in a range of scenarios. To estimate the degree to which LLMs hallucinate, we introduce HALoGEN (evaluating Hal lucinations o f Gen erative Models), a large-scale evaluation suite to measure hallucination in long-form generations of LLMs (Figure 1). HALoGEN consists of prompts spanning nine use-cases, including tasks where a model response is expected ( response-based ) and tasks where a model is expected to abstain from answering ( refusal-based ). For each use case, we implement an automatic verifier that (1) decomposes a model generation into a series of meaningful atomic units specific to the use case, and (2) verifies the factuality of each atomic unit using external tools, programs, or LLM-based classifiers. Figure 1: Hallucination evaluation for code and citation generation, two of nine evaluation settings in HALoGEN . Given an input prompt, we decompose each model response by identifying verifiable atomic units : package imports and paper citations, respectively. Then, we verify each unit to determine whether the unit is factual or hallucinated . Finally, we classify hallucinated facts into one of three categories based on relationship to training data (§ 1 ). We evaluate the responses of 14 LLMs on this benchmark, spanning 150k model generations. Our experimental results show that even the best-performing LLM responses are riddled with hallucination errors, with hallucination scores ranging from 4% to 86% depending on the task for GPT-4 . Further, we find that no single domain is highly predictive of the extent to which models will hallucinate in other domains, highlighting the need for a diverse, multi-domain benchmark such as HALoGEN . We also find LLMs frequently hallucinate responses in scenarios where they should abstain, with even the best-performing model responding 29% of the time, highlighting the need to improve calibration ( Brahman et al. 2024 ) . Armed with the dataset we constructed of prompts and associated generations from several state-of-the-art language models, we trace back hallucinations to pretraining corpora. Through a series of case studies on the identified hallucinations, we isolate hallucinated atomic facts and assign error classes of the following types: • Type A: The correct fact was present in the pretraining data but the model still hallucinated. • Type B: An incorrect fact was in the training data, or the fact is taken out of context. • Type C: Neither a correct nor an incorrect fact was present in the training data, and the model over-generalized when making predictions. Our novel analysis of LLM hallucinations presents a nuanced picture. Model hallucinations do not seem to have a single isolated cause, but rather are likely to originate from a multitude of scenarios which vary across domains. For example, we find that for code-generation tasks, hallucinated software packages can often be found as-is within pretraining corpora ( Type B errors ), whereas for another task where the model hallucinates incorrect educational affiliations for US senators, the correct information is often available within the pretraining data ( Type A errors ). By providing a way to study diverse hallucination behavior in language models, and a framework for identifying the potential sources behind model hallucination, we hope to provide a systematic foundation for truthful LLMs. 2 Related Work The tendency of LLMs to generate unfactual content, or “hallucinate”, has been well-documented in recent surveys ( Zhang et al. 2023 ; Ji et al. 2022 ) . Hallucination detection Early hallucination detection work studied content-grounded tasks such as summarization ( Pagnoni et al. 2021a ) , simplification ( Devaraj et al. 2022b ) , and dialogue ( Dziri et al. 2022 ) . Techniques for these settings identify factual units in the model output, and compare each unit against the source text using entailment-based ( Maynez et al. 2020 ; Kryscinski et al. 2019 ) or QA-based ( Durmus et al. 2020 ) systems. More recently, a number of works have sought to detect hallucinations occurring in open-ended generation. Reference-based approaches evaluate LLMs against trusted reference sources like Wikipedia or web search ( Min et al. 2023 ; Chern et al. 2023 ; Mishra et al. 2024 ) . Prior works have similarly relied on web search to identify hallucinated citations ( Agrawal et al. 2023 ) . Reference-free approaches instead use an LLM itself to detect hallucinations, by comparing the consistency of model responses ( Manakul et al. 2023 ) or examining the model’s logits ( Varshney et al. 2023 ) . Hallucination benchmarks LLM hallucination benchmarks consist of a collection of prompts designed for their potential to lead to hallucinated model output. The accuracy of the model responses to each prompt are then evaluated, either using a more powerful LLM ( Lin et al. 2021b ) , by examining the likelihoods assigned to correct and incorrect completions ( Muhlgay et al. 2023 ) , or by human annotators ( Li et al. 2023 ) . A number of benchmarks are also available to assess LLM factual knowledge in knowledge base completion Mallen et al. 2022 ; Petroni et al. 2019 and