HalluLens: LLM Hallucination Benchmark
Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, Pascale Fung · 2025
Read HalluLens: LLM Hallucination Benchmark by Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, Pascale Fung (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-17550.txt.
Introduction
Yejin Bang and colleagues set out HalluLens, a benchmark to gauge hallucination in large language models. Models often make replies that drift from user input or training data. That hurts trust and slows the uptake of generative AI. A large block to such tests is the lack of one shared frame. Definitions and sorts of hallucination have been out of step.
The team splits hallucination from factuality. They propose a clear taxonomy with two kinds. Intrinsic ones clash with the context that is given. Extrinsic ones clash with training knowledge. The benchmark joins new extrinsic tasks with past intrinsic tasks. Its dynamic test set limits data leakage. It keeps the benchmark robust as models evolve. The paper also studies past benchmarks and stresses their limits and saturation.
Catalog record
| CREATOR | Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, Pascale Fung |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 131 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-17550