KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge

Alex Robertson, Huizhi Liang, Mahbub Gani, Rohit Kumar, Srijith Rajamohan · 2026

Read KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge by Alex Robertson, Huizhi Liang, Mahbub Gani, Rohit Kumar, Srijith Rajamohan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19643.txt.

Description

Large language models write smooth, convincing prose, yet fluency is not truth: answers often hide quiet hallucinations. Alex Robertson and four co-authors present KGHaluBench, a knowledge-graph hallucination benchmark that tests models across knowledge breadth and depth. It builds hard, many-sided questions from the graph on the fly, sizing difficulty with statistics to fight popularity bias.

An automated check pipeline spots abstentions and grades answers at both concept and correctness levels, sorting hallucination types. Trials on 25 frontier models with new accuracy and hallucination metrics show which knowledge factors drive hallucinations across model sizes. Readers get a public benchmark built for fairer, fuller truthfulness testing and future mitigation work.

Catalog record

CREATORAlex Robertson, Huizhi Liang, Mahbub Gani, Rohit Kumar, Srijith Rajamohan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-19643

Browse the full library · About the Tower of Babel