Hallucination Inspector: A Fact-Checking Judge for API Migration Conference: The 30th International Conference on Evaluation and Assessment in Software Engineering; Tue 9 - Fri 12 June 2026; Glasgow, United Kingdom CCS: Software and its engineering CCS: Software and its engineering Software maintenance tools

Marcos Tileria, Santanu Kumar Dash, Profir-Petru Pâr t , achi, Earl T. Barr · 2026

Read Hallucination Inspector: A Fact-Checking Judge for API Migration Conference: The 30th International Conference on Evaluation and Assessment in Software Engineering; Tue 9 - Fri 12 June 2026; Glasgow, United Kingdom CCS: Software and its engineering CCS: Software and its engineering Software maintenance tools by Marcos Tileria, Santanu Kumar Dash, Profir-Petru Pâr t , achi, Earl T. Barr (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20202.txt.

Description

Tileria and three colleagues name a failure mode of language models: scaffolding hallucination, where models invent phantom symbols like imaginary imports, constructors, and constants. Standard metrics cannot reliably catch these invented calling contexts, so the authors propose Hallucination Inspector, a static analysis tool for the job. It checks tree-derived symbols against a knowledge base built directly from API documentation. A preliminary Android migration test shows it finds hallucinations while cutting false positives well below standard metrics and probabilistic judges.

Catalog record

CREATORMarcos Tileria, Santanu Kumar Dash, Profir-Petru Pâr t , achi, Earl T. Barr
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29659
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.20202

Browse the full library · About the Tower of Babel