HalluMat: Detecting Hallucinations in LLM-Generated Materials Science Content Through Multi-Stage Verification

Bhanu Prakash Vangala, Sajid Mahmud, Pawan Neupane, Joel Selvaraj, Jianlin Cheng · 2025

Read HalluMat: Detecting Hallucinations in LLM-Generated Materials Science Content Through Multi-Stage Verification by Bhanu Prakash Vangala, Sajid Mahmud, Pawan Neupane, Joel Selvaraj, Jianlin Cheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-22396.txt.

Description

Bhanu Prakash Vangala and four co-authors build tools to catch language models fabricating materials science. Hallucinated facts in AI-generated research content threaten research integrity, and detection methods need benchmarks of their own. The team introduces HalluMatData, a benchmark dataset for grading hallucination spotting in materials science text, and HalluMatDetector, a multi-stage system that checks generated content several ways.

The detector combines intrinsic checking, lookup from multiple sources, contradiction graph analysis, and metric-based scoring. Hallucination levels varied widely across materials subdomains, with high-entropy alloy queries showing the most factual mismatch. The pipeline cut hallucination rates by about thirty percent versus standard model outputs. A new consistency score also measures how answers drift across reworded versions of the same question.

Catalog record

CREATORBhanu Prakash Vangala, Sajid Mahmud, Pawan Neupane, Joel Selvaraj, Jianlin Cheng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-22396

Browse the full library · About the Tower of Babel