SelfCheck-Eval: A multi-module framework for zero-resource hallucination detection in large language models
Diyana Muhammed, Giusy Giulia Tuccari, Gollam Rabby, Sören Auer, Sahar Vahdati · 2026
Read SelfCheck-Eval: A multi-module framework for zero-resource hallucination detection in large language models by Diyana Muhammed, Giusy Giulia Tuccari, Gollam Rabby, Sören Auer, Sahar Vahdati (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13280721.txt.
Description
Diyana Muhammed and colleagues published a 2026 study of SelfCheck-Eval, a framework for catching made-up answers from large language models without needing reference data or model internals. Hallucination is an obstacle to using these systems in medicine, finance, or education. Existing detectors lean on general questions, so the team built a test set from an American mathematics examination, where wrong reasoning can sound convincing. Their framework combines three checks covering meaning, specialized checks, and consistency with context.
Evaluation exposed a sharp split. Detectors handled life stories well but stumbled on math reasoning errors. The weakness persisted across three improvement strategies: fine-tuning on inference data, preference learning, and guided reasoning steps. The authors conclude the failure is structural rather than a matter of more tuning. They argue for specialized black-box detectors for any model, open or closed, before such systems can be trusted in high-stakes work.
Catalog record
| CREATOR | Diyana Muhammed, Giusy Giulia Tuccari, Gollam Rabby, Sören Auer, Sahar Vahdati |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |