Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity
Prakhar Ganesh, Reza Shokri, Golnoosh Farnadi · 2026
Read Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity by Prakhar Ganesh, Reza Shokri, Golnoosh Farnadi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00723.txt.
Description
Prakhar Ganesh and colleagues rethink how false outputs from language models are judged, arguing that current tests focus only on correctness and miss agreement across prompts. They introduce prompt multiplicity, a way to measure agreement in tests. Their analysis finds large multiplicity, with over 50% mismatch in tests like Med-HALT, suggesting harms are badly misunderstood. For spotting and fixing, they find spotting methods spot agreement rather than correctness, while RAG can add new mismatches. Adding prompt multiplicity to tests gives a better picture of harms and reveals key limits.
Catalog record
| CREATOR | Prakhar Ganesh, Reza Shokri, Golnoosh Farnadi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61296 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.00723