Can LLMs Detect Their Own Hallucinations?
Sora Kadotani, Kosuke Nishida, Kyosuke Nishida · 2025
Read Can LLMs Detect Their Own Hallucinations? by Sora Kadotani, Kosuke Nishida, Kyosuke Nishida (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-11087.txt.
Description
Sora Kadotani, Kosuke Nishida, and Kyosuke Nishida ask whether a large language model can catch its own made-up facts. Such models write fluent text but sometimes state false claims with full confidence. Spotting those slips after the fact would make the models safer to use, if the skill to do so sits inside the model itself.
They frame the task as judging one sentence at a time and use step-by-step prompting to draw knowledge out of the model. In their tests, one widely used model with that prompting caught about fifty-eight percent of its own false claims. The authors conclude that detection works when the needed knowledge is stored in the model.
Catalog record
| CREATOR | Sora Kadotani, Kosuke Nishida, Kyosuke Nishida |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-11087