Why Uncertainty Estimation Methods Fall Short in RAG: An Axiomatic Analysis
Heydar Soudani, Evangelos Kanoulas, Faegheh Hasibi · 2025
Read Why Uncertainty Estimation Methods Fall Short in RAG: An Axiomatic Analysis by Heydar Soudani, Evangelos Kanoulas, Faegheh Hasibi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07459.txt.
Description
Heydar Soudani, Evangelos Kanoulas, and Faegheh Hasibi analyze confidence estimates in twenty twenty-five. Retrieval-augmented generation feeds outside documents into a language model's prompt. Uncertainty estimation should tell users how much to trust each answer. In the RAG setting, the authors show, current methods cannot reliably judge correctness.
They build an axiomatic framework to say exactly what goes wrong. Five constraints define what a good uncertainty method must do once documents join the prompt. Experiments reveal that no existing method satisfies all five axioms. That failure explains their weak performance in retrieval settings. Readers get design rules for the next generation of uncertainty tools, grounded in logic rather than trial and error.
Catalog record
| CREATOR | Heydar Soudani, Evangelos Kanoulas, Faegheh Hasibi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 103 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-07459