URAG: A Benchmark for Uncertainty Quantification in Retrieval-Augmented Large Language Models

Vinh Nguyen, Cuong Dang, Jiahao Zhang, Hoa Tran, Minh Tran, Trinh Chau, Thai Le, Lu Cheng, Suhang Wang · 2026

Read URAG: A Benchmark for Uncertainty Quantification in Retrieval-Augmented Large Language Models by Vinh Nguyen, Cuong Dang, Jiahao Zhang, Hoa Tran, Minh Tran, Trinh Chau, Thai Le, Lu Cheng, Suhang Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19281.txt.

Description

Nguyen and co-authors build URAG, a benchmark for how unsure retrieval-based language models are. Most tests grade these systems on right answers alone, not on trust. URAG turns open tasks into choice questions across health, code, science, math, and general text. That form lets conformal prediction put a firm size on doubt. Eight standard methods are scored on accuracy and set size. Gains in accuracy often pair with lower doubt, but noisy search breaks the link. No single method wins across all fields.

Catalog record

CREATORVinh Nguyen, Cuong Dang, Jiahao Zhang, Hoa Tran, Minh Tran, Trinh Chau, Thai Le, Lu Cheng, Suhang Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94125
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.19281

Browse the full library · About the Tower of Babel