UAQFact: Evaluating Factual Knowledge Utilization of LLMs on Unanswerable Questions
Chuanyuan Tan, Wenbiao Shao, Hao Xiong, Tong Zhu, Zhenhua Liu, Kai Shi, Wenliang Chen · 2025
Read UAQFact: Evaluating Factual Knowledge Utilization of LLMs on Unanswerable Questions by Chuanyuan Tan, Wenbiao Shao, Hao Xiong, Tong Zhu, Zhenhua Liu, Kai Shi, Wenliang Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-23461.txt.
Description
Tan and six co-authors build a test for how language models handle questions that cannot be answered, in this two thousand twenty-five paper. A model that guesses anyway can mislead, so knowing when to stop matters, and older test sets gave models no facts to lean on.
Their bilingual dataset, UAQFact, pairs unanswerable questions with supporting facts drawn from a knowledge graph. Two new tasks check whether models use facts they already know and facts handed to them. Across several model families, performance stayed shaky even when the needed knowledge was present, and adding external facts helped only partly. The benchmark exposes a gap between having knowledge and using it under uncertainty.
Catalog record
| CREATOR | Chuanyuan Tan, Wenbiao Shao, Hao Xiong, Tong Zhu, Zhenhua Liu, Kai Shi, Wenliang Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-23461