Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality
Xiaoyuan Zhu, Kimberly Le Truong, Riccardo Fogliato, Gokul Swamy, Weijian Zhang, Minglai Yang, Longtian Ye, Bangya Liu, Minghao Liu, Andrew Ilyas, Zhiwei Steven Wu, [0.5em] University of Southern California · 2026
Read Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality by Xiaoyuan Zhu, Kimberly Le Truong, Riccardo Fogliato, Gokul Swamy, Weijian Zhang, Minglai Yang, Longtian Ye, Bangya Liu, Minghao Liu, Andrew Ilyas, Zhiwei Steven Wu, [0.5em] University of Southern California (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-04418.txt.
Description
Zhu and colleagues define error verifiability: whether model justifications help users tell correct answers from wrong ones. The team offers the v-bal metric, testing whether reasons let raters assess correctness, checked against human raters showing high accord. Neither post-training and model scaling nor targeted interventions improve it. Two methods succeed: reflect-and-rephrase for mathematical reasoning and oracle-rephrase for factual QA, both adding domain-fitting external information. The results make error verifiability a distinct quality dimension, not arising from accuracy gains alone, needing focused, domain-aware methods.
Catalog record
| CREATOR | Xiaoyuan Zhu, Kimberly Le Truong, Riccardo Fogliato, Gokul Swamy, Weijian Zhang, Minglai Yang, Longtian Ye, Bangya Liu, Minghao Liu, Andrew Ilyas, Zhiwei Steven Wu, [0.5em] University of Southern California |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76288 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.04418