Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality

Xiaoyuan Zhu, Kimberly Le Truong, Riccardo Fogliato, Gokul Swamy, Weijian Zhang, Minglai Yang, Longtian Ye, Bangya Liu, Minghao Liu, Andrew Ilyas, Zhiwei Steven Wu, [0.5em] University of Southern California · 2026

Read Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality by Xiaoyuan Zhu, Kimberly Le Truong, Riccardo Fogliato, Gokul Swamy, Weijian Zhang, Minglai Yang, Longtian Ye, Bangya Liu, Minghao Liu, Andrew Ilyas, Zhiwei Steven Wu, [0.5em] University of Southern California (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-04418.txt.

Description

Zhu and colleagues define error verifiability: whether model justifications help users tell correct answers from wrong ones. The team offers the v-bal metric, testing whether reasons let raters assess correctness, checked against human raters showing high accord. Neither post-training and model scaling nor targeted interventions improve it. Two methods succeed: reflect-and-rephrase for mathematical reasoning and oracle-rephrase for factual QA, both adding domain-fitting external information. The results make error verifiability a distinct quality dimension, not arising from accuracy gains alone, needing focused, domain-aware methods.

Catalog record

CREATORXiaoyuan Zhu, Kimberly Le Truong, Riccardo Fogliato, Gokul Swamy, Weijian Zhang, Minglai Yang, Longtian Ye, Bangya Liu, Minghao Liu, Andrew Ilyas, Zhiwei Steven Wu, [0.5em] University of Southern California
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76288
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.04418

Browse the full library · About the Tower of Babel