Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations
Keyu He, Tejas Srinivasan, Brihi Joshi, Xiang Ren, Jesse Thomason, Swabha Swayamdipta · 2025
Read Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations by Keyu He, Tejas Srinivasan, Brihi Joshi, Xiang Ren, Jesse Thomason, Swabha Swayamdipta (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-25844.txt.
Introduction
Keyu He and co-authors propose in 2025 quality scores for explanations from vision-language models, aimed at users who cannot see the image. Blind and low-vision users lean on text accounts to judge model answers. Past work shows smooth explanations can make wrong answers sound right, which breeds over-trust.
Two scores grade each explanation on faith to the image and on how well it names telling details. Both scores track true correctness better than past quality grades on three task sets. In a user study, showing the scores raised accuracy in judging answers by about eleven percent. False trust in wrong answers fell by a wider margin.
Catalog record
| CREATOR | Keyu He, Tejas Srinivasan, Brihi Joshi, Xiang Ren, Jesse Thomason, Swabha Swayamdipta |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-25844
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations Keyu He, Tejas Srinivasan, Brihi Joshi, Xiang Ren, Jesse Thomason, Swabha Swayamdipta Abstract When people query Vision-Language Models (VLMs) but cannot see the accompanying visual context (e.g. for blind and low-vision users), augmenting VLM predictions with natural language explanations can signal which model predictions are reliable. However, prior work has found that explanations can easily convince users that inaccurate VLM predictions are correct. To remedy undesirable overreliance on VLM predictions, we propose evaluating two complementary qualities of VLM-generated explanations via two quality scoring functions. We propose Visual Fidelity , which captures how faithful an explanation is to the visual context, and Contrastiveness , which captures how well the explanation identifies visual details that distinguish the model’s prediction from plausible alternatives. On the A-OKVQA and VizWiz tasks, these quality scoring functions are better calibrated with model correctness than existing explanation qualities. We conduct a user study in which participants have to decide whether a VLM prediction is accurate without viewing its visual context. We observe that showing our quality scores alongside VLM explanations improves participants’ accuracy at predicting VLM correctness by 11.1%, including a 15.4% reduction in the rate of falsely believing incorrect predictions. These findings highlight the utility of explanation quality scores in fostering appropriate reliance on VLM predictions. 1 Introduction Vision-Language Models (VLMs) are being deployed in applications where users who do not have access to the VLM’s visual context; for example, in assisting blind and low-vision individuals Huh et al. 2024 ; An et al. 2025 ; Kim et al. 2025 , autonomous multimodal digital agents Koh et al. 2024 , and in human-robot collaboration Lukin et al. 2018 . However, despite recent advances in VLM capabilities, they often exhibit unreliable behavior, including hallucinating visual details Li et al. 2023 ; Gunjal et al. 2024 and making overconfident predictions Valdenegro-Toro 2024 . In scenarios where users cannot directly observe the visual context of the VLM, it becomes imperative to enable users to accurately trust model outputs. How can we provide adequate context for users to establish appropriate reliance on model predictions and explanations? Prior work has explored the utility of model explanations to support user decision making Wang & Yin 2021 ; Bansal et al. 2021 . However, natural language explanations can be misleading for inaccurate model predictions Joshi et al. 2023 ; Chaleshtori et al. 2024 ; Si et al. 2024a ; Sieker et al. 2024 . In Figure 1 , we consider the question “ What period of the day does this photo reflect? ” where a VLM (incorrectly) answers “ Noon ” and generates a plausible explanation referencing cues like “ a clock on the building ” or “ lighting and shadows .” This reasoning may appear highly convincing to a user without access to the image, even though the prediction is incorrect. Users are particularly susceptible to such overreliance on explanations in scenarios with such information asymmetry. Figure 1: VLM explanations that sound plausible can mislead users. Contextualizing VLM explanations with quality scores may help users know when to rely on VLM outputs, but existing explanation qualities are not calibrated with VLM prediction accuracy. We propose evaluating two new qualities of VLM explanations: Visual Fidelity and Contrastiveness. These qualities are better calibrated with VLM correctness, and also help users make better decisions about when to believe VLM predictions. Existing explanation metrics Hase et al. 2020 ; Chen et al. 2023 ; Liu et al. 2023b are designed to evaluate explanations generated by text-only LLMs, and we find these are poorly calibrated to VLM prediction accuracy. However, considering the explanation with respect to the accompanying visual context can reveal the reliability of the VLM prediction; for instance, identifying hallucinated details in the explanation ( “shadows suggesting that the sun is at its peak” ) or that the explanation does not mention the critical detail for identifying the correct answer (the time on the clock). We propose evaluating two new qualities of VLM-generated explanations: Visual Fidelity —how faithfully a VLM explanation reflects the accompanying visual context—and Contrastiveness —how well the explanation rules out alternative answers. We introduce training-free methods to estimate both these qualities without relying on ground-truth quality annotations. In experiments with three popular VLMs and two visual reasoning tasks, A-OKVQA Schwenk et al. 2022 and VizWiz Gurari et al. 2018 , we find that our Visual Fidelity and Contrastiveness qualities better distinguish correct VLM predictions than do existing evaluation qualities Hase et al. 2020 ; Chen et al. 2023 ; Liu et al. 2023b , and are also better calibrated with VLM correctness (§ 3 ). We conduct several user studies in which participants evaluated VLM predictions without access to visual inputs (§ 4 ). We find that augmenting VLM explanations with our proposed qualities improves participants’ ability to distinguish between correct and incorrect VLM predictions. In particular, showing the product of Visual Fidelity and Contrastiveness as a single quality score leads to an 11.1% absolute improvement in user accuracy, along with a 15.4% absolute reduction in the rate of users falsely believing inaccurate predictions. Finally, we show that presenting explanation qualities via natural language descriptions instead of scores further improves user decision making accuracy (§ 4.2 ). Overall, our findings highlight the utility of evaluating the quality of VLM-generated explanations and communicating these qualities to users relying on VLM assistants. 2 Quality Scores for VLM Explanations Visual reasoning is the task of answering a textual question about an image by drawing inferences from what is visually observed. A vision-language model (VLM) is given an input x=(I,Q)\in\mathcal{X} consisting of an image I and a question Q , and produces an answer a\in\mathcal{A} , a (closed or open) set of options. Additionally, the VLM generates a natural language explanation E for its prediction a . This explanation can be either a chain-of-thought rationale Wei et al. 2023 that the VLM prediction is conditioned on (IQ\rightarrow Ea) or a post hoc justification generated after the VLM prediction (IQ\rightarrow aE) Wiegreffe et al. 2022b . Algorithm 1 Evaluate Visual Fidelity 0: Image I , question Q , model prediction a , explanation E 0: Visual Fidelity score S_{\mathrm{VF}}(E) 1: \mathcal{Q}^{V}=\{q^{V}_{1},\dots,q^{V}_{K}\}\leftarrow m_{\texttt{QGen}}(E,Q,a) 2: for all q^{V}_{j}\in\mathcal{Q}^{V} do 3: a^{V}_{j}\leftarrow m_{\texttt{Verif}}(q^{V}_{j},I) 4: end for 5: S_{\mathrm{VF}}(E)\leftarrow\frac{\sum_{i=1}^{K}\mathbbm{1}\{a^{V}_{i}=\text{``yes''}\}}{K} Algorithm 2 Evaluate Contrastiveness 0: Question Q , explanation E , set of possible answers \mathcal{A} , model prediction a_{0}\in\mathcal{A} 0: Contrastiveness score S_{\mathrm{Contr.}}(E) 1: P\leftarrow\text{Mask}(E,\mathcal{A}) 2: for all a_{j}\in\mathcal{A} do 3: h_{j}\leftarrow m_{\texttt{QA}\rightarrow\texttt{S}}(Q,a_{j}) 4: end for 5: S_{\mathrm{Contr.}}(E)\leftarrow\frac{\textsc{Pr}_{\texttt{NLI}}(P\text{ entails }h_{0})}{\sum_{a_{j}\in\mathcal{A}}\textsc{Pr}_{\texttt{NLI}}(P\text{ entails }h_{j})} We define an explanation quality scoring function S_{\varphi}(E;I,Q,a,\mathcal{A}) 1 1 1 For brevity, we note arguments for S only when relevant. that maps a VLM-generated explanation E to a score in [0,1] reflecting a specific quality \varphi of the trust cue of an explanation. Existing qualities for natural language explanations, primarily developed and evaluated in text-only settings, typically look at how well an explanation supports the model’s prediction Hase et al. 2020 , its informativeness Chen et al. 2023 , and its plausibility according to common sense priors Liu et al. 2023b . However, evaluating only these qualities is insufficient for VLM-generated explanations since they do not consider the relationship