The Role of Background Information in Reducing Object Hallucination in Vision-Language Models: Insights from Cutoff API Prompting
Masayo Tomita, Katsuhiko Hayashi, Tomoyuki Kaneko · 2025
Read The Role of Background Information in Reducing Object Hallucination in Vision-Language Models: Insights from Cutoff API Prompting by Masayo Tomita, Katsuhiko Hayashi, Tomoyuki Kaneko (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-15389.txt.
Introduction
Masayo Tomita, Katsuhiko Hayashi, and Tomoyuki Kaneko study, in a 2025 paper, why vision language models sometimes name objects that are not in an image. Such object hallucination undermines trust in these models for real tasks. Visual prompting, which highlights a relevant image area, was reported to help, but how much of the image should be kept was unclear.
The authors analyzed success and failure cases of an attention driven prompting method, working through a cutoff application programming interface. The pattern they found turns on background. Prompts that preserved the background context around the target did well. Prompts that cropped too tightly lost the cues the model uses to judge what is plausible. Preserving background proved key to cutting hallucination. The finding gives prompt designers a simple rule: guide the model's eye, but do not blind it to the scene around the object.
Catalog record
| CREATOR | Masayo Tomita, Katsuhiko Hayashi, Tomoyuki Kaneko |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-15389
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Role of Background Information in Reducing Object Hallucination in Vision-Language Models: Insights from Cutoff API Prompting Masayo Tomita, Katsuhiko Hayashi, Tomoyuki Kaneko Abstract Vision-Language Models (VLMs) occasionally generate outputs that contradict input images, constraining their reliability in real-world applications. While visual prompting is reported to suppress hallucinations by augmenting prompts with relevant area inside an image, the effectiveness in terms of the area remains uncertain. This study analyzes success and failure cases of Attention-driven visual prompting in object hallucination, revealing that preserving background context is crucial for mitigating object hallucination. 1 Introduction Vision-Language Models (VLMs), represented by CLIP Radford et al. 2021 and LLaVA Liu et al. 2023 , have advanced rapidly and garnered attention. A Vision-Language Model is equipped with an image encoder and a text encoder, allowing it to process both image and text information simultaneously. With these encoders, multimodal tasks such as visual question answering Liu et al. 2024b , image captioning Karpathy and Fei-Fei 2015 , and image retrieval Zhu et al. 2024 have been realized, and applications have expanded into fields like autonomous driving Zhou et al. 2023 and medical diagnosis Hartsock and Rasool 2024 . However, as these applications expand, the issue of hallucination has become increasingly apparent. Hallucination refers to inconsistencies between the input image and the output text Liu et al. 2024a . Among them, Hallucinations of an object’s presence, attributes, and relationships are defined as object, attribution, and relation hallucinations, respectively. POPE Li et al. 2023 has been proposed and widely used as a method to evaluate the object hallucination. POPE works by asking questions about objects that are either present or absent in an image and verifying whether the responses are correct Li et al. 2023 . Prompting in VLMs has been proposed as methods to reduce hallucination Gu et al. 2023 . In particular, visual prompting, which emphasizes target objects by adding marks has been shown to suppress hallucination Wu et al. 2024 . Among these techniques, a method of visual prompting known as Attention Prompting on Image (API) has been introduced; it is automatic way of visual prompting, which demonstrated improved accuracy in VQA and object hallucination Yu et al. 2024 . Figure 1: API Prompting Process: Without Cutoff Yu et al. 2024 vs. With Cutoff (Proposed). However, in the work reported by Yu et al. 2024 , experiments mostly focused on VQA, leaving a deeper analysis of its effect on object hallucination unexamined. This study analyzes success and failure cases of API prompting to explore effective visual prompting for reducing object hallucination. Specifically, this work examines the relationship between attention-target object alignment and API prompting accuracy and evaluates whether removing the background reduces object hallucination as a validation of cases where attention was focused solely on the target object. 2 Related Work Visual prompting techniques is reported to enhance VQA accuracy and reduce hallucinations by emphasizing key image regions. For example, the Engineering Visual Prompting method addes red circles and arrows by hand ( Shtedritski et al. 2023 ; Cai et al. 2024 ) , while a semi-automated approach overlays segmentation data from models such as SEEM ( Zou et al. 2023 ) . To experiment on over 20,000 objects, API prompting, which is fully automated approach, was employed. 3 Method Figure 2: Process of Visual Attention Evaluation. 3.1 API Prompting API Prompting (sketched in Figure 1 ) is a Visual Prompting method that highlights important parts in an image using a Visual Attention Heatmap derived from a Vision-Language Model ( Yu et al. 2024 ) . The Attribution Map, representing the contribution of image tokens to model outputs, is extracted from a VLM (referred to as Heatmap VLM or H-VLM), convolved, resized to match the image size, and then overlaid on the original image. Following the study by Yu et al. 2024 , Vision-Transformer-based CLIP and LLaVA are employed as Heatmap VLMs. The methods for extracting Visual Attention Attribution Maps from each model are described below. CLIP Attribution Map CLIP computes similarity between text and image representations, and the Attribution Map \Psi is obtained by decomposing the similarity function \text{sim}(\hat{I},\hat{T}) . Since later MSA layers greatly impact image representation ( Gandelsman et al. 2024 ) , the similarity function is approximated as: \displaystyle\text{sim}(\hat{I},\hat{T})\approx\text{sim}\left(\sum_{\ell=L^{\prime}}^{L}\mathcal{L}(\text{MSA}^{\ell}([Z^{\ell-1}]))_{\text{cls}},\hat{T}\right). (1) To filter out irrelevant regions, a complementary Attribution Map \Psi^{\text{comp}} is introduced: \displaystyle\Psi_{i,j}^{\text{comp}} \displaystyle\triangleq 1-\text{sim}(\mathcal{L}(Z_{t}^{L}),\hat{T}), (2) \displaystyle\text{where}\quad t=1+j+P\cdot(i-1). Combining both maps, the final CLIP Attribution Map is defined as: \displaystyle\Psi=\Psi^{\text{cls}}+\Psi^{\text{comp}}-\Psi^{\text{comp}}\odot\Psi^{\text{cls}}. (3) LLaVA Attribution Map LLaVA can provide an Attribution Map \Psi using Multi-Head Self-Attention (MSA) weights between output text tokens and image tokens. The Attribution Map is computed by averaging over all output tokens and attention heads: \displaystyle\Psi_{i,j} \displaystyle\triangleq\frac{1}{MH}\sum_{m=1}^{M}\sum_{h=1}^{H}A_{m,t}^{(\bar{L},h)}, (4) \displaystyle\text{where}\quad t=j+P\cdot(i-1). Here, M is the number of output tokens, H is the number of attention heads, P is the number of patches per image side, and A^{(\bar{L},h)} represents cross-attention between output text and image tokens at layer \bar{L} and attention head h . 3.2 Background Role Examination To assess the necessity of background information for object recognition, ground truth segmentation data is used as a Heatmap during API Prompting and the accuracy of output is evaluated (hereafter referred to as API - Seg.). Binary segmentation masks, overlaid in gray are input into VLMs to evaluate their impact on output accuracy. If POPE response accuracy remains unchanged, background information is deemed unnecessary. 3.3 Minimum Cutoff Minimum cutoff redefines the minimum value in segmentation or Visual Attention Heatmap based on a threshold. Since a threshold of 0.5 showed improvement in Table 5 , values below 0.5 in the cutoff are replaced with 0.5, refining segmentation granularity. Dataset Model Prompting Acc. Prec. Rec. TNR F1 MSCOCO LLaVA w/o prpt. 86.23 84.21 89.19 83.27 86.63 API (CLIP) \blacktriangle 86.52 \blacktriangle 84.78 \triangledown 89.02 \blacktriangle 84.02 \blacktriangle 86.85 API (CLIP) w Cutoff \blacktriangle 88.59 \blacktriangle 85.84 \blacktriangle 92.43 \blacktriangle 84.75 \blacktriangle 89.01 API (LLaVA) \triangledown 86.11 \blacktriangle 84.72 \triangledown 88.12 \blacktriangle 84.10 \triangledown 86.39 API (LLaVA) w Cutoff \blacktriangle 87.98 \blacktriangle 85.10 \blacktriangle 92.09 \blacktriangle 83.88 \blacktriangle 88.46 API - Seg. - - \triangledown 71.78 - - API - Seg. w Cutoff - - \blacktriangle 89.24 - - Table 1: POPE results on MSCOCO datasets with API Prompting. H-VLM Output Visual Attention Alignment Prec. Rec. IoU MSE CLIP Correct (87%) \blacktriangle 13.52 \blacktriangle 83.94 \blacktriangle 11.68 \blacktriangle 28.46 Incorrect (13%) 6.09 77.02 4.78 35.22 LLaVA Correct (86%) \blacktriangle 10.62 64.62 \blacktriangle 8.10 13.60 Incorrect (14%) 6.17 \blacktriangle 69.78 4.44 \blacktriangle 11.60 Table 2: Alignment of CLIP/LLaVA Visual Attention. 3.4 Evaluation of Visual Attention Alignment To assess how well visual attention focuses on target objects, Precision, Recall, Intersection over Union (IoU), and Mean Squared Error (MSE) are computed between the object segmentation data and the Visual Attention Heatmap as depicted in Figure 2 . Visual Attention Heatmaps are converted into binary arrays using thresholds set to the average value of each heatmap. 4 Experiment 4.1 Setting A total of 3,860 images from the MSCOCO dataset Lin et al. 2014 were used to generate visual attention heatmaps from CLIP and LLaVA. These heatmaps were overlaid onto the original images as black masks, and the masked images, along with POPE questions, were input into LLaVA to evaluate the