HalLoc : Token-level Localization of Hallucinations for Vision Language Models

Eunkyu Park, Minyeong Kim, Gunhee Kim · 2025

Read HalLoc : Token-level Localization of Hallucinations for Vision Language Models by Eunkyu Park, Minyeong Kim, Gunhee Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-10286.txt.

Introduction

Park, Kim, and Kim build HalLoc in twenty twenty-five, a data set for spotting made up details in AI image answers word by word. Vision language models often invent objects or facts that are not in the picture. Past detectors give a flat yes or no and run slowly. HalLoc holds a large set of samples with each token marked and typed.

Samples span question answering, instruction following, and caption writing. A light model trained on it flags suspect words as the text is made, with graded confidence. It plugs into live models without slowing them much. The authors release the data and code for open use.

Catalog record

CREATOREunkyu Park, Minyeong Kim, Gunhee Kim
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-10286

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HalLoc : Token-level Localization of Hallucinations for Vision Language Models Eunkyu Park, Minyeong Kim, Gunhee Kim Abstract Hallucinations pose a significant challenge to the reliability of large vision-language models, making their detection essential for ensuring accuracy in critical applications. Current detection methods often rely on computationally intensive models, leading to high latency and resource demands. Their definitive outcomes also fail to account for real-world scenarios where the line between hallucinated and truthful information is unclear. To address these issues, we propose HalLoc, a dataset designed for efficient, probabilistic hallucination detection. It features 150K token-level annotated samples, including hallucination types, across Visual Question Answering (VQA), instruction-following, and image captioning tasks. This dataset facilitates the development of models that detect hallucinations with graded confidence, enabling more informed user interactions. Additionally, we introduce a baseline model trained on HalLoc, offering low-overhead, concurrent hallucination detection during generation. The model can be seamlessly integrated into existing VLMs, improving reliability while preserving efficiency. The prospect of a robust plug-and-play hallucination detection module opens new avenues for enhancing the trustworthiness of vision-language models in real-world applications. The HalLoc dataset and code are publicly available at: https://github.com/dbsltm/cvpr25_halloc . 1 Introduction Vision-Language Models (VLMs) ( Ye et al. 2023 ; Liu et al. 2023 ; Zhu et al. 2023 ) harness the language generation capabilities of Large Language Models (LLMs) ( Achiam et al. 2023 ; Touvron et al. 2023 ) , along with the visual understanding provided by vision foundation models ( Dosovitskiy et al. 2021 ; Wang et al. 2023c ) , to comprehend and describe visual content in natural language. Although VLMs have shown remarkable performance across various multimodal tasks, they are prone to producing descriptions that are inconsistent with the visual content, a phenomenon commonly referred to as hallucination ( Rohrbach et al. 2019 ; Liu et al. 2023 ; Wang et al. 2023a ; Li et al. 2023 ) . Figure 1 : Motivation behind HalLoc : HalLoc aims to provide a practical, lightweight add-on for VLMs that simultaneously detects probabilistic hallucinations during response generation. By localizing tokens with high hallucination probabilities in real time, our model enhances use interaction by alerting at potentially unreliable content, prompting closer scrutiny or prioritizing external verification. Previous studies ( Li et al. 2023 ; Sun et al. 2023 ; Fu et al. 2024 ; Wang et al. 2024 ; Wu et al. 2024 ; Qiu et al. 2024 ; Nie et al. 2024 ) have explored the susceptibility of VLMs to various types of hallucinations, including those involving objects, visual attributes, spatial relations, and environmental details. The prevalence and diversity of these hallucinations pose significant challenges to real-world deployment, since they can lead to misinformation and undermine user trust ( Biten et al. 2021 ) . Current approaches to hallucination detection often leverages powerful off-the-shelf models like GPT-4 OpenAI et al. 2024 to cross-verify outputs ( Liu et al. 2024 ; Jing et al. 2024 ; Chen et al. 2024a ) . While effective, these methods introduce substantial computational overhead and latency, bringing challenges to their applications requiring real-time feedback or operating under stringent computational constraints. Moreover, existing methods typically yield definitive outputs as either hallucinated or truthful . This dichotomy fails to capture the nuanced spectrum of confidence levels needed for ambiguous cases where the distinction is unclear. Table 1 : Comparison of HalLocalizer with existing hallucination detection methods. Note that Discrete output refers to definitive classes for detection and Probability output stands for probability as confidence levels of hallucinations. Methods Granularity Type Classification Detection Output VLM Tasks Discrete Probability CHAIR ( Rohrbach et al. 2019 ) Token Object Hand-crafted Rule ✓ Img Captioning GAVIE ( Liu et al. 2024 ) Response Object GPT OpenAI et al. 2024 as judge ✓ Instruction Following FAITHScore ( Jing et al. 2024 ) Atomic Fact None off-the-shelf ✓ VQA, Img Captioning UNIHD ( Chen et al. 2024a ) Segment Obj, Attr, Scene-text, Fact off-the-shelf ✓ VQA, Img Captioning MHalDetect ( Gunjal et al. 2024 ) Sentence None Trained Model ✓ Img Captioning HalLocalizer (ours) Token Obj, Attr, Rel, Sce Trained Model ✓ VQA, Instruction Following, Img Captioning Table 2 : Comparison of HalLoc with existing hallucination detection datasets. Benchmarks Size Granularity Type Annotation VLM Tasks HaELM ( Wang et al. 2023b ) 56K Response None Img Captioning MHalDetect ( Gunjal et al. 2024 ) 16K Segment None Img Captioning MHaluBench ( Chen et al. 2024a ) 400 Segment Obj, Attr, Scene-text, Fact VQA, Img Captioning HalLoc (ours) 155K Token Obj, Attr Rel, Sce VQA, Img Captioning, Instruction Following Therefore, there is a strong need for an efficient hallucination detection mechanism that offers probabilistic assessments. A plug-and-play hallucination detection module, as illustrated in Figure 1 , can be seamlessly integrated into existing VLMs. Such a system enables more nuanced interactions with model outputs, benefiting users and downstream applications. For instance, users can be alerted to content requiring additional scrutiny without dismissing it outright, and systems can prioritize external verification for content with a higher likelihood of being hallucinated. To address this need, we introduce HalLoc, a large-scale dataset comprising 155K samples spanning visual question answering (VQA), instruction following, and image captioning tasks. Each sample is meticulously annotated at the token level with specific types of hallucinations, facilitating comprehensive training and evaluation of hallucination detection models. Leveraging HalLoc, we develop a simple yet effective baseline model that performs concurrent hallucination detection with minimal computational overhead. Our model can be integrated into existing Vision-Language Models (VLMs) in a plug-and-play fashion, enhancing their reliability without sacrificing efficiency. Our contributions can be outlined as follows: 1. Introduction of HalLoc dataset: We present HalLoc, a large-scale dataset comprising 155,000 samples annotated at the token level for specific types of hallucinations. Spanning tasks like visual question answering, instruction following, and image captioning, HalLoc facilitates comprehensive training and evaluation of hallucination detection models in VLMs. 2. Development of an efficient hallucination detection Model: Leveraging HalLoc, we develop a simple yet effective baseline model that performs concurrent hallucination detection with minimal computational overhead. Unlike existing methods that provide binary outputs, our model offers probabilistic assessments, capturing nuanced confidence levels for ambiguous cases. 3. Plug-and-Play integration with existing VLMs: Our hallucination detection model is designed to be seamlessly integrated into existing VLMs in a plug-and-play fashion. This enhances the reliability of VLMs without sacrificing efficiency, addressing the need for an efficient hallucination detection mechanism suitable for real-world applications with computational constraints. 2 Related Work Hallucination Detection Methods Early work on hallucination detection focused on identifying non-existent objects in generated outputs. CHAIR ( Rohrbach et al. 2019 ) detects hallucinated objects by comparing mentioned objects with ground-truth data, using a simple rule-based approach limited to object-level hallucinations. Later approaches use powerful off-the-shelf models for detecting hallucinations. GAVIE ( Liu et al. 2024 ) employs GPT-4 ( OpenAI et al. 2024 ) to assign single accuracy scores to responses. FAITHScore ( Jing et al. 2024 ) uses a chain of large models to break down responses into atomic facts and verify them via visual entailment. UNIHD ( Chen et al. 2024a ) identifies and classifies hallucinations at the semantic segment level by also leveraging a chain of off-the-shelf models including GPT-4

Read the complete text