HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token

Sai Akhil Kogilathota, Sripadha Vallabha E G, Luzhe Sun, Jiawei Zhou · 2026

Read HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token by Sai Akhil Kogilathota, Sripadha Vallabha E G, Luzhe Sun, Jiawei Zhou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05465.txt.

Introduction

Kogilathota and colleagues test whether hallucination risk in vision-language models can be forecast from internal representations before generation begins, using one forward pass. Across eight modern models and varied tasks, they test visual-only features, vision tokens inside the text decoder, and query tokens that fuse visual and textual information. Probes on these representations detect risk without decoding, reaching 0.93 AUROC on three models: Gemma-3-12B, Phi-4-VL, and Molmo 7B, with late query-token states most predictive for most models.

Catalog record

CREATORSai Akhil Kogilathota, Sripadha Vallabha E G, Luzhe Sun, Jiawei Zhou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65781
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.05465

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token Sai Akhil Kogilathota, Sripadha Vallabha E G, Luzhe Sun, Jiawei Zhou Abstract Hallucinations remain a persistent challenge for vision–language models (VLMs), which often describe nonexistent objects or fabricate facts. Existing detection methods typically operate after text generation, making intervention both costly and untimely. We investigate whether hallucination risk can instead be predicted before any token is generated by probing a model’s internal representations in a single forward pass. Across a diverse set of vision–language tasks and eight modern VLMs, including Llama-3.2-Vision, Gemma-3, Phi-4-VL, and Qwen2.5-VL, we examine three families of internal representations: (i) visual-only features without multimodal fusion, (ii) vision-token representations within the text decoder, and (iii) query-token representations that integrate visual and textual information before generation. Probes trained on these representations achieve strong hallucination-detection performance without decoding, reaching up to 0.93 AUROC on Gemma-3-12B, Phi-4-VL 5.6B, and Molmo 7B. Late query-token states are the most predictive for most models, while visual or mid-layer features dominate in a few architectures (e.g., ∼ \sim 0.79 AUROC for Qwen2.5-VL-7B using visual-only features). These results demonstrate that (1) hallucination risk is detectable pre-generation , (2) the most informative layer and modality vary across architectures, and (3) lightweight probes has the potential to enable early abstention, selective routing, and adaptive decoding to improve both safety and efficiency. 1 Introduction Recent vision–language models (VLMs) such as Gemma-3 ( 33 ) , LLaVA-Next ( 21 ) , Llama-3.2-Vision ( 25 ) , Phi-4-VL ( 16 ) , Molmo ( 27 ) , Qwen2.5-VL ( 2 ) , SmolVLM ( 35 ) , and FastVLM ( 34 ) have achieved remarkable progress in generating coherent, human-like text grounded in visual inputs, redefining the frontier of multimodal AI ( 12 ; 17 ; 18 ) . Yet despite these advances, VLMs remain prone to hallucination, producing descriptions of nonexistent objects, invented attributes, or confident but unsupported claims that are factually inconsistent with the image. Such failures undermine the reliability and trustworthiness of these systems, especially in high-stakes applications such as autonomous navigation, medical imaging, and assistive technologies ( 30 ; 20 ) . Current approaches to mitigating hallucinations in vision–language models are largely reactive . Post hoc evaluation metrics such as CHAIR ( 30 ) , POPE ( 19 ) , and FaithScore ( 14 ) require complete caption generation to identify hallucinated content, making them computationally expensive and unsuitable for real-time assessment. Other methods aim to reduce hallucinations during generation through decoding-time interventions such as HALC ( 4 ) , Uncertainty-Guided Dropout Decoding ( 6 ) , and perception-driven grounding augmentation ( 8 ) . However, these techniques still depend on autoregressive decoding and cannot estimate hallucination risk before generation begins. This reveals a critical gap: few existing methods exploit internal model representations to anticipate hallucinations preemptively and enable real-time risk detection prior to decoding. Figure 1: Pre-generation hallucination prediction scores on examples across 8 different VLM task domains (benchmarking data distribution in Fig. 3 (a)) for Qwen2.5-VL-7B. The probes scores reflect how likely the probe thinks the model will hallucinate based on different pre-generation features (VF, VT, QT as defined in Sec. 3.3 ). To address this gap, we introduce HALP ( HALlucination Prediction via Pre-Generation Probing ), a lightweight framework that predicts hallucination risk from pre-generative internal states of vision–language models. HALP performs a single forward pass over an image–query pair and extracts three types of representations: (1) pooled visual features from the vision encoder, (2) decoder states at the last vision tokens across multiple layers, and (3) decoder states at the final text query token at corresponding depths. Simple MLP probes trained on these representations estimate whether a model is likely to hallucinate based on the visual context before any token is generated. Once trained, these probes require neither hallucination labels nor decoding, enabling efficient, generation-free risk prediction. To assess generality, we construct a 10k-example benchmark covering diverse hallucination stressors, including object presence (AMBER ( 1 ) , POPE ( 19 ) ), diagram and math reasoning (MathVista ( 23 ) ), compositional and open-ended knowledge (MME ( 7 ) ), illusion-style traps (HaloQuest ( 5 ) ), and targeted hallucination cases (HallusionBench ( 10 ) ). Figure 1 shows qualitative hallucination-detection examples across our evaluation domains, where HALP’s pre-generation probes estimate hallucination risk before any output tokens are generated. High scores predicts VLM hallucinations, while low scores align with truthful responses. Experiments on eight state-of-the-art open-source VLMs show that hallucination signals are accessible before generation : probes trained on internal states reach high AUROC without decoding, with query-token representations near the top of the decoder most predictive ( \sim 0.93 for Gemma-3, Phi-4-VL, and Molmo). The most informative locus varies by architecture while deep query-token states dominate for most models, others such as Qwen-2.5-VL and Llama-3.2-Vision achieve strong detection from only visual features ( \sim 0.79 and \sim 0.77, repsectively) or mid-decoder vision tokens, suggesting distinct failure pathways across models. Overall, lightweight probes generalize across architectures and representation types, offering a practical, low-cost solution for early hallucination risk assessment. 1 1 1 Code & data at https://github.com/Zesearch/HALP . 2 Background and Related Work Hallucination in Vision-Language Models Hallucination in vision–language models (VLMs) refers to the generation of content that is inconsistent with the visual input. Prior work has identified multiple forms of hallucination, including object hallucinations, attribute hallucinations, and relational hallucinations ( 30 ; 32 ) . Such errors undermine the reliability of VLMs, particularly in safety-critical settings such as medical imaging, autonomous navigation, and assistive technologies ( 30 ; 20 ; 13 ) . Hallucinations are commonly attributed to misalignment between linguistic priors and visual grounding, as well as biases introduced during training. For instance, models may describe nonexistent objects, assign incorrect attributes to visible entities, or infer spurious relationships between objects. These failure modes highlight a persistent disconnect between linguistic fluency and visual grounding, underscoring the need for effective hallucination detection and mitigation in vision–language systems. Representation Probing Representation probing is a diagnostic approach where lightweight classifiers or regressors are trained on internal model activations to assess the encoding of specific properties. In NLP, probes have been used to capture syntactic dependencies, part-of-speech tags, and coreference relations ( 11 ; 24 ; 9 ) . In vision, linear probes on convolutional features have been employed to reveal emergent object detectors in scene-classification networks ( 39 ) . Recent work has extended probing to Vision Transformers, where intermediate layers encode rich class-specific and scene-level semantics ( 3 ; 31 ) . These studies demonstrate the effectiveness of probing as a lightweight tool for detecting task-relevant features ( 31 ) , motivating its use to identify hallucination signals early in VLMs. Pre-Generation Hallucination Detection Recent advances have demonstrated that hallucination-related signals are present in the internal states of language models before generation occurs. 28 showed that LLMs’ internal representations encode substantial information about truthfulness, with critical signals concentrated around specific tokens. They trained classifiers on hidden states to predict factual correctness, achieving high accuracy across multiple layers. Similarly, 37 designed probes trained on Transformer internal representations to predict hallucinatory behavior across grounded generation tasks, finding that probes can reliably detect hallucinations at

Read the complete text