Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddings
Aakriti Agrawal 1 , Gouthaman KV 2 , Rohith Aralikatti 3 , Gauri Jagatap 2 , Jiaxin Yuan 1 , Vijay Kamarshi 2 , Andrea Fanelli 2 , Furong Huang 1,4 Correspondence: [email protected] · 2025
Read Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddings by Aakriti Agrawal 1 , Gouthaman KV 2 , Rohith Aralikatti 3 , Gauri Jagatap 2 , Jiaxin Yuan 1 , Vijay Kamarshi 2 , Andrea Fanelli 2 , Furong Huang 1,4 Correspondence: [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-05017.txt.
Introduction
Aakriti Agrawal and co-authors aim to cut false claims in large vision-language models by fixing their text vectors. Such models may name things that are not in a photo, as when an empty counter gets a bowl of fruit that is not there.
Their study shows a root cause: models lean too hard on text and too little on the image, since sight data is just tacked onto a language core. Attention drifts toward words and away from visual proof. By refining the text side, the authors seek output that stays tied to what the image shows.
Catalog record
| CREATOR | Aakriti Agrawal 1 , Gouthaman KV 2 , Rohith Aralikatti 3 , Gauri Jagatap 2 , Jiaxin Yuan 1 , Vijay Kamarshi 2 , Andrea Fanelli 2 , Furong Huang 1,4 Correspondence: [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-05017
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddings Aakriti Agrawal 1 , Gouthaman KV 2 , Rohith Aralikatti 3 , Gauri Jagatap 2 , Jiaxin Yuan 1 , Vijay Kamarshi 2 , Andrea Fanelli 2 , Furong Huang 1,4 Correspondence: [email protected] Abstract Hallucinations in Large Vision-Language Models (LVLMs) remain a persistent challenge, often stemming from inadequate integration of visual information during multimodal reasoning. A key cause is the model’s over-reliance on textual priors and underutilization of visual cues, leading to outputs that are linguistically fluent but visually inaccurate. For example, given an image of an empty kitchen countertop, an LVLM might hallucinate a “bowl of fruit” or “cup of coffee,” relying on language associations rather than visual evidence. Most LVLMs incorporate visual features by appending them to the input stream of a pre-trained LLM and training on large-scale vision-language datasets. Our systematic analysis reveals that this strategy often leads to over-dependence on textual information due to the inherent bias of LLMs towards language-dominant representations. This imbalance skews attention towards the text over visual content, weakening the model’s ability to ground outputs in visual inputs. To address this, we propose a simple yet effective visual feature incorporation method that encourages the model to learn visually-informed textual embeddings distinct from those of the base LLM and promotes a more balanced attention distribution. Experimental results across multiple hallucination benchmarks demonstrate that our method significantly reduces hallucinations and fosters more balanced multimodal reasoning. Notably, our approach achieves substantial gains, including +9.33% on MMVP-MLLM, +2.99% on POPE-AOKVQA, up to +3.4% on Merlin, and +3% on the hard-data split of HallusionBench. 1 Introduction The advent of LLMs has transformed NLP, enabling tasks like machine translation, dialogue, and content generation with unprecedented accuracy and fluency. Building on this, Large Vision-Language Models (LVLMs) ( 16 , 34 , 23 ) integrate visual and linguistic understanding in a unified framework, bridging text and visual modalities. This synergy has advanced tasks such as captioning ( 6 ) , question-answering ( 14 ) , multimodal retrieval ( 17 ) , etc. As LVLMs advance, their adoption in domains such as healthcare, autonomous driving, and education is accelerating, expanding the role of AI in real-world multimodal applications. Figure 1: Hallucinations in Video-LLaVA ( 16 ) . Despite this progress, LVLMs remain prone to hallucinations—outputs that are fluent but not grounded in the visual input. These errors, which include fabricating or misinterpreting visual content, undermine reliability and hinder deployment in safety-critical settings. Fig. 1 illustrates failure cases from Video-LLaVA ( 16 ) , a widely adopted LVLM. In one case, the model captions a scene as “moving it towards a person,” despite the absence of both the person and action in the video—highlighting both object and action hallucination. Beyond these, LVLMs hallucinations span a wide range. Attribute hallucination involve assigning incorrect visual properties, such as describing “a red car” when the car is actually blue, or denying visible objects (e.g., “there are no chickens” despite their presence). Relation hallucination fabricate spatial or contextual links, such as claiming “the person is jumping over the fence” when the person is merely standing beside it. In video-based settings, hallucinations may also include fabricated temporal dynamics, such as asserting “the person enters the room” when the individual never appears in the frame. We hypothesize that a fundamental source of hallucinations in LVLMs arises from the prevailing architectural paradigm in which the visual information is appended as embeddings (typically extracted using frozen pre-trained visual encoders) to the input sequence of a pre-trained LLM (Fig. 2 , top). This fused input is then pass to the model and fine-tune on large-scale vision-language datasets, such as image/video captioning, and VQA ( 16 , 10 , 23 ) , etc. While this approach offers modularity, data efficiency, and leverages the strong language generation capabilities of LLMs, it introduces a structural asymmetry: the LLM backbone, trained solely on text, remains inherently biased toward language-driven reasoning ( 2 , 3 ) . As a result, during fine-tuning, the model may tend to fall back to text priors, under-utilizing the visual embeddings and treating them as secondary in the reasoning process. This modality imbalance may lead to a systematic misalignment between visual evidence and generated text, manifesting during inference as hallucinations: outputs that are linguistically coherent and semantically plausible, yet factually incorrect or unsupported by the visual input. Motivated by this, we systematically investigate modality imbalance in LVLMs as a potential source of hallucinations, with a focus on the dominant practice of appending visual embeddings to the input textual tokens of pre-trained LLMs ( 16 , 10 , 23 ) . We use Video-LLaVA ( 16 ) as baseline due to its strong performance, modular design, and community adoption, making it well-suited for controlled analysis and evaluation. Our analysis reveals that the prevailing approach of simply appending visual embeddings to the textual input sequence causes the model to over-rely on language while under-utilizing visual information, thereby exacerbating hallucinations. This arises because the backbone LLM, optimized for text, disproportionately emphasizes textual tokens during self-attention operations within the transformer layers. To address this modality imbalance, we propose a method that integrates visual information directly into text embeddings at the token level, enabling more balanced attention and cross-modal representations that better ground generation in visual input. By embedding visual semantics within the language representation, our method fosters balanced cross-modal reasoning, and reduces hallucinations. Extensive evaluation across multiple hallucination benchmarks demonstrates consistent and statistically significant gains, highlighting the effectiveness and generalizability of our method. Figure 2: Top: Architecture of typical LVLMs like Video-LLaVA, which fuse language and vision embeddings by simple concatenation. Bottom: Our modified architecture with a concatenation block that appends the averaged vision embedding to each token embedding, followed by a projection layer . This encourages the model to learn visually informed textual embeddings and better attend to visual input during training. 2 Related Works Large Vision-Language Models (LVLMs) extend pre-trained LLMs to handle visual inputs, typically by appending visual embeddings—extracted from frozen image or video encoders—to the language token sequence. This token-level fusion strategy enables architectural modularity and reusability of LLMs without major modifications. Notable models following this approach include LLaVA ( 19 ) , MiniGPT-4 ( 38 ) , Video-LLaVA ( 16 ) , Video-ChatGPT ( 22 ) , Bunny ( 10 ) , and Video-LLaMA ( 34 ) . Among these, Video-LLaVA has emerged as a foundational model due to its strong benchmark performance, open-source nature, and straightforward temporal extension via frame-wise token concatenation ( 26 ) . Tang et al. ( 26 ) further identify Video-LLaVA as a key reference model that underpins many derivatives, including Video-ChatGPT (chat applications), Bunny (efficiency), and Video-LLaMA (fine-grained fusion). Models like Flamingo ( 1 ) and BLIP-2 ( 14 ) use complex cross-attention to integrate modalities dynamically across transformer layers. Although more flexible, they incur higher computational costs and less modularity. Empirical results ( 19 ) show simpler token-appending strategies often match or outperform these methods in accuracy and efficiency. For its simplicity, extensibility, and strong performance, we adopt Video-LLaVA ( 16 ) as our base model to investigate visual feature integration limitations, focusing on attention distribution, modality alignment, and hallucination. Hallucination Detection