Latent Noise Mask for Reducing Visual Redundancy in Multimodal Large Language Models
Kai Jiang, Ruishu Zhu, Siqi Huang, Hongyuan Zhang & Xuelong Li · 2026
Read Latent Noise Mask for Reducing Visual Redundancy in Multimodal Large Language Models by Kai Jiang, Ruishu Zhu, Siqi Huang, Hongyuan Zhang & Xuelong Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30168.txt.
Introduction
Kai Jiang, Ruishu Zhu, Siqi Huang, Hongyuan Zhang, and Xuelong Li clean up the visual evidence that multimodal models reason over. Such models turn images into long token sequences. Question-relevant cues often drown in dense, redundant visual tokens. Their framework, Lens, purifies that evidence inside the latent space. It works without changing the model's backbone or token order.
A single learnable Lens Evidence Token scores each visual token. Scores say how strongly a token supports the current question. Guided by those scores, Lens injects adaptive noise into weak tokens. Low-relevance tokens are softly suppressed rather than deleted outright. Only one temporary control token and a small noise generator are added. Gains reach several points on most visual question answering datasets. Grounding tasks, which locate the referred object, improve similarly. Cleaner evidence helps more than longer reasoning traces.
Catalog record
| CREATOR | Kai Jiang, Ruishu Zhu, Siqi Huang, Hongyuan Zhang & Xuelong Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.30168
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Latent Noise Mask for Reducing Visual Redundancy in Multimodal Large Language Models Kai Jiang, Ruishu Zhu, Siqi Huang, Hongyuan Zhang & Xuelong Li Abstract Multimodal large language models (MLLMs) often fail in fine-grained visual reasoning, as question-relevant visual cues are diluted by dense and redundant image tokens. Recent multimodal reasoning methods usually extend chain-of-thought from language models into visual or latent spaces, seeking to add intermediate reasoning states while overlooking the negative impact of redundant visual tokens. We propose L at E nt N oise ma S k ( Lens ), a question-conditioned visual evidence purification framework that empowers MLLMs to reason with cleaner visual cues in latent space. Lens introduces a lightweight Lens Evidence Token (LET) to score which visual tokens support the current question and preserve them during decoding. Guided by the LET scores, it injects adaptive latent noise into low-relevance tokens, softly suppressing distractors without changing the model backbone or token sequence. With only one temporary learnable control token and a lightweight noise generator, Lens adds minimal overhead while improving the base MLLM by 2.4–6.4 points on most VQA datasets and by 4.1–6.4 points on grounding tasks. These results show that multimodal reasoning can benefit more directly from cleaner question-relevant visual evidence than from simply extending the reasoning trace. 1 Introduction Multimodal large language models (MLLMs) have become a central interface for visual understanding and multimodal reasoning ( 1 ; 19 ; 36 ) . By mapping images into token sequences that can be processed by large language models, they inherit strong language reasoning ability and support tasks such as visual question answering, spatial reasoning, chart understanding, and visual grounding. However, reliable visual reasoning often depends on a few local cues rather than the whole image . Accordingly, many failures arise when small objects, fine-grained attributes, or spatial relations are diluted by background regions, nearby distractors, or language priors ( 37 ; 43 ) . The key bottleneck is not only weak reasoning but also the difficulty of isolating question-relevant evidence from dense visual tokens. Figure 1: Motivation of Lens . Existing multimodal reasoning methods often add visual or latent reasoning states, yet redundant image tokens can still distract the model. Lens learns Lens Evidence Token (LET) to score visual evidence and softly suppress irrelevant visual tokens in latent space. Recent works address this gap by making MLLMs reason longer or reason in richer spaces. Textual chain-of-thought prompting encourages step-by-step inference and has been extended to multimodal tasks ( 38 ; 44 ; 8 ) . Other methods introduce visual cues through image crops, bounding boxes, visual tools, or image generation ( 30 ; 46 ; 33 ) . Latent-space methods further move intermediate reasoning into continuous representations, e.g., Mirage interleaves latent visual tokens with text, DMLR refines latent think tokens and injects relevant patches, and VisMem stores latent vision memories during generation ( 39 ; 23 ; 41 ) . Most recent multimodal reasoning methods improve reasoning by adding more textual, visual, or latent context. These studies show that visual and latent spaces can support reasoning beyond pure text, but they fail to directly reduce the influence of irrelevant image tokens. However, this additive view leaves a simpler question underexplored. Images are already encoded as dense visual token sequences, and many tokens are unrelated to a specific question. For instance, when answering the color of a small object, background tokens and unrelated objects may dominate the context. When grounding a target object, visually similar neighbors may provide strong but misleading cues. When the needed evidence is already present, the main challenge becomes visual selectivity rather than visual availability. In such cases, adding more reasoning context not only leaves distractors untouched, but can also make decoding harder by increasing the amount of irrelevant context that the model must sift through. In this paper, we study multimodal reasoning from the perspective of visual evidence purification. Lens learns which visual tokens support the current question and suppresses the remaining tokens before they interfere with decoding. This differs from token pruning ( 40 ; 2 ; 48 ) or hard region selection ( 30 ; 6 ; 49 ) . Removing tokens can be brittle when the relevance prediction is imperfect, and it may change the token layout expected by the backbone. Instead, we use a soft latent-space intervention that keeps the original sequence structure while reducing the influence of distractor tokens. As shown in Fig. 1 , the goal is not to generate longer textual thoughts or add more visual thoughts, but to clean the visual evidence that the model already receives. To this end, we propose Latent Noise Mask, denoted as Lens , a question-conditioned visual evidence suppression framework for MLLMs. Lens provides a lightweight question-conditioned intervention that purifies visual evidence without external tools, costly rationales, or backbone modification. It introduces a temporary Lens Evidence Token (LET) that estimates the relevance of each visual token to the current question. Instead of using expensive chain-of-thought traces, Lens uses object-level annotations as evidence supervision by mapping question-relevant boxes to visual patches. Guided by the LET scores, it preserves evidence tokens and injects adaptive latent noise into low-relevance tokens. The backbone and token sequence structure remain unchanged, making the framework easy to attach to existing MLLMs. Figure 2: Overview of Lens . Evidence probing appends a temporary \langle\mathrm{mask}\rangle token, whose hidden state is decoded by a simple MLP head f_{\theta} into the LET scores. A threshold \tau converts the scores into a suppression gate, and a small noise generator predicts token-specific latent noise for low-gate visual tokens. The control token is removed before final decoding, so the MLLM receives the purified sequence \widetilde{\mathbf{V}} with the original token layout. Our experiments show that this shift from visual addition to visual suppression is effective. Across VQA and grounding benchmarks, Lens improves the base MLLM by 2.4–6.4 points on most VQA datasets and by 4.1–6.4 points on grounding tasks. Qualitative results further show that the predicted LET scores align with question-relevant regions and reduce attention to distractors. The results show that cleaner question-relevant visual evidence can improve multimodal reasoning more directly than simply extending the reasoning trace. Our contributions are summarized as follows: • We reveal that question irrelevant visual distractors in dense visual tokens can compete with key evidence and hinder fine grained visual reasoning, which motivates us to explore for suppressing visual interference than simply adding more tokens with visual information. • We propose a lightweight visual evidence purification framework that learns question conditioned token relevance and softly suppresses low relevance visual tokens in latent space. • Experiments show the advantage in both accuracy and efficiency over other methods, demonstrating the effectiveness and generality of reasoning with cleaner visual evidence. 2 Related Work Visual and multimodal reasoning Chain-of-thought prompting improves language reasoning by exposing intermediate rationales ( 38 ; 16 ) , and has been adapted to multimodal tasks that require visual evidence ( 44 ; 27 ) . Recent MLLM methods add visual evidence through grounded rationales, selected regions, structured prompts, visual sketching, image editing, or tool use ( 30 ; 7 ; 26 ; 10 ; 46 ; 33 ; 6 ; 20 ; 45 ; 35 ; 4 )