VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context

Xiaoqian Shen Mohamed Elhoseiny · 2026

Read VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context by Xiaoqian Shen Mohamed Elhoseiny (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30288.txt.

Introduction

Xiaoqian Shen and Mohamed Elhoseiny improve fine-grained perception in vision-language models facing very long visual inputs. High-resolution images and long videos flood such models with tokens. An attention sink effect lets irrelevant tokens absorb too much attention. Existing fixes crop regions and re-encode them in extra passes. Those passes cost time and lean on unreliable coordinate predictions.

Their framework, VisReflect, reflects on visual content inside the latent space instead. The model generates continuous visual reflections of question-relevant features. Reflections emphasize salient regions or frames in just one forward pass. No discrete intermediate predictions are decoded at any point. Evaluations span high-resolution image benchmarks up to 8K resolution. Video understanding benchmarks are included in the test suite too. Gains reach several points on image tasks and smaller ones on video. Inference runs much faster than zooming-based rivals on video.

Catalog record

CREATORXiaoqian Shen Mohamed Elhoseiny
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30288

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context Xiaoqian Shen Mohamed Elhoseiny Abstract Large Vision Language Models (LVLMs) have achieved remarkable success on vision–language tasks, yet fine-grained perception over high-resolution images and long-context videos remains challenging. As the number of visual tokens increases, the visual attention sink phenomenon becomes increasingly severe, causing irrelevant tokens to absorb a disproportionate amount of attention mass. Recent approaches attempt to mitigate this issue by explicitly predicting bounding boxes or temporal spans and re-encoding the cropped visual regions. Such methods depend on unreliable numeric localization in the discrete token space and incur significant computational overhead due to additional forward passes. In this work, we propose VisReflect , a simple yet effective framework that improves fine-grained perception in long visual contexts through latent visual reflection. Instead of decoding intermediate predictions into discrete tokens, the model generates continuous visual reflection that represents question-relevant visual features in the latent space. These reflections selectively emphasize salient regions or frames, guiding attention towards relevant visual tokens within a single forward pass. We conduct comprehensive evaluations on challenging high-resolution image benchmarks, including BLINK, V ∗ , and HRBench-4K/8K, as well as video understanding benchmarks such as MVBench, VideoMME, and MLVU. Our method consistently improves over strong baselines, achieving gains of 4.1 % 4.1\% on image benchmarks and 1.8 % 1.8\% on video benchmarks. Compared with zooming-based methods, our model achieves comparable performance while reducing inference time by roughly 44% on video understanding. 1 Introduction Large Vision Language Models (LVLMs) have recently achieved remarkable progress on advanced vision–language tasks, demonstrating strong performance in visual question answering and video reasoning. By integrating powerful language models with visual encoders, the models are able to reason over multimodal inputs and exhibit impressive capabilities in complex perception and reasoning scenarios. However, despite these successes, a persistent challenge remains: fine-grained perception in long visual contexts. Figure 1 : Existing zooming-based methods localize relevant regions or frames by predicting coordinates in discrete token space and performing repeated forward passes. In contrast, VisReflect generates latent visual reflection tokens in continuous visual space, enabling the model to internally recall question-relevant visual features within a single forward pass. When processing high-resolution images, LVLMs must handle thousands of visual tokens to achieve fine-grained perceptual understanding. This challenge is further amplified in long-context video reasoning, where visual token sequences grow substantially. Ideally, the model should selectively attend to visual information that is most relevant to the textual query. However, as the number of visual tokens increases, the Visual Attention Sink [ 14 ] phenomenon becomes more severe, causing irrelevant tokens to absorb a disproportionate amount of attention mass. Consequently, the model struggles with small-object perception in high-resolution images or temporally localized reasoning in long-form videos. To address this challenge, recent studies have introduced the “ Thinking-with-Images/Videos ” paradigm [ 44 , 45 , 26 , 11 , 30 ] , which aims to preserve global context while capturing fine-grained details in long visual sequences. Specifically, these methods prompt the model to predict the bounding box of a region of interest or the temporal span of salient frames. Based on these predictions, a second fine-grained forward pass is performed, where visual tools are invoked to crop or zoom into the selected image regions or video segments, and the refined inputs are re-encoded for detailed understanding. Although this paradigm yields performance improvements, it exposes two critical limitations. (i) Reliability of numeric predictions. The model generates deterministic tokens to represent bounding box coordinates or temporal intervals, which are expressed purely in the textual space and do not faithfully model the underlying visual feature space. As a result, the second forward pass is highly sensitive to localization errors. (ii) Increased computational cost due to additional prefilling. Iterative cropping or zooming requires the visual features to be re-forwarded to the language model, substantially increasing inference latency and computational overhead. These limitations lead us to rethink, how should LVLMs retrieve relevant visual evidence from massive token sequences. Instead of explicitly zooming into pixel-space regions through discrete token predictions, can a model directly recall relevant visual representations within its latent space? Motivated by this perspective, we propose VisReflect , a simple yet effective framework that enhances fine-grained perception in high-resolution images and long-context videos without explicit coordinate-based cropping or temporal zooming. Instead of predicting discrete localization tokens, it generates continuous visual reflection embeddings that recall the visual features of salient regions or frames in the latent space. These reflection embeddings guide attention toward informative visual tokens, enabling the model to focus on relevant regions or frames within a single forward pass. As a result, our approach provides a computationally efficient alternative to multi-pass zooming strategies, offering a unified solution for long-context visual understanding. We evaluate our method on challenging high-resolution image benchmarks, including BLINK [ 12 ] , V ∗ , and HRBench [ 36 ] , as well as on long-video benchmarks such as VideoMME [ 10 ] and MLVU [ 46 ] , which demand precise small-object perception and temporally localized segment reasoning. Experimental results demonstrate an average improvement of 4.1\% over the base model on image benchmarks and a 1.8\% gain over strong baselines on video benchmarks. Moreover, our VisReflect achieves comparable performance than multi-pass “thinking-with-video” approaches while reducing inference time by approximately 44%. Contribution. Our contribution is summarized as follows: • We propose VisReflect , a novel framework that reflects question-relevant visual information from long visual contexts directly in the continuous embedding space, avoiding unreliable numeric localization and costly multi-pass inference required by prior tool-using methods. • We analyze the attention maps from the visual reflection tokens to the original visual inputs. The reflected visual representations guide the model to focus its attention on informative regions or frames within large visual contexts, mitigating the attention sink caused by massive visual tokens. • Comprehensive experiments demonstrate that VisReflect improves fine-grained perception in both high-resolution images and long-form videos, achieving gains of +4.1% and +1.8% . It achieves comparable performance to zooming-based methods that require multiple forward passes, while reducing video inference time by 44% . 2 Related Work 2.1 Thinking with Images Large vision language models (LVLMs) [ 23 , 47 ] have recently achieved strong performance across a wide range of visual understanding and reasoning tasks. Building upon these foundations, several studies [ 4 , 27 , 5 , 28 ] enhance spatial perception by enabling models to predict explicit coordinate outputs, allowing them to localize small or fine-grained objects within images. More recently, research on fine-grained multimodal reasoning has moved toward a “Thinking-with-Images” paradigm, in which models dynamically retrieve localized visual evidence during inference rather than depending exclusively on a single global image representation [ 42 , 38 , 45 , 15 , 44 , 35 ] . In this context, DeepEyes [ 45 ] and Mini-o3 [ 15 ] employ reinforcement learning to incentivize the usage of visual tools such as zooming-in, cropping or searching, while Thyme [ 44 ] and PixelReasoner [ 35 ] enable models to generate code or perform pixel-space operations to manipulate visual inputs dynamically. Although effective, this paradigm incurs substantial inference overhead due to repeated visual encoding passes or iterative tool-calling loops, leading to increased latency.

Read the complete text