DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models

Yangfu Li ♣ {}^{\,\clubsuit} Hongjian Zhan ♣ , {}^{\,\clubsuit,} Jiawei Chen ♣ {}^{\,\clubsuit} Yuning Gong ♡ , ♢ {}^{\,\heartsuit,\,\diamondsuit} Qi Liu ♣ {}^{\,\clubsuit} Yue Lu ♣ {}^{\,\clubsuit} ♣ East China Normal University, ♡ Sichuan University, ♢ Shanghai AI Laboratory {yfli_cee, qiliu}@stu.ecnu.edu.cn, [email protected], [email protected] · 2026

Read DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models by Yangfu Li ♣ {}^{\,\clubsuit} Hongjian Zhan ♣ , {}^{\,\clubsuit,} Jiawei Chen ♣ {}^{\,\clubsuit} Yuning Gong ♡ , ♢ {}^{\,\heartsuit,\,\diamondsuit} Qi Liu ♣ {}^{\,\clubsuit} Yue Lu ♣ {}^{\,\clubsuit} ♣ East China Normal University, ♡ Sichuan University, ♢ Shanghai AI Laboratory {yfli_cee, qiliu}@stu.ecnu.edu.cn, [email protected], [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03857.txt.

Introduction

Yangfu Li and colleagues propose DeepScan, a training-free framework that improves visually grounded reasoning in large vision-language models. People locate visual evidence by spotting key cues and then relating them to the wider scene, and DeepScan copies that bottom-up style. Its first stage, Hierarchical Scanning, explores local cues and extracts evidence at several scales instead of attempting one-shot localization of the full evidence, which distracting context often defeats.

A Refocusing stage then improves the localized evidence view by combining the vision-language model with visual experts. Evidence-Enhanced Reasoning then merges views at different levels through a hybrid evidence memory to produce accurate, interpretable answers. DeepScan raises performance across diverse visual tasks and model sizes without extra training, scoring strongly on the fine-grained V-star benchmark with the model tested.

Catalog record

CREATORYangfu Li ♣ {}^{\,\clubsuit} Hongjian Zhan ♣ , {}^{\,\clubsuit,} Jiawei Chen ♣ {}^{\,\clubsuit} Yuning Gong ♡ , ♢ {}^{\,\heartsuit,\,\diamondsuit} Qi Liu ♣ {}^{\,\clubsuit} Yue Lu ♣ {}^{\,\clubsuit} ♣ East China Normal University, ♡ Sichuan University, ♢ Shanghai AI Laboratory {yfli_cee, qiliu}@stu.ecnu.edu.cn, [email protected], [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56934
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.03857

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models Yangfu Li ♣ {}^{\,\clubsuit} Hongjian Zhan ♣ , {}^{\,\clubsuit,} Jiawei Chen ♣ {}^{\,\clubsuit} Yuning Gong ♡ , ♢ {}^{\,\heartsuit,\,\diamondsuit} Qi Liu ♣ {}^{\,\clubsuit} Yue Lu ♣ {}^{\,\clubsuit} ♣ East China Normal University, ♡ Sichuan University, ♢ Shanghai AI Laboratory {yfli_cee, qiliu}@stu.ecnu.edu.cn, [email protected], [email protected] Abstract Humans can robustly localize visual evidence and provide grounded answers even in noisy environments by identifying critical cues and then relating them to the full context in a bottom-up manner. Inspired by this, we propose DeepScan, a training-free framework that combines Hierarchical Scanning, Refocusing, and Evidence-Enhanced Reasoning for visually grounded reasoning in Large Vision–Language Models (LVLMs). Unlike existing methods that pursue one-shot localization of complete evidence, Hierarchical Scanning performs local cue exploration and multi-scale evidence extraction to recover evidence in a bottom-up manner, effectively mitigating the impacts of distractive context. Refocusing then optimizes the localized evidence view through collaboration of LVLMs and visual experts. Finally, Evidence-Enhanced Reasoning aggregates multi-granular views via a hybrid evidence memory and yields accurate and interpretable answers. Experimental results demonstrate that DeepScan significantly boosts LVLMs in diverse visual tasks, especially in fine-grained visual understanding. It achieves 90.6% overall accuracy on V* when integrated with Qwen2.5-VL-7B. Moreover, DeepScan provides consistent improvements for LVLMs across various architectures and model scales without additional adaptation cost. The code will be open-source at DeepScan . 1 Introduction Humans excel at grounding critical visual information and reasoning based on the located evidence, which is a hallmark of intelligence recognized in the cognitive and vision science. 38 ; 41 ; 40 . While this is natural to humans, it remains challenging for Large Vision-Language Models (LVLMs) to replicate this behavior, leading to suboptimal performance in complex visual tasks 34 ; 42 ; 39 ; 16 . To reproduce this ability in LVLMs, some approaches have employed reinforcement learning 31 ; 52 ; 35 ; 50 with visually supervised rewards ( e.g . , IoU) or context engineering 42 ; 46 , enabling active perception and localization of evidence during reasoning. Another line of research augments LVLM grounding with auxiliary modules 4 ; 26 ; 45 ; 51 . More recently, several studies 13 ; 2 ; 24 integrate external visual experts, such as GroundingDINO 20 ; 27 and LangSAM 21 , to localize visual evidence based on the consensus between these experts and LVLMs, significantly enhancing fine-grained visual understanding in LVLMs. Despite these advances, most existing methods follow a top-down grounding paradigm: they first perform an image-level search for coarse-grained proxies, such as region proposals 31 ; 52 ; 35 ; 50 ; 46 , detection boxes 13 ; 24 , or textual descriptions 39 , and then refine these proxies to obtain fine-grained evidence. However, this paradigm requires the one-shot localization of complete evidence regions from the entire image, which is easily affected by noisy context, i.e . , attention sink 43 , or semantically similar objects, i.e . , attention drift 6 , leading to suboptimal performance. In such cases, the LVLM either refuses to answer or makes uninformed guesses based on incorrect evidence, as illustrated in . By contrast, humans intuitively adopt a bottom-up routine in challenging visual tasks. For example, when playing the “ spot-the-difference ” puzzles, people scan local patches for subtle discrepancies within largely similar content, then verify these cues at the image level to recover the target while suppressing distracting context. Figure 1 : Performance of LVLMs and visually grounded reasoning variants on V*. DeepScan achieves highly competitive results. Inspired by this human behavior, we propose DeepScan , a training-free framework for visually grounded reasoning in LVLMs, comprising Hierarchical Scanning, Refocusing, and Evidence-Enhanced Reasoning. Unlike existing methods, Hierarchical Scanning leverages patch-wise cue exploration and image-level evidence extraction to robustly localize visual evidence in a bottom-up manner. Refocusing further refines the surrounding context of the extracted evidence through collaboration between the visual experts and LVLMs, reducing information loss. Finally, Evidence-Enhanced Reasoning utilizes a Hybrid Evidence Memory to provide LVLMs with multi-granular evidence views, producing accurate and well-grounded answers. shows that DeepScan performs competitively with leading approaches. Moreover, as a training-free framework, DeepScan scales seamlessly to larger models ( e.g . , Qwen2.5-VL-72B), yielding consistently improved performance. Extensive experiments across diverse visual tasks further demonstrate its effectiveness. Our contributions are fourfold: ❶ We introduce DeepScan , a training-free framework that boosts LVLM performance by explicitly localizing, recalibrating, and integrating evidence before answering. ❷ We propose Hierarchical Scanning , a bottom-up visual grounding paradigm that mitigates noisy context via local cue exploration and multi-scale evidence extraction. ❸ We present Refocusing , a collaborative search paradigm that recalibrates the evidence view through interactions between LVLMs and external visual experts. ❹ Comprehensive experiments validate the superiority of DeepScan. It provides average improvements of 16.3% on V* and 5.5% on TreeBench for Qwen2.5-VL-7B. Detailed ablation studies across LVLM architectures and scales further validate the generalizability of our method. 2 Related Work 2.1 Large Vision-Language Models Early breakthroughs in Large Vision-Language Models (LVLMs), such as Flamingo 1 and BLIP-2 14 , integrated visual features into an LLM via cross-attention. In contrast, LLaVA 19 maps features from a frozen vision encoder ( e.g . , CLIP 25 ) into the LLM’s semantic space with a lightweight MLP. This feature-projection paradigm has driven rapid progress, with subsequent work scaling LVLMs and tackling increasingly complex tasks, such as OCR 48 ; 17 ; 47 , general VQA 18 ; 10 ; 53 , and latent visual reasoning 49 ; 11 . A critical frontier is to handle high-resolution input. For instance, LLaVA-NeXT 18 and InternVL1.5 5 support arbitrary resolutions, while Qwen2-VL 37 and Qwen2.5-VL 3 introduce mRoPE for resolution generality. Despite these advances, current LVLMs lack an explicit mechanism to perceive and localize task-relevant visual evidence, which leads to hallucinations, especially in high-resolution scenes. We address this gap with DeepScan, a training-free framework that couples LVLMs with external experts to achieve visually grounded reasoning, yielding more accurate and interpretable answers. Figure 2 : Overall architecture of DeepScan : Hierarchical Scanning progressively recovers visual evidence from in-patch cues formulated as point-based proxies using Local Cue Exploration and Multi-Scale Evidence Extraction, e.g . , c^{1}_{T}\mapsto e_{t-1} in step T ; Refocusing further refines the surrounding context for the fused evidence via interactions between LVLMs and visual experts; Evidence-Enhanced Reasoning leverages a Hybrid Evidence Memory to provide multi-granular information to the LVLM, enabling detailed yet comprehensive answers. 2.2 Visually Grounded Reasoning Recent agents with “think-with-images” capabilities ( e.g . , GPT-o3 23 ) perform visually grounded reasoning that localizes and emphasizes the critical visual content during inference through dynamic image manipulation 44 ; 33 ; 23 . Building on this, subsequent works employ SFT and reinforcement learning (RL) with visual rewards 36 ; 52 ; 35 ; 50 ; 32 ; 9 , or curiosity-driven objectives 31 to reproduce this ability in general LVLMs; SEAL 42 further integrates explicit evidence localization and visual memory. However, these approaches are costly and challenging to generalize to different architectures and model scales. To overcome this, a parallel line explores training-free paradigms that leverage external visual experts 13 ; 2 ; 45 ; 51 , tree search

Read the complete text