From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning

Ruilin Luo, Chufan Shi, Yizhen Zhang, Cheng Yang, Songtao Jiang, Tongkun Guan Ruizhe Chen Ruihang Chu Peng Wang Mingkun Yang, Yujiu Yang Junyang Lin Zhibo Yang · 2026

Read From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning by Ruilin Luo, Chufan Shi, Yizhen Zhang, Cheng Yang, Songtao Jiang, Tongkun Guan Ruizhe Chen Ruihang Chu Peng Wang Mingkun Yang, Yujiu Yang Junyang Lin Zhibo Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03825.txt.

Introduction

Ruilin Luo and others study the cold-start initialization stage of multimodal large reasoning models, the phase that prepares such models for reinforcement learning. They introduce the Visual Attention Score, a metric that measures how much a model attends to visual tokens, and find that reasoning results is tightly linked to it. Surprisingly, a multimodal cold-start does not raise this score, a failure the authors call Lazy Attention Localization, while a text-only cold-start does.

Training-free interventions that shift attention during inference improve results, confirming the effect is causal. Building on this, the authors propose the AVAR framework, which combines visual-anchored data synthesis, attention-guided objectives, and visual-anchored rewards. It gains notably across seven multimodal reasoning benchmarks on the model tested, and ablations credit each component.

Catalog record

CREATORRuilin Luo, Chufan Shi, Yizhen Zhang, Cheng Yang, Songtao Jiang, Tongkun Guan Ruizhe Chen Ruihang Chu Peng Wang Mingkun Yang, Yujiu Yang Junyang Lin Zhibo Yang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57248
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.03825

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning Ruilin Luo, Chufan Shi, Yizhen Zhang, Cheng Yang, Songtao Jiang, Tongkun Guan Ruizhe Chen Ruihang Chu Peng Wang Mingkun Yang, Yujiu Yang Junyang Lin Zhibo Yang Abstract The cold-start initialization stage plays a pivotal role in training Multimodal Large Reasoning Models (MLRMs), yet its mechanisms remain insufficiently understood. To analyze this stage, we introduce the Visual Attention Score (VAS), an attention-based metric that quantifies how much a model attends to visual tokens. We find that reasoning performance is strongly correlated with VAS ( r = 0.9616 r=0.9616 ): models with higher VAS achieve substantially stronger multimodal reasoning. Surprisingly, multimodal cold-start fails to elevate VAS, resulting in attention distributions close to the base model, whereas text-only cold-start leads to a clear increase. We term this counter-intuitive phenomenon Lazy Attention Localization . To validate its causal role, we design training-free interventions that directly modulate attention allocation during inference, performance gains of 1–2% without any retraining. Building on these insights, we further propose A ttention-Guided V isual A nchoring and R eflection (AVAR), a comprehensive cold-start framework that integrates visual-anchored data synthesis, attention-guided objectives, and visual-anchored reward shaping. Applied to Qwen2.5-VL-7B, AVAR achieves an average gain of 7.0% across 7 7 multimodal reasoning benchmarks. Ablation studies further confirm that each component of AVAR contributes step-wise to the overall gains. The code, data, and models are available at https://github.com/lrlbbzl/Qwen-AVAR . 1 Introduction Recent advances in Reinforcement Learning (RL) have significantly enhanced the reasoning capabilities of Large Language Models (LLMs) such as OpenAI o1 ( 14 ) , Qwen-Max ( 33 ) and DeepSeek-R1 ( 27 ; 12 ) . Building upon this success, recent research has leveraged RL to construct Multimodal Large Reasoning Models (MLRMs), aiming to equip them with stronger cross-modal reasoning capabilities ( 61 ; 47 ; 40 ; 54 ; 18 ; 25 ) . However, applying these techniques directly exposes a critical but underexplored stage of the training pipeline: the cold-start initialization stage that precedes the RL stage. Understanding and optimizing this stage remains a core limitation of current MLRMs. A surprising and counter-intuitive phenomenon illustrates this limitation: A text-only cold-start yields substantial improvements for MLRMs in subsequent RL tuning, whereas multimodal cold-start provides only marginal gains ( 39 ; 40 ; 54 ) . This phenomenon reveals a bottleneck in current training paradigms: MLRMs fail to leverage multimodal signals during cold-start, leading to inefficient resource use and limiting the potential of RL for multimodal reasoning. Despite its importance, this paradoxical outcome still lacks a clear quantitative explanation. To shed light on this paradox, we re-examine multimodal reasoning through the lens of attention allocation (Sec. 3 ). We introduce Visual Attention Score (VAS) to quantify how much a model attends to visual tokens. Correlating VAS with reasoning performance across representative MLRMs, we find that reasoning performance is strongly correlated with VAS ( r=0.9616 , Figure 1 a ): models with higher VAS achieve stronger multimodal reasoning, while those with low VAS perform worse. Furthermore, we find that multimodal cold-start fails to increase VAS, leaving distributions close to the base model. In contrast, text-only cold-start induces an increase in visual attention and stronger visual grounding (Figure 1 b ). We term this phenomenon Lazy Attention Localization . It reveals that the effectiveness of cold-start arises not from multimodal alignment but from reasoning patterns internalized through text-only data, which enable models to preserve visual grounding in inference. Building on this observation, we design a set of training-free pilot experiments that directly manipulate attention allocation at inference time (Sec. 4 ). By amplifying attention to visual tokens and reducing redundant focus on system tokens, we observe consistent gains in multimodal reasoning without any retraining. Across models with different baseline performance levels, including Qwen2.5-VL-7B, Revisual-R1-CS and OVR-CS, our method yields average improvements of 1–2%. These results provide causal evidence that attention distribution is a decisive factor for reasoning capability. We therefore consider whether redundant attention to system tokens can be reduced and reallocated to strengthen visual tokens during training. Motivated by this insight, we propose A ttention-Guided V isual A nchoring and R eflection (AVAR), a framework that explicitly reshapes attention allocation during cold-start training (Sec. 5 ). AVAR first employs a three-stage data synthesis pipeline that embeds visual anchors throughout the reasoning process, orchestrating models to generate synthetic data with built-in visual reflection. It then introduces attention-guided training objectives that enhance visual anchoring by encouraging focus on visual tokens while suppressing reliance on system tokens. Finally, during reinforcement learning, AVAR incorporates visual-anchored reward shaping, ensuring models not only produce correct answers but also maintain strong visual grounding across extended reasoning chains. Extensive experiments across 7 multimodal reasoning benchmarks demonstrate the effectiveness of AVAR (Sec. 6 ). Compared to the baseline Qwen2.5-VL-7B, our final model AVAR-Thinker achieves an average gain of 7.0%, with the strongest improvements on MathVision (+12.2%) for multi-step geometric reasoning and HallusionBench (+8.8%) for robustness against visual hallucinations. Systematic ablation studies further validate the overall pipeline design, clearly showing that each component of AVAR contributes step-wise to the observed performance gains. In summary, our work makes the following contributions: • We introduce the Visual Attention Score (VAS), a metric that quantifies attention to visual tokens and strongly correlates with reasoning performance. Using VAS, we uncover Lazy Attention Localization , showing that multimodal cold-start fails to raise visual attention while text-only initialization increases it. This explains the underlying cause of multimodal cold-start ineffectiveness. • We design training-free interventions that manipulate attention allocation at inference time by reducing redundant attention to system tokens and reallocating it to visual tokens. These interventions achieve consistent gains of 1–2% across different models, establishing causal evidence for the role of visual attention in multimodal reasoning. • We propose AVAR, a cold-start framework that reshapes attention allocation by combining visual-anchored data synthesis, attention-guided objectives, and visual-anchored reward shaping. It shifts redundant attention from system to visual tokens, enabling stronger visual grounding. Applied to Qwen2.5-VL-7B, AVAR achieves a 7.0% average gain across 7 multimodal benchmarks. Figure 1: Analysis of different models’ performance and Visual Attention Score (VAS) distribution. (a) Model performance by mean VAS; (b) VAS distribution across layers. 2 Related Works 2.1 Multimodal Large Reasoning Model Multimodal large reasoning models (MLRMs) aim to tackle reasoning tasks in multimodal scenarios, such as STEM problems ( 21 ; 36 ; 45 ) , perception-related tasks ( 58 ; 16 ; 15 ) . Recent works have focused on improving the curation of cold-start thinking data ( 13 ; 26 ; 7 ; 34 ; 8 ) and exploring RL-based approaches ( 55 ; 47 ; 50 ; 23 ; 59 ; 1 ) . Several studies highlight that high-quality unimodal “thinking data” can substantially improve reasoning capabilities of MLRMs ( 40 ; 42 ; 5 ; 31 ; 35 ) . However, these works stop short of uncovering mechanisms behind this effect, and they have yet to investigate how multimodal reasoning data, particularly in “reasoning-with-image” settings, should be optimized. 2.2 Visual Attention Analysis Recent studies have analyzed how Multimodal Large Language Models (MLLMs) allocate attention across textual and visual information, revealing that inappropriate attention to visual tokens remains a bottleneck.

Read the complete text