Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training

Qihuang Zhong, Liang Ding, Wenjie Xuan, Juhua Liu, Bo Du, Dacheng Tao · 2026

Read Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training by Qihuang Zhong, Liang Ding, Wenjie Xuan, Juhua Liu, Bo Du, Dacheng Tao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11931.txt.

Introduction

Qihuang Zhong and five colleagues improve multimodal reasoning through vision-aware self-improvement training. Self-generated reasoning traces suffer data imbalance, with easy samples over-trained and hard samples under-trained, plus language prior bias that neglects visual cues. Their VISTA framework reuses partial correct traces through prefix resampling and scores visual focus with a vision-aware attention score. It boosted reasoning across models and tasks, gaining up to 13.66 percent on average for Qwen2.5-VL-3B-Instruct.

Catalog record

CREATORQihuang Zhong, Liang Ding, Wenjie Xuan, Juhua Liu, Bo Du, Dacheng Tao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69942
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.11931

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training Qihuang Zhong, Liang Ding, Wenjie Xuan, Juhua Liu, Bo Du, Dacheng Tao Abstract Post-training with explicit reasoning traces is common to improve the reasoning capabilities of Multimodal Large Language Models (MLLMs). However, acquiring high-quality reasoning traces is often costly and time-consuming. Hence, the self-improvement paradigm has emerged, enabling MLLMs to self-generate reasoning traces for training without external supervision. Despite its effectiveness, we reveal two shortcomings in the self-improvement training of MLLMs: 1) data imbalance, where simple samples are over-trained, but the challenging yet crucial samples are under-trained; 2) language prior bias, where MLLMs overly rely on linguistic priors while neglecting the visual cues. To this end, we propose VISTA , a VI sion-aware S elf-improvement T raining framework for enhancing the multimodal re A soning of MLLMs. Specifically, VISTA first introduces a prefix resampling strategy to reuse the partial correct reasoning traces for efficient data collection, and then designs a vision-aware attention score to quantify the model’s focus on visual information. Extensive experiments show that VISTA can be applied to various post-training scenarios, i.e. , supervised fine-tuning and preference learning, and effectively enhances the multimodal reasoning performance across various MLLMs and tasks, e.g. , bringing up to +13.66% average performance gains for Qwen2.5-VL-3B-Instruct. 1 Introduction Inspired by the success of OpenAI o1/o3 ( Jaech et al. 2024 ) and DeepSeek-R1 ( Guo et al. 2025 ) , multimodal reasoning, which extends long chain-of-thought (CoT) capabilities to multimodal large language models (MLLMs), has recently attracted significant research attention ( Wang et al. 2024b ; Wang et al. 2025 ) . While post-training with explicit reasoning trajectories can effectively enhance the multimodal reasoning of MLLMs, their performance highly relies on high-quality intermediate reasoning traces ( Yang et al. 2025b ) , which are expensive and time-consuming to obtain ( Peng et al. 2025 ) . Hence, recent literature ( Deng et al. 2025 ) introduces a self-improvement paradigm, where MLLMs iteratively improve themselves by training with their self-generated reasoning data, thus reducing the reliance on external supervision. Figure 1: Comparison of two self-generated reasoning traces . As seen, although predicting the correct answer, MLLMs may still exhibit visual hallucinations during intermediate reasoning processes, due to over-reliance on language priors and neglect of visual cues. Encouragingly, our proposed vision-aware attention score (VAS) can accurately identify these hallucinated solutions. In general, self-improvement training usually contains three steps: (1) data collection, where models are prompted to generate multiple reasoning solutions for each query; (2) data organization, where the ground-truth answers are used to verify the correctness of candidate solutions and the incorrect solutions will be filtered out; (3) model optimization, where models are self-trained with the selected correct solutions. Although self-improvement training has demonstrated remarkable performance ( Zelikman et al. 2022 ; Singh et al. 2023 ) , through a series of analyses (§ 2.1 ), we reveal that the self-improvement of MLLMs still suffers from two key shortcomings: ❶ data imbalance , i.e. , during data collection, MLLMs can readily generate numerous correct solutions for simple queries but struggle to produce sufficient correct ones for difficult queries; ❷ language prior bias , i.e. , MLLMs tend to over-rely on language priors while neglecting visual cues, leading to visual hallucinations that cannot be filtered by answer correctness alone (as illustrated in Figure 1 ). Notably, these two problems share a common root: current self-improvement methods rely solely on answer correctness as the quality signal. This single signal proves inadequate in two complementary ways: (1) in quantity , it fails to yield sufficient correct solutions for challenging queries; (2) in quality , it cannot distinguish visually-grounded solutions from hallucinated ones that happen to arrive at correct answers. These observations naturally give rise to two questions: 1) How can we obtain sufficient correct solutions for challenging yet crucial queries? 2) How can we effectively identify and filter out undesired hallucinated solutions? Some prior studies also recognize these issues and attempt to address them ( Tong et al. 2024 ; Ding et al. 2025 ; He et al. 2025 ) . For ❶, Tong et al. 2024 propose to allocate more trials to difficult queries, and Ding et al. 2025 attempt to use the ground-truth answers to guide the reasoning. Although effective, these methods neglect the prior failed solutions that also contain some useful information, which are wasted and inefficient. For ❷, He et al. 2025 design a metric to quantify the language prior bias by removing the image context and measuring the change of attention scores. Despite its remarkable performance, it requires two forward passes, leading to much computation overhead. Different from prior works, in this paper, we propose VISTA , a VI sion-aware S elf-improvement T raining framework that enhances the multimodal re A soning of MLLMs via two novel strategies. First, to alleviate data imbalance, inspired by prior findings ( Ji et al. 2025 ) – errors of failed solutions often occur in the later reasoning traces , we introduce a prefix resampling strategy that reuses the correct prefix contexts from failed solutions for efficient data collection. In practice, it first identifies partially correct reasoning traces by locating the critical tokens that exert a significant influence on subsequent reasoning steps, and then uses these traces as prefix context to resample new solutions. Second, to mitigate the language prior bias, we design a simple-yet-effective Vision-aware Attention Score (VAS) approach that leverages the model’s internal attention information to quantify its focus on the visual cues during reasoning. Intuitively, if the model allocates limited attention to the visual context, it may drift away from the visual cues, thus leading to potential visual hallucinations. Overall, by collecting more correct solutions for difficult queries and filtering the undesired solutions with lower VAS, VISTA can ensure the diversity and reliability of self-training data, thus bringing better reasoning performance for MLLMs. Extensive experiments on five cutting-edge MLLMs and five popular multimodal reasoning tasks show that VISTA not only outperforms all counterparts by a clear margin, i.e. , bringing up to +13.66% average performance gains against the initial reasoning models, but also works well in various post-training paradigms, e.g. , supervised fine-tuning and preference learning. More encouragingly, in-depth analyses prove that VISTA reduces the visual hallucinations and improves the out-of-distribution (OOD) performance effectively. To summarize, our contributions are three-fold: (1) We reveal two shortcomings of self-improvement training in MLLMs, and propose VISTA to address them and boost MLLMs’ multimodal reasoning capabilities without extra supervision. (2) VISTA introduces two simple-yet-effective strategies that can be adopted to various MLLMs and post-training settings. (3) Extensive results show that VISTA can significantly and consistently improve MLLMs’ multimodal reasoning performance and model generalization. 2 Revisiting Self-improvement in MLLMs 2.1 Preliminary Figure 2: (a) Distribution of the number of correct solutions in a single query. (b) Distribution of self-generated training samples for different difficulty levels, where level-1 denotes the simplest and level-4 denotes the hardest. (c) Attention allocation between system prompts, visual, and instruction tokens across different model layers. Here, we use the Qwen2.5-VL-3B-Instruct as the base model. Given a base MLLM \mathcal{M}_{base} and a multimodal dataset \mathcal{D}=\{(x_{i},y_{i})\} , where x_{i}=\{x^{\text{sys}}_{i},x^{\text{vis}}_{i},x^{\text{ins}}_{i}\} is the query and y_{i} is corresponding ground-truth

Read the complete text