Advancing Multimodal Reasoning Capabilities of Multimodal Large Language Models via Visual Perception Reward

Tong Xiao, Xin Xu, Zhenya Huang, Hongyu Gao, Quan Liu, Qi Liu, Enhong Chen · 2025

Read Advancing Multimodal Reasoning Capabilities of Multimodal Large Language Models via Visual Perception Reward by Tong Xiao, Xin Xu, Zhenya Huang, Hongyu Gao, Quan Liu, Qi Liu, Enhong Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07218.txt.

Introduction

Tong Xiao and co-authors add a seeing reward to multimodal reasoning training in a 2025 paper. Multimodal large language models must first perceive an image well before they can reason about it. Past reinforcement methods rewarded correct answers but ignored perception itself. Tests show those methods leave perception weak, which caps reasoning gains.

Their method, Perception-R1, rewards the model for matching text notes about the image taken from good reasoning traces. A judging model scores that match during training. The reward lifts both perception and reasoning, they report. Tests across several benchmarks reach top performance on most, the authors say.

Catalog record

CREATORTong Xiao, Xin Xu, Zhenya Huang, Hongyu Gao, Quan Liu, Qi Liu, Enhong Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-07218

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Advancing Multimodal Reasoning Capabilities of Multimodal Large Language Models via Visual Perception Reward Tong Xiao, Xin Xu, Zhenya Huang, Hongyu Gao, Quan Liu, Qi Liu, Enhong Chen Abstract Enhancing the multimodal reasoning capabilities of Multimodal Large Language Models (MLLMs) is a challenging task that has attracted increasing attention in the community. Recently, several studies have applied Reinforcement Learning with Verifiable Rewards (RLVR) to the multimodal domain in order to enhance the reasoning abilities of MLLMs. However, these works largely overlook the enhancement of multimodal perception capabilities in MLLMs, which serve as a core prerequisite and foundational component of complex multimodal reasoning. Through McNemar’s test, we find that existing RLVR method fails to effectively enhance the multimodal perception capabilities of MLLMs, thereby limiting their further improvement in multimodal reasoning. To address this limitation, we propose Perception-R1, which introduces a novel visual perception reward that explicitly encourages MLLMs to perceive the visual content accurately, thereby can effectively incentivizing both their multimodal perception and reasoning capabilities. Specifically, we first collect textual visual annotations from the CoT trajectories of multimodal problems, which will serve as visual references for reward assignment. During RLVR training, we employ a judging LLM to assess the consistency between the visual annotations and the responses generated by MLLM, and assign the visual perception reward based on these consistency judgments. Extensive experiments on several multimodal reasoning benchmarks demonstrate the effectiveness of our Perception-R1, which achieves state-of-the-art performance on most benchmarks using only 1,442 training data. Our code and dataset will be available at https://github.com/tongxiao2002/Perception-R1 . 1 Introduction Multimodal reasoning is a fundamental capability for AI systems to solve complex real-world tasks and represents a critical step toward artificial general intelligence (AGI). Since the emergence of Multimodal Large Language Models (MLLMs), extensive studies have been proposed to enhance their multimodal reasoning capabilities. Early efforts focused on prompt engineering techniques ( zhang2023multimodal ; zheng2023ddcot ; zhang2024cocot ; mitra2024compositional ) , while subsequent works leveraged supervised fine-tuning (SFT) with high-quality reasoning datasets ( gao2023g ; shi2024math ; peng2024multimath ; zhang2024mavis ) . More recently, the success of reasoning models such as OpenAI-o1 ( jaech2024openai ) , DeepSeek-R1 ( guo2025deepseek ) , and Kimi-k1.5 ( team2025kimi ) has demonstrated the efficacy of large-scale reinforcement learning (RL), particularly RL with Verifiable Rewards (RLVR), in enhancing advanced reasoning capabilities of LLMs and MLLMs, thereby drawing significant attention from the research community. Motivated by these pioneering works, a growing number of studies ( meng2025mm ; peng2025lmm ; yang2025r1 ; huang2025vision ; deng2025openvlthinker ) have emerged to further advance the multimodal reasoning capabilities of MLLMs leveraging RLVR, which trains MLLMs solely with rewards derived from the correctness of the final answer. For instance, MM-Eureka ( meng2025mm ) and R1-VL ( zhang2025r1 ) successfully stabilized the training of RLVR in the multimodal domain by incorporating additional training tricks and mechanisms. R1-OneVision ( yang2025r1 ) and Vision-R1 ( huang2025vision ) first constructed a large-scale high-quality multimodal dataset by addressing the modality gap issue, then applied a training pipeline consisting of cold-start followed by RLVR to train MLLMs on the dataset. All these works effectively enhanced the multimodal reasoning capabilities of MLLMs, leading to substantial performance improvements across most benchmarks. Multimodal reasoning can be naturally decomposed into multimodal perception and logical reasoning ( amizadeh2020neuro ; zhou2024proreason ) , where multimodal perception is responsible for accurately understanding the multimodal input and supplying essential information for subsequent reasoning, thereby serving as the foundation for effective multimodal reasoning. Although RLVR-trained MLLMs demonstrate improved reasoning capabilities, our detailed analysis reveal that existing RLVR fails to effectively improve the multimodal perception capabilities of MLLMs, making it a major bottleneck that restricts their further advancement in multimodal reasoning. For example, as illustrated in Figure 1 , the original MLLM (left in Figure 1 ) makes severe multimodal perception errors (e.g., referring to “right triangle \triangle OAE ” that does not exist in the image), indicating its limited multimodal perception capabilities. Nevertheless, it still manage to guess the correct answer. This makes existing RLVR method, which optimize MLLMs solely based on answer accuracy, struggles to correct perception errors and may even reinforce this flawed reasoning path. Consequently, the resulting MLLM (middle in Figure 1 ) still exhibit weak multimodal perception capabilities similar to it original counterpart (e.g., referring to “ RG ” that does not exist), hindering the development of genuine multimodal reasoning capabilities. Figure 1 : A comparison of three MLLMs on a geometry problem. Both Qwen2.5-VL-7B-IT and its RLVR-trained variant make severe perception errors but manage to guess the answer, whereas our Perception-R1 first accurately describes the image and then solves the problem correctly. We attribute this challenge to the rewards sparsity for multimodal perception when training MLLMs with existing RLVR, making it difficult to effectively enhance the multimodal perception capabilities of MLLMs. To address this challenge, we propose Perception-R1, which incorporates a novel and effective visual perception reward into the multimodal RLVR training process. The visual perception reward provides an additional reward signal beyond answer accuracy, explicitly encouraging MLLMs to perceive visual content accurately, thereby alleviating reward sparsity in RLVR training and facilitating more effective multimodal reasoning by strengthening the MLLMs’ perceptual foundation. Specifically, we introduce visual annotations into RLVR as auxiliary references, encouraging MLLMs to generate perception-accurate responses that closely align with them during training. To obtain such visual annotations, we first collect CoT trajectories of problems with accurate final answers from state-of-the-art multimodal reasoning model, treating the visual information embedded in these correct trajectories as accurate owing to the model’s strong perception capabilities, and then employ an LLM to extract natural language visual annotations from them. During RLVR training, visual perception reward is assigned based on the consistency between the visual annotations and the responses of MLLM, as evaluated by a judging LLM via prompting. By incorporating the visual perception reward into RLVR, our Perception-R1 achieves new state-of-the-art (SOTA) performance on several multimodal math benchmarks using only 1,442 training samples, surpassing Vision-R1, which requires 200K data samples for training in total. In summary, our contributions are threefold: (1). We investigate the behaviors of RLVR-trained MLLMs and their original counterparts, and find that their multimodal perception capabilities are not statistically significantly different, remaining a major bottleneck that limits further advancement in multimodal reasoning. (2). We propose Perception-R1, which introduces a novel visual perception reward into RLVR. By providing an additional perception reward signal, Perception-R1 alleviates the reward sparsity in multimodal perception and effectively enhances the multimodal reasoning capabilities of MLLMs. (3). Extensive experiments on several multimodal benchmarks demonstrate the superiority of our proposed Perception-R1, which exhibits significantly improved multimodal perception capabilities and achieves SOTA performance on most benchmarks using only 1,442 training samples. 2 Related Work 2.1 Multimodal Large Language Models Multimodal Large Language Models (MLLMs) have witnessed rapid advancements in recent years. Most studies ( qwenvl ; Qwen2-VL ; Qwen2.5-VL ; llava2023 ; chen2024internvl ) developed MLLMs by aligning a visual encoder to a pre-trained LLM through vision-language adaptors (VL-adaptors), making the modality alignment the core of MLLMs development. Early efforts focused on architectural designs to enhance alignment, exploring various forms of VL-adaptors and visual encoders. From the perspective of VL-adaptor, three mainstream types have been widely studied: cross-attention modules ( qwenvl ; Dai2023InstructBLIPTG

Read the complete text