Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning

Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, Wenhu Chen, University of Waterloo, HKUST, USTC, Vector InstituteProject Page: https://tiger-ai-lab.github.io/Pixel-Reasoner/ · 2025

Read Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning by Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, Wenhu Chen, University of Waterloo, HKUST, USTC, Vector InstituteProject Page: https://tiger-ai-lab.github.io/Pixel-Reasoner/ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15966.txt.

Introduction

Haozhe Wang and co-authors teach, in 2025, vision-language models to reason inside the image itself. Text-only chains of thought limit models on visual tasks, so the authors give models tools to zoom into regions and select video frames, inspecting evidence directly. Models, though, start reluctant to use unfamiliar tools.

Training runs in two phases: first, imitation of synthesized traces that demonstrate the visual operations, then reinforcement learning with a curiosity bonus that rewards exploring the new tools without abandoning text reasoning. The resulting seven-billion-parameter model scored strongly on several visual benchmarks. The authors argue pixel-space reasoning raises fidelity where pictures carry the answer.

Catalog record

CREATORAlex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, Wenhu Chen, University of Waterloo, HKUST, USTC, Vector InstituteProject Page: https://tiger-ai-lab.github.io/Pixel-Reasoner/
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-15966

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, Wenhu Chen, University of Waterloo, HKUST, USTC, Vector InstituteProject Page: https://tiger-ai-lab.github.io/Pixel-Reasoner/ Abstract Chain-of-thought reasoning has significantly improved the performance of Large Language Models (LLMs) across various domains. However, this reasoning process has been confined exclusively to textual space, limiting its effectiveness in visually intensive tasks. To address this limitation, we introduce the concept of reasoning in the pixel-space. Within this novel framework, Vision-Language Models (VLMs) are equipped with a suite of visual reasoning operations, such as zoom-in and select-frame. These operations enable VLMs to directly inspect, interrogate, and infer from visual evidences, thereby enhancing reasoning fidelity for visual tasks. Cultivating such pixel-space reasoning capabilities in VLMs presents notable challenges, including the model’s initially imbalanced competence and its reluctance to adopt the newly introduced pixel-space operations. We address these challenges through a two-phase training approach. The first phase employs instruction tuning on synthesized reasoning traces to familiarize the model with the novel visual operations. Following this, a reinforcement learning (RL) phase leverages a curiosity-driven reward scheme to balance exploration between pixel-space reasoning and textual reasoning. With these visual operations, VLMs can interact with complex visual inputs, such as information-rich images or videos to proactively gather necessary information. We demonstrate that this approach significantly improves VLM performance across diverse visual reasoning benchmarks. Our 7B model, Pixel-Reasoner , achieves 84% on V* bench, 74% on TallyQA-Complex, and 84% on InfographicsVQA, marking the highest accuracy achieved by any open-source model to date. These results highlight the importance of pixel-space reasoning and the effectiveness of our framework. 0 0 footnotetext: Corresponding to: [email protected], [email protected], [email protected] 1 Introduction Recent advancements have demonstrated remarkable progress in developing complex reasoning abilities in Vision-Language Models (VLMs). Leading models, such as OpenAI GPT4o/GPT-o1 ( Hurst et al. 2024 ; Jaech et al. 2024a ) , Gemini-2.5 ( Team et al. 2023 ) , VL-Rethinker ( Wang et al. 2025 ) achieve superior performance on various multimodal reasoning benchmarks like MathVista ( Lu et al. 2023 ) , MMMU ( Yue et al. 2024 ) , MEGA-Bench ( Chen et al. 2024 ) , etc. A common paradigm underpinning these state-of-the-art VLMs involves processing multimodal queries to extract relevant cues, followed by a reasoning process (CoT ( Wei et al. 2022 ) ) conducted purely in the textual format. Despite their success, the prevailing textual reasoning paradigm faces an inherent limitation: relying solely on text tokens to express intermediate reasoning steps can constrain the depth and accuracy achievable by Vision-Language Models (VLMs) on visually intensive tasks. The lack of direct interaction with visual inputs—such as drawing lines/marks, highlighting regions, or zooming in—hinders the model’s ability to interact with the information-rich images. As a result, VLMs often struggle to capture fine-grained visual details, including tiny objects, subtle spatial relationships, small embedded text, and nuanced actions in videos. Figure 1 : Illustration of Pixel Reasoner. When asked a visually-rich question, Pixel-Reasoner first inspects the visual inputs. Then it iteratively refines its understanding and evolve its reasoning by leveraging visual operations, such as zoom-in for images and select-frames for videos, ultimately arriving at a conclusion. Figure 2 : The Learning Trap. Our approach combines a warm-start instruction tuning phase and curiosity-driven RL phase to overcome the learning trap. These limitations motivates a fundamental rethinking of how VLMs engage with the visual modality during reasoning more seamlessly. This leads us to pose the research question: Can VLMs perform reasoning steps more directly within the visual modality itself, leveraging computational visual manipulations as actions to guide reasoning? We introduce the concept of pixel-space reasoning, proposing a novel paradigm where reasoning is not exclusively confined to verbalized format but actively incorporates operations applied directly to the visual inputs. Rather than solely translating visual observations into textual cues, the model could actively manipulate and interact with the visual information throughout the reasoning process – employing operations like ‘zoom-in’ , or ‘select-frame’ . These visual operations serve as integral steps within its reasoning chain, empowering the model to inspect, interrogate, and infer from visual evidence with enhanced fidelity. We frame this problem as developing a VLM endowed with a suite of visual operations. This novel framework involves strategically selecting and applying appropriate visual operations to the visual inputs, progressively refining its understanding, evolving its reasoning and ultimately arriving at a conclusion. To instill this novel capability of pixel-space reasoning, we follow the common post-training paradigm of instruction tuning and reinforcement learning (RL). However, cultivating pixel-space reasoning presents significant challenges. Firstly, existing VLMs exhibit limited zero-shot proficiency in executing pre-defined visual operations, thus requiring meticulous instruction tuning to establish a foundational understanding of these new visual operations. This initial training phase must also preserve the model’s inherent self-correction abilities, thereby preparing for trial-and-errors in the subsequent RL phase. Secondly, the warm-started model exhibits a significant disparity in proficiency between its well-established textual reasoning and its emergent pixel-space reasoning capabilities, which creates a "learning trap" that impedes the effective acquisition of pixel-space reasoning. On one hand, the model’s initial incompetence in visual operations garners more negative feedback than textual reasoning. On the other hand, a significant portion of training queries may not strictly necessitate visual operations, allowing the model to bypass these under-developed skills. These factors trap the cultivation of pixel-space reasoning, causing the premature cessation of efforts to utilize visual operations and improve pixel-space reasoning. To address these challenges, our approach combines a warm-start instruction tuning phase and a reinforcement learning phase. For instruction tuning, we synthesize 7,500 reasoning traces that facilitate the cultivation of both mastery over visual operations and self-correction capabilities. Following this meticulous warm-start instruction tuning, our RL approach leverages a curiosity-driven reward scheme to balance the exploration and exploitation of pixel-space reasoning to incentivize the pixel-level reasoning. The RL phase collect another 7,500 examples from several public image and video datasets ( Feng et al. 2025 ; Xu et al. 2025 ) . Our final model Pixel-Reasoner , built on top of Qwen2.5-VL-7B ( Bai et al. 2025 ) , is able to show significant improvement across several visual reasoning benchmarks (with information-rich images/video) like V* ( Wu and Xie 2024 ) , TallyQA ( Acharya et al. 2019 ) , MVBench ( Li et al. 2024a ) and InfographicsVQA ( Mathew et al. 2021 ) . On these benchmarks, Pixel-Reasoner shows best known open-source performance and even exceeds proprietary models like Gemini-2.5-Pro ( Team et al. 2024a ) and GPT-4o ( Hurst et al. 2024 ) . We further conduct comprehensive ablation studies to provide insights into how our framework effectively cultivates pixel-space reasoning. Our contributions are listed as follows: 1. We introduce the concept of pixel-space reasoning for the first time. 2. We identified a learning trap when cultivating this novel reasoning ability. 3. We proposed a novel two-staged post-training approach, featuring a meticulous instruction tuning stage, and a curiosity-driven RL stage. 4. We achieved state-of-the-art results on visually-intensive benchmarks with pixel-space reasoning. 2 Problem Formulation We introduce pixel-space reasoning , a novel paradigm enabling models to integrate operations directly applied to visual inputs, rather than solely relying on textual reasoning. Formally, consider a vision-language query \mathbf{x}=[V,L]

Read the complete text