OpenThinkIMG : Learning to Think with Images via Visual Tool Reinforcement Learning
Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, Yu Cheng · 2025
Read OpenThinkIMG : Learning to Think with Images via Visual Tool Reinforcement Learning by Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, Yu Cheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08617.txt.
Introduction
Zhaochen Su and co-authors teach vision language models to think with images in a 2025 paper. People solve visual problems by reaching for tools, zooming, cropping, measuring, but models rarely learn when to do the same. Their open framework, OpenThinkIMG, standardizes vision tool interfaces and generates practice trajectories for training.
Static imitation alone generalized poorly, so the team added reinforcement learning that rewards task success from tool feedback. On chart reasoning tests, the trained small agent beat its imitation only twin by about twenty nine points. It also topped supervised tool learning rivals and passed a flagship closed model by about nine points. The authors release the full stack for community use.
Catalog record
| CREATOR | Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, Yu Cheng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-08617
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OpenThinkIMG : Learning to Think with Images via Visual Tool Reinforcement Learning Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, Yu Cheng Abstract While humans can flexibly leverage interactive visual cognition for complex problem-solving, enabling Large Vision-Language Models (LVLMs) to learn similarly adaptive behaviors with visual tools remains challenging. A significant hurdle is the current lack of standardized infrastructure, which hinders integrating diverse tools, generating rich interaction data, and training robust agents effectively. To address these gaps, we introduce OpenThinkIMG , the first open-source, comprehensive end-to-end framework for tool-augmented LVLMs. It features standardized vision tool interfaces, scalable trajectory generation for policy initialization, and a flexible training environment. Furthermore, considering supervised fine-tuning (SFT) on static demonstrations offers limited policy generalization for dynamic tool invocation, we propose a novel reinforcement learning (RL) framework V-ToolRL to train LVLMs to learn adaptive policies for invoking external vision tools. V-ToolRL enables LVLMs to autonomously discover optimal tool-usage strategies by directly optimizing for task success using feedback from tool interactions. We empirically validate V-ToolRL on challenging chart reasoning tasks. Our RL-trained agent, built upon a Qwen2-VL-2B , significantly outperforms its SFT-initialized counterpart (+28.83 points) and surpasses established supervised tool-learning baselines like Taco and CogCom by an average of +12.7 points. Notably, it also surpasses prominent closed-source models like GPT-4.1 by +8.68 accuracy points. We hope OpenThinkIMG can serve as a foundational framework for advancing dynamic, tool-augmented visual reasoning, helping the community develop AI agents that can genuinely “think with images”. Code https://github.com/zhaochen0110/OpenThinkIMG 1 Introduction “The eye sees only what the mind is prepared to comprehend.” — Robertson Davies Recent advances in large vision-language models (LVLMs) have significantly expanded the capabilities of AI agents to jointly reason over visual and textual inputs Liu et al. (2023a) ; Zhu et al. (2023) ; Su et al. (2024a) . By leveraging techniques such as chain-of-thought (CoT) prompting ( Wei et al., 2022 ; Su et al., 2024b ) , these models have achieved impressive performance on a broad range of multimodal tasks, such as visual question answering ( Antol et al., 2015 ) , mathematical reasoning ( Lu et al., 2024 ) , and image captioning ( Sharma et al., 2018 ) . However, most current approaches still rely primarily on textual intermediate reasoning, even when dealing with inherently visual problems. In contrast, human reasoning is often deeply intertwined with visual cognition ( Zhang and Norman, 1994 ; Larkin and Simon, 1987 ) . People not only describe what we see but also think with images, using sketches ( Goel, 1995 ) , highlights ( Kosslyn, 1994 ) , and spatial cues ( Tversky, 2005 ; Tversky et al., 2002 ) to externalize, decompose, and manipulate complex visual information. For example, when solving a geometry problem, people will draw auxiliary lines or mark key points on a diagram to reveal hidden relationships and guide reasoning. These biological mechanisms motivate employing visual tools as cognitive scaffolds for LVLMs. By integrating tools directly into the reasoning loop, we enable models to iteratively manipulate and interpret visual content, providing a more grounded and interpretable decision-making pathway. This paradigm shift from pure text-based reasoning to tool-augmented visual cognition holds promise for solving tasks that demand fine-grained spatial understanding, iterative perception, and precise interaction with visual content. Recent efforts have begun to explore tool-augmented multimodal reasoning ( Hu et al., 2024 ; Ma et al., 2024a ; Fu et al., 2025 ) by equipping agents with the ability to interact with external visual tools, compose intermediate visual representations, and learn action trajectories through synthetic supervision ( Liu et al., 2023b ; Hu et al., 2024 ; Ma et al., 2024b ) . While demonstrating tool integration potential, these SFT-centric approaches, typically relying on orchestrated tool-use sequences from static datasets, limit holistic learning across the tool-use lifecycle. These approaches gives rise to several fundamental challenges: ❶ Heterogeneous tool definitions and interfaces : Tools with identical names (e.g., “segment” or “grounding”) often differ in behavior due to backend implementations or task-specific assumptions, hindering standardization and reproducibility. ❷ High cost of trajectory generation : Producing training data for tool-based reasoning is resource-intensive, often relying on manual templates or brittle heuristics that limit scalability and accuracy verification. ❸ Limited training generalization : Existing methods typically adopt SFT on offline trajectories. However, SFT alone struggles to generalize to unseen tools or tasks, and lacks mechanisms for exploration and dynamic adaptation ( Su et al., 2024c ; Jin et al., 2025 ; Chu et al., 2025a ; DeepSeek-AI, 2025 ) . To address these challenges, in this paper, we introduce OpenThinkIMG , the first comprehensive end-to-end framework unifying these critical stages for tool-augmented LVLMs. Specifically, OpenThinkIMG provides a unified infrastructure for standardizing heterogeneous tool interfaces, scaling the generation of tool-use trajectories, and supporting efficient training of multimodal agents. Beyond traditional SFT approaches, we further propose V-ToolRL , a reinforcement learning framework that enables models to autonomously explore and discover optimal tool usage strategies with vision tools. By tightly integrating flexible tool management, scalable trajectory synthesis, and dynamic agent adaptation, OpenThinkIMG offers a practical foundation for building next-generation LVLMs with enhanced visual reasoning capabilities. Our main contributions are summarized as follows: • We introduce OpenThinkIMG , the first open and extensible end-to-end framework for tool-augmented LVLMs. It features a unified registry for diverse vision tools and backbone models, a distributed deployment strategy for efficient and scalable tool inference, and an integrated E2E training pipeline that incorporates our proposed novel V-ToolRL methodology for adaptive tool use. All code and resources are publicly available and will be actively maintained to foster community collaboration and further development in tool-augmented reasoning. • We propose a scalable and adaptable three-stage pipeline for constructing high-quality vision tool-use trajectories. This pipeline leverages the model’s capabilities for initial action planning, performs automated tool call completion and rationale parsing, and incorporates multi-stage filtering with rule-based validation and human oversight to ensure data quality for both supervised fine-tuning and reinforcement learning. • We empirically validate V-ToolRL on complex chart reasoning tasks. Our approach boosts the performance of a 2B parameter base model by +29.83 accuracy points and surpasses larger 8B/13B open-source tool-augmented agents by an average of 12.7 points. Detailed experiments and qualitative studies further illustrate the learned tool-use efficiency, the development of complex reasoning narratives, and the superior interpretability of our method. Figure 1 : Overview of the OpenThinkIMG framework. Top: User requests are processed by an LVLM backbone, which generates tool requests managed by a central Tool Controller. The controller calls independently deploy vision tool services (e.g., Point, OCR), and their outputs are fed back to the LVLM for iterative reasoning. Bottom: Cold-Start initializes model via SFT on pre-generated trajectories. V-ToolRL employs K-turn rollouts where the model interacts with the tool environment to learn an adaptive policy, resulting in the final model. 2 OpenThinkIMG Framework In this section, we will detail the architecture of OpenThinkIMG , a comprehensive, community-driven framework designed to streamline the integration of vision tools, scale the synthesis of tool-use trajectories, and support efficient training of multimodal agents. It encompasses a unified registry for tools and models, a distributed deployment strategy for dynamic inference,