MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning
Fanqing Meng ∗ Lingxiao Du ∗ Zongkai Liu ∗ Zhixiang Zhou ∗ Quanfeng Lu Tiancheng Han Daocheng Fu Botian Shi Wenhai Wang Junjun He Kaipeng Zhang Ping Luo Yu Qiao Qiaosheng Zhang † Wenqi Shao † Shanghai AI Laboratory Shanghai Innovation Institute Shanghai Jiao Tong University The University of Hong Ko · 2025
Read MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning by Fanqing Meng ∗ Lingxiao Du ∗ Zongkai Liu ∗ Zhixiang Zhou ∗ Quanfeng Lu Tiancheng Han Daocheng Fu Botian Shi Wenhai Wang Junjun He Kaipeng Zhang Ping Luo Yu Qiao Qiaosheng Zhang † Wenqi Shao † Shanghai AI Laboratory Shanghai Innovation Institute Shanghai Jiao Tong University The University of Hong Ko (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-07365.txt.
Introduction
Fanqing Meng and a large team train a model that reasons over both images and text with rule based reward learning, in a paper from 2025. Text only reason models gained a lot from such training, but image plus text reason work stayed small and easy.
The team built a hard math set with images, checked answers, and worked steps, drawn from many school fields. Their model, MM-Eureka, trained on that set with online filtering and a two stage plan for steady gains. It beat much larger open models on image math tasks. The full pipeline was shared in the open.
Catalog record
| CREATOR | Fanqing Meng ∗ Lingxiao Du ∗ Zongkai Liu ∗ Zhixiang Zhou ∗ Quanfeng Lu Tiancheng Han Daocheng Fu Botian Shi Wenhai Wang Junjun He Kaipeng Zhang Ping Luo Yu Qiao Qiaosheng Zhang † Wenqi Shao † Shanghai AI Laboratory Shanghai Innovation Institute Shanghai Jiao Tong University The University of Hong Ko |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-07365
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning Fanqing Meng ∗ Lingxiao Du ∗ Zongkai Liu ∗ Zhixiang Zhou ∗ Quanfeng Lu Tiancheng Han Daocheng Fu Botian Shi Wenhai Wang Junjun He Kaipeng Zhang Ping Luo Yu Qiao Qiaosheng Zhang † Wenqi Shao † Shanghai AI Laboratory Shanghai Innovation Institute Shanghai Jiao Tong University The University of Hong Kong Abstract DeepSeek R1, and o1 have demonstrated powerful reasoning capabilities in the text domain through stable large-scale reinforcement learning. To enable broader applications, some works have attempted to transfer these capabilities to multimodal reasoning. However, these efforts have been limited by the limited difficulty of selected tasks and relatively small training scales, making it challenging to demonstrate strong multimodal reasoning abilities. To address this gap, we introduce the MMK12 dataset and MM-Eureka with 7B and 32B parameters. The former is a high-quality multimodal mathematics reasoning dataset featuring diverse knowledge domains with human-verified answers and solution processes. The latter is a multimodal model employing rule-based reinforcement learning on MMK12, utilizing online filtering and two-stage training strategy to enhance training stability. MM-Eureka demonstrates remarkable performance gains in multimodal mathematical reasoning, outperforming previous powerful models like InternVL2.5-78B or InternVL2.5-38B-MPO. In particular, MM-Eureka achieves competitive or superior performance compared to both open-source and closed-source models, and trails slightly behind o1 in multidisciplinary reasoning tasks. We open-source our complete pipeline to foster further research in this area. We release all our codes, models, data, etc. at https://github.com/ModalMinds/MM-EUREKA † † footnotetext: \dagger Corresponding Authors: [email protected]; [email protected] * Equal contribution Figure 1: The overview of our proposed MMK12 and MM-Eureka. MMK12 training set has diverse multimodal mathematical questions with verified answer and process, while its evaluation set has multiple-choice questions for each discipline including math, physics, chemistry, and biology. MM-Eureka built on MMK12 has powerful performance in multimodal reasoning and it also exists aha-moment like DeepSeek R1. 1 Introduction Large-scale reinforcement learning (RL) ( Sutton et al. 1998 ) has demonstrated remarkable progress in improving the reasoning ability of Large Language Models (LLMs), particularly in the math and code domains ( OpenAI 2024 ; DeepSeek-AI et al. 2025 ) . Recent research, such as o1 ( OpenAI 2024 ) and DeepSeek-R1 ( DeepSeek-AI et al. 2025 ) , shows that large-scale RL can achieve breakthrough improvements in complex reasoning tasks during post-training phases, sometimes even without supervised fine-tuning (SFT) ( Radford et al. 2019 ) . Despite great success in the text domain, many real-world reasoning tasks such as interpreting scientific diagrams and geometrical reasoning can only be effectively solved with the image input. However, transferring large-scale RL techniques that work well for LLMs to multimodal scenarios remains underexplored. Recently, many works in the community have attempted to transfer the rule-based RL used in DeepSeek-R1 to multimodal scenarios. However, these works explore relatively small model sizes and fail to achieve stable training over extended periods like DeepSeek-R1. For instance, R1-V ( Chen et al. 2025 ) demonstrates improvements only in simple counting tasks, which covers limited complexity, while the model size remains modest. LMM-R1 ( Peng et al. 2025 ) achieves gains in accuracy reward for geometric reasoning; however, such success has not been verified in large-scale training with image-text data. Although Kimi k1.5 ( Team et al. 2025a ) has achieved competitive results in multimodal reasoning, it has not open-sourced its model or training data to the community. Therefore, how to achieve stable rule-based RL training in the multimodal reasoning domain remains an important and unsolved problem for the open-source community. To address this gap, as shown in Figure 1 , we first collect MMK12 * * * K12 refers to the full span of primary and secondary education, and the K12 evaluation sets used in our work primarily contain questions from the secondary level. , a high-quality and diverse multimodal mathematical reasoning dataset. problems. Subsequently, we implement MM-Eureka with two variants based on the MMK12 training set, i.e. MM-Eureka-7B built on Qwen2.5-VL-Instruct-7B and MM-Eureka-32B built on Qwen2.5-VL-Instruct-32B, which demonstrate excellent performance in multidisciplinary multimodal reasoning. For instance, MM-Eureka-7B achieves 66.1 on MMK12 evaluation sets, only 0.2 points below Intern2.5-VL-78B. On MathVista, it reaches 73.0 , even surpassing InternVL2.5-VL-78B. MM-Eureka-32B demonstrates stronger performance, scoring 72.3 on multidisciplinary K12 evaluation sets, which exceeds both Qwen2.5-VL-72B’s 70.3 and closed-source models like Gemini2-Flash, ranking second only to o1’s 73.9 . On commonly used multimodal mathematical reasoning benchmarks, MM-Eureka-32B achieves 73.4 on WeMath, outperforming all open-source models and most closed-source models including Claude3.7 Sonnet. On MathVista, it reaches 74.8 , surpassing all open-source and closed-source models. Both variants demonstrate significant improvements in multidisciplinary K12 and mathematical reasoning performance, outperforming most open-source models of similar sizes. Specifically, despite growing interest in multimodal reasoning models, high-quality datasets for multimodal mathematical reasoning remain scarce. For example, Geo3k ( Lu et al. 2021 ) and RCOT ( Deng et al. 2024 ) focus only on geometry problems. Although MAVIS ( Zhang et al. 2024b ) comprises data involving geometry and functions, it consists of synthetic data, lacking diversity in images and questions. To this end, we construct MMK12, a K12-level multimodal mathematical reasoning dataset. The training set covers a wide range of domains, including functions, geometry, and equations, spanning from elementary through high school curricula. For evaluation, MMK12 includes four disciplines: mathematics, physics, chemistry, and biology. To facilitate testing, we collect 500 multimodal multiple-choice questions for each discipline and carefully verify that there is no overlap with the training set. Subsequently, to achieve stable rule-based RL training over extended periods, we utilize the online filter strategy from PRIME ( Cui et al. 2025 ) that dynamically filters prompts with zero advantage during training, e.g., those answered either completely correctly or incorrectly. For MM-Eureka-32B, we further introduce a two-stage training strategy: the first stage leverages MMK12 by RL without KL divergence to develop the model’s general reasoning abilities while reducing training cost; the second stage fine-tunes the model on Geo3k with KL regularization to mitigate domain-specific deficiencies and enhance training stability. We use the GRPO ( DeepSeek-AI et al. 2025 ) as our basic RL approach, which offers higher efficiency compared to the commonly used PPO ( Schulman et al. 2017b ) . These strategies enable us to achieve stable, long-horizon RL training and consistent performance gains for both 7B and 32B model variants, while maintaining high training efficiency. Through the journey of developing MM-Eureka, we have several findings. First, it is difficult for the model to acquire new knowledge through RL training. Instead, performance improvements come from increasing the probability that the model generates correct answers during inference. Second, we discover that simple rule-based RL exhibits outstanding generalization capabilities. Training solely on mathematical data leads to simultaneous improvements in physics, chemistry, biology, and other disciplines. Third, rule-based RL generalizes better than other post-training strategies such as SFT ( Ouyang et al. 2022 ) and COT SFT ( Guo et al. 2024 ) across various tasks. Our goal is to share our implementation experiences and complete the open-source pipeline with the community, including data, code, and models. We believe this comprehensive open-source framework would help the community better explore the multimodal reasoning task. The main contributions are summarized as follows: • We utilize