InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency

Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu,, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, Jingjing Xie · 2025

Read InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency by Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu,, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, Jingjing Xie (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-18265.txt.

Introduction

Weiyun Wang and a large team ship a new line of open vision-and-text models, InternVL3.5, built to run wide, think hard, and cost less. A two-stage reward training drives the thought gain: offline reward rounds first for a steady landing, then live rounds to hone the fit. Their 2025 line also routes image size by need, so vision marks shrink where detail is spare.

A split setup puts the vision half and the word half on split cards, which evens the load. The line gains 16.0 percent on reasoning grades and runs 4.05-fold faster next to its past form. New skills span screen acts and embodied acts in the world. The top form ties the best open peers and nears top commercial forms. All forms and code sit free.

Catalog record

CREATORWeiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu,, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, Jingjing Xie
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE103 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-18265

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu,, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, Jingjing Xie, Zehao Li, Bowen Yang, Yuchen Duan, Xuehui Wang, Zhi Hou, Haoran Hao, Tianyi Zhang, Songze Li, Xiangyu Zhao, Haodong Duan, Nianchen Deng, Bin Fu, Yinan He, Yi Wang, Conghui He, Botian Shi, Junjun He, Yingtong Xiong, Han Lv, Lijun Wu, Wenqi Shao, Kaipeng Zhang, Huipeng Deng, Biqing Qi, Jiaye Ge, Qipeng Guo, Wenwei Zhang, Songyang Zhang, Maosong Cao, Junyao Lin, Kexian Tang, Jianfei Gao, Haian Huang, Yuzhe Gu, Chengqi Lyu, Huanze Tang, Rui Wang, Haijun Lv, Wanli Ouyang, Limin Wang, Min Dou, Xizhou Zhu, Tong Lu, Dahua Lin,, Jifeng Dai, Weijie Su, Bowen Zhou, Kai Chen, Yu Qiao, Wenhai Wang, Gen Luo InternVL Team, Shanghai AI Laboratory Code: https://github.com/OpenGVLab/InternVL Model: https://huggingface.co/OpenGVLab/InternVL3_5-241B-A28B Abstract We introduce InternVL3.5 , a new family of open-source multimodal models that significantly advances versatility, reasoning capability, and inference efficiency along the InternVL series. A key innovation is the Cascade Reinforcement Learning (Cascade RL) framework, which enhances reasoning through a two-stage process: offline RL for stable convergence and online RL for refined alignment. This coarse-to-fine training strategy leads to substantial improvements on downstream reasoning tasks, e.g., MMMU and MathVista. To optimize efficiency, we propose a Visual Resolution Router (ViR) that dynamically adjusts the resolution of visual tokens without compromising performance. Coupled with ViR, our Decoupled Vision-Language Deployment (DvD) strategy separates the vision encoder and language model across different GPUs, effectively balancing computational load. These contributions collectively enable InternVL3.5 to achieve up to a +16.0% gain in overall reasoning performance and a 4.05 × \times inference speedup compared to its predecessor, i.e. , InternVL3. In addition, InternVL3.5 supports novel capabilities such as GUI interaction and embodied agency. Notably, our largest model, i.e. , InternVL3.5-241B-A28B, attains state-of-the-art results among open-source MLLMs across general multimodal, reasoning, text, and agentic tasks—narrowing the performance gap with leading commercial models like GPT-5. All models and code are publicly released. † † * equal contribution; 🖂 corresponding authors; \dagger technical leaders. Figure 1 : Comparison between InternVL3.5 and leading MLLMs in general capabilities. Hatched bars represent closed-source commercial models. We report average scores on a set of multimodal general, reasoning, text, and agentic benchmarks: MMBench v1.1 (en) [ 71 ] , MMStar [ 11 ] , BLINK [ 36 ] , HallusionBench [ 41 ] , AI2D [ 55 ] , OCRBench [ 72 ] , MMVet [ 168 ] , MME-RealWorld (en) [ 178 ] , MVBench [ 63 ] , VideoMME [ 35 ] , MMMU [ 170 ] , MathVista [ 76 ] , MathVision [ 134 ] , MathVerse [ 175 ] , DynaMath [ 189 ] , WeMath [ 100 ] , LogicVista [ 153 ] , MATH500 [ 45 ] , AIME24 [ 84 ] , AIME25 [ 85 ] , GPQA [ 106 ] , MMLU-Pro [ 146 ] , GAOKAO [ 177 ] , IFEval [ 185 ] , SGP-Bench [ 102 ] , VSI-Bench [ 161 ] , ERQA [ 121 ] , SpaCE-10 [ 38 ] , and OmniSpatial [ 50 ] . 1 Introduction The recent trend of Multimodal Large Language Models (MLLMs) [ 128 , 126 , 46 ] has gone beyond simple multimodal understanding and gradually focused on more general, complex, and realistic tasks such as text-related tasks [ 84 , 12 , 44 , 177 , 49 , 106 ] , reasoning tasks [ 161 , 134 , 76 , 170 , 43 ] and agentic tasks [ 101 , 48 , 156 , 174 , 160 , 116 , 50 , 161 , 38 ] . In these aspects, commercial models have created huge gaps with current open-source models, as shown in Table 2 . Thereby, recent open-source efforts [ 90 , 46 , 126 , 137 ] aim to explore advanced reinforcement learning (RL) methods to mitigate the gap and pursue higher multimodal intelligence. However, despite much effort in RL algorithms [ 109 , 183 , 90 , 23 , 24 , 171 , 133 , 152 ] and verifiers [ 143 , 136 , 82 ] , a stable, effective, and scalable reinforcement learning framework for MLLMs still remains an open problem in the community. Furthermore, the growth of multimodal capabilities, e.g., long visual context and high-resolution understanding [ 187 , 180 , 81 , 147 , 5 , 140 ] , often comes with ever increasing computational costs, which have become a crucial bottleneck of real-world applications. In this work, we introduce InternVL3.5, an advanced family of InternVL series [ 15 , 14 , 13 , 187 , 80 , 79 , 37 ] with stronger capabilities in versatility, reasoning, and efficiency. Compared to InternVL3 [ 187 ] , InternVL3.5 achieves superior performance through our proposed Cascade RL framework, which enhances reasoning capabilities in an efficient, scalable, and stable manner. Cascade RL consists of two complementary substages: an offline RL stage [ 142 ] , which efficiently achieves satisfactory performance, and an online RL stage [ 183 ] , which carefully refines the output distribution and further push the performance upper bound of the model. The offline stage serves as an effective warm-up, ensuring high-quality rollouts for the subsequent online stage, thereby enabling the progressive improvement of MLLM reasoning abilities. In practice, Cascade RL demonstrates promising salability and stability, with a clear gain seen from InternVL3.5-1B to InternVL3.5-241B (Figure 5 ). In addition, we equip InternVL3.5 with a much faster inference speed than its predecessor through two novel methods, namely Visual Resolution Router (ViR) and Decoupled Vision-Language Deployment (DvD). In particular, ViR aims to dynamically select the best trade-off resolution of visual tokens, thus reducing the inference costs with a negligible performance sacrifice. In practice, ViR can be efficiently integrated into InternVL3.5 with a light training stage namely Visual Consistency Learning (ViCO). Furthermore, DvD aims to deploy ViTs and LLMs on separate GPUs to maximize computational parallelism and hardware utilization. These two methods can be seamlessly combined to realize the hardware-friendly implementation for InternVL3.5. We conduct extensive experiments on public benchmarks to compare InternVL3.5 with existing MLLMs. As shown in Figure 1 , the InternVL3.5 series consistently maintains a leading position among open source MLLMs in terms of overall score. Compared to the latest commercial model, i.e., GPT-5 [ 98 ] , InternVL3.5-241B-A28B narrows the gap to 3.9%. In addition, our detailed ablation study demonstrates that InternVL3.5 achieves up to +16.0% improvement in overall reasoning performance and 4.05 \times speed-up in inference efficiency compared to its predecessor ( i.e. , InternVL3 [ 187 ] ). For example, InternVL3.5-8B and InternVL3.5-241B-A28B achieve scores of 73.4 and 77.7, respectively, on the MMMU benchmark [ 170 ] , showing their strong reasoning capabilities among existing open source MLLMs. In terms of versatility, InternVL3.5 remains competitive against both open-source and closed-source MLLMs in text tasks, GUI tasks, embodied tasks, SVG-based understanding and generation, etc . For example, InternVL3.5-30B-A3B and InternVL3.5-241B-A28B surpass the latest

Read the complete text