Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models
Xiaoyu Zhan, Wenxuan Huang, Hao Sun, Xinyu Fu, Changfeng Ma, Shaosheng Cao, Bohan Jia, Shaohui Lin, Zhenfei Yin, Lei Bai, Wanli Ouyang, Yuanqi Li, Jie Guo, Yanwen Guo · 2025
Read Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models by Xiaoyu Zhan, Wenxuan Huang, Hao Sun, Xinyu Fu, Changfeng Ma, Shaosheng Cao, Bohan Jia, Shaohui Lin, Zhenfei Yin, Lei Bai, Wanli Ouyang, Yuanqi Li, Jie Guo, Yanwen Guo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01618.txt.
Introduction
Xiaoyu Zhan and thirteen colleagues probe whether multimodal large language models, systems that read images and text, truly understand three-dimensional space. Two-dimensional image skill does not guarantee cross-view consistency: knowing how a scene looks from another angle. That gap is what their Viewpoint Learning task is built to measure and fix.
They release Viewpoint-100K, a dataset of 100,000 object-centered image pairs from diverse viewpoints with question-answer pairs. Training runs in two stages. First, supervised fine-tuning on the dataset injects viewpoint knowledge. Then reinforcement learning, with group relative policy optimization, broadens it to wider questions. A hybrid cold-start scheme keeps the reasoning coherent. The result is a marked activation of the baseline model's spatial reasoning across several tasks.
Catalog record
| CREATOR | Xiaoyu Zhan, Wenxuan Huang, Hao Sun, Xinyu Fu, Changfeng Ma, Shaosheng Cao, Bohan Jia, Shaohui Lin, Zhenfei Yin, Lei Bai, Wanli Ouyang, Yuanqi Li, Jie Guo, Yanwen Guo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-01618
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models Xiaoyu Zhan, Wenxuan Huang, Hao Sun, Xinyu Fu, Changfeng Ma, Shaosheng Cao, Bohan Jia, Shaohui Lin, Zhenfei Yin, Lei Bai, Wanli Ouyang, Yuanqi Li, Jie Guo, Yanwen Guo Abstract Recent advances in Multimodal Large Language Models (MLLMs) have significantly improved 2D visual understanding, prompting interest in their application to complex 3D reasoning tasks. However, it remains unclear whether these models can effectively capture the detailed spatial information required for robust real-world performance, especially cross-view consistency, a key requirement for accurate 3D reasoning. Considering this issue, we introduce Viewpoint Learning, a task designed to evaluate and improve the spatial reasoning capabilities of MLLMs. We present the Viewpoint-100K dataset, consisting of 100K object-centric image pairs with diverse viewpoints and corresponding question-answer pairs. Our approach employs a two-stage fine-tuning strategy: first, foundational knowledge is injected to the baseline MLLM via Supervised Fine-Tuning (SFT) on Viewpoint-100K, resulting in significant improvements across multiple tasks; second, generalization is enhanced through Reinforcement Learning using the Group Relative Policy Optimization (GRPO) algorithm on a broader set of questions. Additionally, we introduce a hybrid cold-start initialization method designed to simultaneously learn viewpoint representations and maintain coherent reasoning thinking. Experimental results show that our approach significantly activates the spatial reasoning ability of MLLM, improving performance on both in-domain and out-of-domain reasoning tasks. Our findings highlight the value of developing foundational spatial skills in MLLMs, supporting future progress in robotics, autonomous systems, and 3D scene understanding. 1 Introduction Multimodal Large Language Models (MLLMs) hurst2024gpt4o ; 2025gpto4mini ; team2023gemini ; wang2024qwen2vl ; bai2025qwen25vl ; chen2024intern2.5 ; chen2024internvl ; wu2024deepseekvl2 have recently achieved significant advances in visual understanding and inference. Naturally, the researchers song2025robospatial ; shiri2024empirical ; ray2024sat ; yang2024thinking demonstrate considerable interest in their abilities for spatial reasoning tasks. In the computer vision field, addressing 3D tasks typically requires first establishing cross-view consistency through methods such as camera calibration weng1992camera and stereo matching brown2003advances ; barnard1982computational . However, recent studies ranasinghe2024learning ; ray2024sat aim to enable MLLMs to directly perceive such consistency from multi-view images or sequential video frames, allowing accurate spatio-temporal reasoning. This development raises a critical question: Do these MLLMs have the potential to capture the fine-grained 3D spatial information needed for robust and reliable visual-spatial performance in real-world 3D scenarios? As a rule, all 3D objects maintain 3D consistency in space, characterized by stable spatial relationships and geometric properties. This consistency ensures that objects retain 2D continuity in timeline when projected onto a 2D plane. Our key concern is whether existing MLLMs, which are trained primarily on video data emphasizing 2D continuity, can achieve an understanding of spatial 3D consistency, as opposed to merely tracking continuous pixel-level evolution or correlated pixel mappings. Furthermore, the camera projection commonly introduces subtle distortions, imperceptible to humans, which complicate the establishment of relationships between 2D continuity and 3D consistency. Previous work yang2024thinking ; liu2024coarse ; zhang2025do has made it evident that current MLLMs still struggle to capture cross-view consistency. However, additional spatial information liu2024coarse ; daxberger2025mm and visual prompts liu20233daxiesprompts ; lei2025scaffolding can effectively improve their spatial reasoning ability. Although MLLMs do not seem to have acquired an explicit understanding of fundamental 3D mapping relationships, they demonstrate sensitivity to simple prompts that implicitly relate to these spatial concepts. We believe that the visual-spatial intelligence yang2024thinking of MLLMs has not been fully exploited due to inappropriate data utilization. Exploring how to effectively leverage existing data to teach MLLMs to reason and solve problems in 3D space represents a highly valuable research direction. In response to this challenge, we focus on a fundamental yet essential task, Viewpoint Learning , aimed at evaluating and activating the spatial reasoning ability in MLLM. Identifying viewpoints in image pairs or videos constitutes a pivotal step towards achieving an understanding of 3D consistency. This task is advantageous due to its straightforward data acquisition process, ease of ground-truth calculation, and simple evaluation metrics. Capitalizing on these benefits, we introduce the Viewpoint-100K dataset, which comprises 100K real-world, object-centric image pairs captured from distinct viewpoints, each paired with ego-centric or object-centric question-answer pairs (QAs). Figure 1: We aim to activate the MLLM’s spatial reasoning ability with Viewpoint Learning and the two-stage fine-tuning strategy. To effectively activate the spatial reasoning ability of MLLMs, we propose a two-stage fine-tuning strategy. The first stage is dedicated to the injection of foundational knowledge, emphasizing the critical importance of viewpoint understanding in both video comprehension and spatial reasoning. For this purpose, we use Supervised Fine-Tuning (SFT) with our Viewpoint-100K dataset, which ensures that the model develops a correct understanding of spatial relationships and viewpoint transformations. To maintain the coherent reasoning process and instruction-following behavior, we additionally employ a hybrid cold-start initialization enhanced by human-assisted pseudo chain-of-thoughts (CoTs). In the second stage, our aim is to preserve the acquired viewpoint-related knowledge while simultaneously improving the model’s generalization capacity across broader spatial tasks. We apply Reinforcement Learning (RL) through the Group Relative Policy Optimization (GRPO) algorithm shao2024deepseekmath , further fine-tuning the model on the SAT dataset ray2024sat , a synthetic dataset for spatial aptitude training. This phase is designed to refine the model’s ability to transfer knowledge from basic viewpoint tasks to more abstract and complex spatial reasoning challenges. It enables models to better perceive, interpret, and reason about 3D space, which are critical skills for deployment in real-world applications requiring advanced spatial reasoning ability. Our experiments across multiple benchmarks demonstrate the effectiveness of Viewpoint Learning in activating the spatial reasoning ability in MLLMs. In particular, this enhancement extends to out-of-domain inference tasks, showcasing the versatility and robustness of models trained with our approach. As MLLMs continue to evolve, fundamental tasks like Viewpoint Learning will play a crucial role in advancing their ability to understand and interact with the world in three dimensions, paving the way for more sophisticated applications in autonomous systems, robotics, and beyond. In summary, the main contributions of this work are threefold. • We introduce viewpoint learning, by which can activate the spatial reasoning ability in MLLMs, leading to strong out-of-domain generalization capabilities in visual and spatial reasoning. • We propose the Viewpoint-100K dataset, including 100K auto-generated ego-centric or object-centric QAs based on real-world, object-centric image pairs. • We employ a two-stage fine-tuning strategy that involves foundational knowledge injection and generalization enhancement, aiming to effectively achieve viewpoint learning. We further present a hybrid cold-start initialization method to maintain the reasoning thinking. 2 Related Work 2.1 Multimodal Large Language Models for Spatial Reasoning. MLLMs hurst2024gpt4o ; 2025gpto4mini ; team2023gemini ; wang2024qwen2vl ; bai2025qwen25vl ; chen2024intern2.5 ; chen2024internvl ; wu2024deepseekvl2 have demonstrated exceptional capability across various tasks. Recently, several studies liu2024coarse ; hong20233d ; daxberger2025mm ; man2024situational ; cai2024spatialbot ; shiri2024empirical ; song2025robospatial ; ranasinghe2024learning ; ray2024sat ; zhou2025r1zerosahamomentvisual ; huang2025mllm ; wu2025spatialmllm have been devoted to applying MLLMs to the field of 3D Reasoning. liu2024coarse introduces coarse, object-level correspondences into the input images, enhancing the spatio-temporal reasoning capabilities of MLLMs without the need for fine-tuning. This work reveals that current MLLMs struggle to capture cross-view consistency but that this limitation can be mitigated by providing additional cross-view information. Several recent studies hong20233d ; daxberger2025mm ; man2024situational ; cai2024spatialbot have similarly advanced the integration of MLLMs with 3D