MiMo-Embodied: X-Embodied Foundation Model Technical Report
Xiaomi Embodied Intelligence Team · 2025
Read MiMo-Embodied: X-Embodied Foundation Model Technical Report by Xiaomi Embodied Intelligence Team (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-16518.txt.
Introduction
Xiaoshuai Hao and 43 co-authors release MiMo-Embodied, an open foundation model built to serve both autonomous driving and embodied AI, the two fields that put learned models in physical motion. Most models specialize in one domain; this report tests whether a single model can lead in both at once.
MiMo-Embodied sets new records on 17 embodied AI benchmarks covering task planning, affordance prediction, and spatial understanding, and also leads on 12 driving benchmarks spanning perception, status prediction, and planning. The authors credit multi-stage training, curated data, and chain-of-thought plus reinforcement fine-tuning. They argue the two domains transfer positively to each other, and they publish the model, code, and a detailed account of the design for others to build on.
Catalog record
| CREATOR | Xiaomi Embodied Intelligence Team |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 80 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-16518
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MiMo-Embodied: X-Embodied Foundation Model Technical Report Xiaomi Embodied Intelligence Team Abstract We open-source MiMo-Embodied, the first cross-embodied foundation model to successfully integrate and achieve state-of-the-art performance in both Autonomous Driving and Embodied AI. MiMo-Embodied sets new records across 17 embodied AI benchmarks in Task Planning , Affordance Prediction and Spatial Understanding , while also excelling in 12 autonomous driving benchmarks across Environmental Perception , Status Prediction , and Driving Planning . Across these tasks, MiMo-Embodied significantly outperforms existing open-source, closed-source, and specialized baselines. Our results indicate that through multi-stage learning, curated data construction, and CoT/RL fine-tuning, these two domains exhibit strong positive transfer and mutually reinforce one another. We provide a detailed analysis of our model design and training methodologies to facilitate further research. Code and models are available at https://github.com/XiaomiMiMo/MiMo-Embodied . \contribution See Contributions and Acknowledgments section for a full author list. Figure 1 : Performance Comparison in Autonomous Driving and Embodied AI Benchmarks . MiMo-Embodied achieves state-of-the-art performance on both benchmarks, surpassing previous open-source, closed-source, and specialized VLMs, highlighting its superior capabilities in various autonomous driving and embodied AI tasks. 1 Introduction Vision-Language Models (VLMs) [ 25 , 50 , 5 , 55 , 20 , 8 ] demonstrate significant promise in advancing multimodal perception, understanding, and reasoning capabilities. Recently, some specialized embodied VLMs have emerged in domains such as autonomous driving and embodied AI, highlighting their interactions with the physical world [ 27 , 49 , 35 , 21 , 50 ] . Specifically, specialized VLMs for embodied AI, such as RoboBrain [ 27 , 49 ] and VeBrain [ 35 ] , emphasize individual capabilities like task planning and spatial understanding, delivering critical information for robots during physical interactions. Similarly, specialized VLMs in autonomous driving, such as RoboTron-Drive [ 24 ] and DriveLMM-o1 [ 26 ] , focus on enhancing specific aspects of driving performance, including environmental perception, status prediction, and driving planning, providing necessary information support for autonomous driving systems. While these specialized embodied VLMs have made significant progress, they are constrained by their narrowly defined application scenarios. The focus of embodied AI on indoor tasks and autonomous driving on outdoor roads creates a significant domain gap, which hinders the cross-domain generalization of capabilities. We summarize the challenges faced by specialized embodied VLMs as follows: (1) Lack of Unified Embodied VLMs . Existing VLMs focus on a single domain and lack a unified VLM that bridges autonomous driving and embodied AI. This fragmentation hinders the generalization of spatial understanding and reasoning across diverse indoor and outdoor scenarios, ultimately limiting the model’s capability to interact effectively with the physical world in dynamic environments. (2) Absence of Comprehensive Cross-Embodiment Capability Evaluation . Existing VLMs only assess partial capabilities in either autonomous driving or embodied AI, lacking a comprehensive cross-embodiment capability evaluation to assess the overall performance of specialized embodied VLMs. To address these challenges, we present MiMo-Embodied, a unified VLM that merges the tasks of autonomous driving and embodied AI into a single model. To our knowledge, MiMo-Embodied is the first open-source VLM to integrate these critical areas, significantly enhancing understanding and reasoning in dynamic physical environments. To evaluate MiMo-Embodied, we conduct a series of comprehensive cross-embodiment capability benchmarks to assess the overall performance of specialized embodied VLMs. As shown in Figure 2 , our MiMo-Embodied model focuses on key capabilities in both autonomous driving and embodied AI. For autonomous driving, we evaluate three core capabilities: (1) Environmental Perception : understanding traffic scenes through semantic and geometric analysis to scene comprehension, region-level interpretation and potential hazards detection; (2) Status Prediction : forecasting the behaviors of agents and multi-agent interactions to facilitate proactive decision-making; and (3) Driving Planning : generating safe driving maneuvers with explainable justifications based on traffic logic, ensuring compliance with road rules while optimizing safety and efficiency. For embodied AI, we evaluate three core capabilities: (1) Affordance Prediction : inferring actionable interaction possibilities from visual scenes to enable effective interactions; (2) Task Planning : translating abstract instructions into executable action sequences for accomplishing specific goals; and (3) Spatial Understanding : reasoning about spatial relationships, including directions, distances, and layouts, to facilitate navigation and interaction within the environment. Finally, MiMo-Embodied not only advances the state of specialized embodied VLMs but also sets a new standard for integrating diverse competencies, paving the way for more intelligent and adaptable systems in complex real-world applications. To equip MiMo-Embodied with the capability to handle both embodied AI and autonomous driving tasks, we constructed a diverse dataset of spanning general visual-language understanding, embodied tasks, and autonomous driving scenarios. This dataset provides the multimodal foundation for MiMo-Embodied’s core capabilities in perception, prediction, and planning for both autonomous driving and embodied AI. Based on the constructed dataset, we develop a progressive four-stage training strategy, which is critical to achieving state-of-the-art performance. The training stages comprises: Stage 1: General and Embodied Knowledge Learning establish core affordance understanding, high-level task planning, and spatial reasoning abilities based on general visual knowledge. Stage 2: Autonomous Driving Knowledge Learning integrates cross-domain understanding through mixed supervision from both autonomous driving and embodied data. Stage 3: Chain-of-Thought Fine-tuning enhances complex reasoning by incorporating generated rationales. Stage 4: Reinforcement Learning Fine-tuning further refines task-specific performance via GRPO [ 19 ] optimization. Extensive evaluations demonstrate MiMo-Embodied’s strength on 17 benchmarks in embodied AI, and leading performance across 12 benchmarks in autonomous driving, which outperforms current open-source and closed-source general VLMs, as well as specialized models. This report offers a comprehensive overview of the development of MiMo-Embodied, including its model architecture, pretraining dataset, training strategy, benchmark evaluation results, and application cases. The remaining sections of this report are organized as follows: Section 2 details the model architecture, establishing the foundation for our unified framework. Section 3 outlines the data curation and construction process across three categories: general multimodal understanding, embodied AI, and autonomous driving. Section 4 elaborates on the four-stage training strategy, which is crucial to achieving our state-of-the-art performance. Section 5 presents comprehensive quantitative and qualitative results, validating the superior performance of our model across various domains. Figure 2 : Overview of MiMo-Embodied Capabilities. MiMo-Embodied supports both Autonomous Driving and Embodied AI tasks, featuring 12 benchmarks in Autonomous Driving that cover Environmental Perception, Status Prediction and Driving Planning, along with 17 benchmarks in Embodied AI tasks focusing on Affordance Prediction, Task Planning, and Spatial Understanding. 2 Architecture Figure 3 : Model architecture of MiMo-Embodied. The MiMo-Embodied model architecture is designed for embodied AI and autonomous driving tasks, effectively processing single images, multiple images, and videos, and consists of three main components: (1) a Vision Transformer for encoding visual inputs; (2) a projector that maps visual encodings to a latent space aligned with a LLM; and (3) the LLM itself for textual understanding and reasoning. 2.1 Core Components of MiMo-Embodied The MiMo-Embodied architecture consists of three main components: (1) a Vision Transformer (ViT) for encoding visual inputs; (2) a projector that transforms these visual encodings into a latent space aligned with the Large Language Model (LLM); and (3) the LLM, which is responsible for text understanding and reasoning. To effectively process high-resolution inputs, we adopt the vision encoder from MiMo-VL [ 59 ] . Similarly, the LLM backbone and the projector are initialized