RoboMIND 2.0: A Multimodal, Bimanual Mobile Manipulation Dataset for Generalizable Embodied Intelligence
Chengkai Hou, Kun Wu, Jiaming Liu, Zhengping Che, Di Wu, Fei Liao, Guangrun Li, Jingyang He, Qiuxuan Feng, Zhao Jin, Chenyang Gu, Zhuoyang Liu, Nuowei Han, Xiangju Mi, Yaoxu Lv, Yankai Fu, Gaole Dai, Langzhe Gu, Tao Li, Yuheng Zhang, Yixue Zhang, Xinhua Wang, Shichao Fan, Meng Li, Zhen Zhao, Ning Li · 2025
Read RoboMIND 2.0: A Multimodal, Bimanual Mobile Manipulation Dataset for Generalizable Embodied Intelligence by Chengkai Hou, Kun Wu, Jiaming Liu, Zhengping Che, Di Wu, Fei Liao, Guangrun Li, Jingyang He, Qiuxuan Feng, Zhao Jin, Chenyang Gu, Zhuoyang Liu, Nuowei Han, Xiangju Mi, Yaoxu Lv, Yankai Fu, Gaole Dai, Langzhe Gu, Tao Li, Yuheng Zhang, Yixue Zhang, Xinhua Wang, Shichao Fan, Meng Li, Zhen Zhao, Ning Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-24653.txt.
Introduction
Chengkai Hou, Kun Wu, Jiaming Liu, Zhengping Che, and co-authors release RoboMIND 2.0, a large real-world dataset for training robots that use two arms and can move around. It holds more than three hundred thousand dual-arm trajectories across six robot types and hundreds of tasks. Tactile episodes and mobile manipulation runs add contact and travel.
Alongside the physical data, the team built digital twins of the real settings and released a simulated set as well. The aim is better generalization to long, two-handed tasks in messy places. Robot learning researchers get scale, variety, and paired real and virtual material in one resource.
Catalog record
| CREATOR | Chengkai Hou, Kun Wu, Jiaming Liu, Zhengping Che, Di Wu, Fei Liao, Guangrun Li, Jingyang He, Qiuxuan Feng, Zhao Jin, Chenyang Gu, Zhuoyang Liu, Nuowei Han, Xiangju Mi, Yaoxu Lv, Yankai Fu, Gaole Dai, Langzhe Gu, Tao Li, Yuheng Zhang, Yixue Zhang, Xinhua Wang, Shichao Fan, Meng Li, Zhen Zhao, Ning Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 104 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-24653
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RoboMIND 2.0: A Multimodal, Bimanual Mobile Manipulation Dataset for Generalizable Embodied Intelligence Chengkai Hou, Kun Wu, Jiaming Liu, Zhengping Che, Di Wu, Fei Liao, Guangrun Li, Jingyang He, Qiuxuan Feng, Zhao Jin, Chenyang Gu, Zhuoyang Liu, Nuowei Han, Xiangju Mi, Yaoxu Lv, Yankai Fu, Gaole Dai, Langzhe Gu, Tao Li, Yuheng Zhang, Yixue Zhang, Xinhua Wang, Shichao Fan, Meng Li, Zhen Zhao, Ning Liu, Zhiyuan Xu, Pei Ren, Junjie Ji, Haonan Liu, Kuan Cheng, Shanghang Zhang, Jian Tang Abstract While data-driven imitation learning has revolutionized robotic manipulation, current approaches remain constrained by the scarcity of large-scale, diverse real-world demonstrations. Consequently, the ability of existing models to generalize across long-horizon bimanual tasks and mobile manipulation in unstructured environments remains limited. To bridge this gap, we present RoboMIND 2.0, a comprehensive real-world dataset comprising over 𝟑𝟏𝟎 K 310\text{K} dual-arm manipulation trajectories collected across six distinct robot embodiments and 𝟕𝟑𝟗 739 complex tasks. Crucially, to support research in contact-rich and spatially extended tasks, the dataset incorporates 𝟏𝟐 K 12\text{K} tactile-enhanced episodes and 𝟐𝟎 K 20\text{K} mobile manipulation trajectories. Complementing this physical data, we construct high-fidelity digital twins of our real-world environments, releasing an additional 𝟐𝟎 K 20\text{K} -trajectory simulated dataset to facilitate robust sim-to-real transfer. To fully exploit the potential of RoboMIND 2.0, we propose MIND-2 system, a hierarchical dual-system framework optimized via offline reinforcement learning. MIND-2 integrates a high-level semantic planner (MIND-2-VLM) to decompose abstract natural language instructions into grounded subgoals, coupled with a low-level Vision-Language-Action executor (MIND-2-VLA), which generates precise, proprioception-aware motor actions. Extensive evaluations across six distinct robotic embodiments validate the effectiveness of our dataset and demonstrate that MIND-2 system significantly outperforms four single-task baselines (covering both 2D image and 3D point cloud modalities as well as four state-of-the-art VLA models. Furthermore, we observe that integrating tactile modalities yields measurable gains in fine-grained manipulation tasks. Finally, experimental results show that mixing real and simulated data during training consistently enhances physical execution performance, validating both the fidelity of our simulation benchmarks and the cost-efficiency of synthetic data augmentation. Our full dataset, simulation assets, and training code are publicly released to advance research in general-purpose robotic manipulation. 0 0 footnotetext: 1 Beijing Innovation Center of Humanoid Robotics, Beijing, China. Kun Wu: [email protected] ; Zhengping Che: [email protected] ; Jian Tang: [email protected] 2 School of Computer Science, Peking University, Beijing, China. Chengkai Hou: [email protected] ; Jiaming Liu: [email protected] ; Shanghang Zhang: [email protected] ∗ Co-first authors: Chengkai Hou, Kun Wu, Jiaming Liu, Di Wu, Fei Liao, Guangrun Li, Jingyang He, Qiuxuan Feng, Zhao Jin, and Zhengping Che; ‡ Co-first core authors: Chengkai Hou, Kun Wu, and Jiaming Liu; † Project lead: Zhengping Che; Other co-first authors are listed in alphabetical order; {}^{\text{\Letter}} Corresponding authors: Shanghang Zhang and Jian Tang. Figure 1 : Overview of the RoboMIND 2.0. We introduce RoboMIND 2.0, a large-scale dataset comprising 310K dual-arm trajectories collected from six heterogeneous robot embodiments, totaling over 1,000 hours. The dataset features rich modalities, including 12K tactile-enriched sequences and 20K mobile manipulation trajectories. Collected through a unified teleoperation and quality assurance pipeline, RoboMIND 2.0 ensures consistent proprioception and provides fine-grained natural language annotations. To support scalable training and evaluation, we release digital-twin USD assets and 20K simulation trajectories aligned with real-world tasks. Building on this foundation, we propose MIND-2, a dual-system controller that integrates a slow high-level planner MIND-2-VLM with a fast low-level policy MIND-2-VLA, enabling robust long-horizon mobile manipulation across diverse scenarios. 1 Introduction Manipulation is a core challenge in robotics, and attaining human-level proficiency in dynamic and dexterous manipulation tasks has long been a central goal of the field [ 51 , 8 , 7 , 61 ] . High-quality datasets hold the promise of enabling data-driven acquisition of complex and dexterous robotic skills [ 69 , 50 , 89 ] . By learning from rich, diverse, and task-relevant demonstrations, an effective imitation or supervised learning method should in principle be able to acquire highly proficient manipulation policies that are well-aligned with the physical constraints and dynamics of real-world deployment scenarios [ 84 , 99 , 17 , 32 , 51 , 61 , 60 , 8 , 7 , 26 ] . Just as human expertise is built through repeated observation and practice, robots rely on large-scale, high-fidelity datasets to generalize across tasks, objects, and environments—making dataset quality and coverage a cornerstone of scalable robot learning. While many publicly available robotic manipulation datasets claim to offer diversity, this diversity is typically confined to a single dimension—such as object variety, task types, environments, or robot embodiments—rather than providing comprehensive, multi-dimensional coverage essential for robust and generalizable policy learning. For instance, widely used benchmarks like Open X-Embodiment [ 68 ] , RH20T [ 27 ] , DROID [ 50 ] , and RoboMIND 1.0 [ 89 ] primarily consist of single-arm, fixed-base manipulation data and lack any examples of bimanual coordination, despite its prevalence in real-world scenarios. Recent efforts have begun to address this gap. Datasets such as AgiBot World [ 10 ] and Galaxea Open-World [ 45 ] introduce rich bimanual manipulation data with diverse tasks and realistic execution contexts, and notably include high-resolution tactile sensing—an important step toward multimodal perception. However, both rely exclusively on a single robot embodiment (typically a humanoid platform), which severely limits their utility for studying cross-embodiment generalization. On the other hand, RoboCOIN [ 92 ] expands embodiment diversity by collecting data across multiple dual-arm platforms and provides a digital twin for simulation. Yet, the per-embodiment task coverage remains sparse, with limited trajectories and scenarios per robot, hindering effective training of long-horizon, temporally extended policies. Compounding these limitations, the vast majority of existing datasets—including even the most recent ones—capture only visual observations and basic actuation states, omitting critical physical interaction signals such as tactile feedback. This absence not only reduces multimodal richness but also impairs a model’s ability to reason about contact, slip, and fine manipulation—capabilities central to dexterous robotic behavior. To contextualize these gaps, Table 1 presents a detailed comparison of RoboMIND 2.0 against a broad spectrum of contemporary datasets—including RT-1 [ 9 ] , BC-Z [ 43 ] , BridgeData V2 [ 23 ] , RoboSet [ 6 ] , BRMData [ 101 ] , Dabb-E [ 80 ] , and the aforementioned Open X-Embodiment [ 69 ] , AgiBot World [ 10 ] , and RoboCOIN [ 92 ] —across key axes such as trajectory scale, task and skill diversity, embodiment configuration, multimodal sensing, detailed annotation, and sim-to-real alignment. In our paper, we introduce RoboMIND 2.0, a comprehensive dataset of robotic manipulation comprising approximately 310K bi-manual operation trajectories from 759 manipulation tasks collected from six diverse robot platforms (e.g., Franka [ 33 ] , UR5e [ 75 ] , AgileX [ 74 ] , ARX [ 2 ] , Tien Kung [ 12 ] , and Tian Yi [ 4 ] ). Our manipulation tasks emphasize 129 fundamental robotic skills and involve 1,139 distinct objects, with around 97K examples of common “pick-and-place” operations on different type of objects and over 50K instances of interactive manipulation tasks such as switching, dragging, and tool use (see Figure 1 ). In terms of scenario distribution, our dataset covers a variety of settings, including domestic scenes