RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration

Huajie Tan, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Yaoxu Lyu, Mingyu Cao, Zhongyuan Wang, Shanghang Zhang · 2025

Read RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration by Huajie Tan, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Yaoxu Lyu, Mingyu Cao, Zhongyuan Wang, Shanghang Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-03673.txt.

Introduction

Huajie Tan and co-authors release an open robot operating framework that lets unlike robots plan together under one brain and skill library, in work from twenty twenty-five. Single robot systems struggle to switch bodies, schedule long jobs, and fix errors live. Layered vision language models plan well but rarely coordinate teams.

Three parts do the work: a brain model for global seeing and decisions, a plug in skill library for execution, and a shared memory that syncs team states in space and time. Cloud links speed the chatter for large deployments. Real world trials across scenarios showed the system running mixed robot bodies on long tasks. The authors frame it as a shift to multi agent embodied intelligence.

Catalog record

CREATORHuajie Tan, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Yaoxu Lyu, Mingyu Cao, Zhongyuan Wang, Shanghang Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-03673

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration Huajie Tan, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Yaoxu Lyu, Mingyu Cao, Zhongyuan Wang, Shanghang Zhang Abstract The dawn of embodied intelligence has ushered in an unprecedented imperative for resilient, cognition-enabled multi-agent collaboration across next-generation industrial ecosystems, revolutionizing paradigms in autonomous manufacturing, adaptive service robotics, and cyber-physical production architectures. However, current robotic systems face significant limitations, such as limited cross-embodiment adaptability, inefficient task scheduling, and insufficient dynamic error correction. While End-to-end Vision Language Action (VLA) models demonstrate inadequate long-horizon planning and task generalization, hierarchical VLA models suffer from a lack of cross-embodiment compatibility and multi-agent coordination capabilities. To address these challenges, we introduce RoboOS , the first open-source embodied system built on a Brain-Cerebellum hierarchical architecture, enabling a paradigm shift from single-agent to multi-agent intelligence. Specifically, RoboOS consists of three key components: (1) the Embodied Brain Model (RoboBrain) , a multimodal large language model (MLLM) designed for global perception and high-level decision-making; (2) the Cerebellum Skill Library , a modular, plug-and-play toolkit that facilitates seamless execution of multiple skills; and (3) Real-Time Shared Memory , a spatiotemporal synchronization mechanism for coordinating multi-agent states. By integrating hierarchical information flow, RoboOS bridges the Embodied Brain and the Cerebellum Skill Library, facilitating robust planning, scheduling, and error correction for long-horizon tasks, while ensuring efficient multi-agent collaboration through Real-Time Shared Memory. Furthermore, we enhance edge-cloud communication and cloud-based distributed inference to facilitate high-frequency interactions and enable scalable deployment. Extensive real-world experiments across various scenarios, such as restaurants, households, and supermarket settings , demonstrate RoboOS’s versatility in supporting heterogeneous embodiments, including single-arm, dual-arm, humanoid, and wheeled . This capability offers a scalable and practical solution for cross-embodiment collaboration, advancing the frontiers of embodied intelligence. Project website: RoboOS . † † footnotetext: ∗ Equal contribution. † Project leader. {}^{\text{\Letter}} Corresponding author. Keywords: Embodied System, Multi-Robot Collaboration, Cross-Embodiment 1 Introduction The rapid evolution of embodied intelligence has ushered in a transformative era for industrial automation, service robotics, and smart manufacturing, where robust multi-agent collaboration has become essential [ 1 , 2 , 3 , 4 ] . Despite significant advancements, current robotic systems face persistent limitations, including poor cross-embodiment adaptability, inefficient task scheduling, and inadequate dynamic error correction. While End-to-end Vision Language Action (VLA) models like OpenVLA [ 5 ] , RDT-1B [ 6 ] , and \pi_{0} [ 7 ] demonstrate weak long-horizon planning and task generalization, hierarchical VLA frameworks such as Helix [ 8 ] , Gemini-Robotics [ 9 ] , GR00T-N1 [ 10 ] , Hi-Robot [ 11 ] and \pi_{0.5} [ 12 ] suffer from fragmented cross-embodiment compatibility and challenges in scalable multi-agent coordination. These issues highlight the urgent need for a unified system that bridges high-level cognition with low-latency execution while facilitating seamless collaboration among heterogeneous robots. To address these gaps, we introduce RoboOS , the first open-source embodied system built on a biologically inspired Brain-Cerebellum hierarchical architecture [ 13 , 14 , 15 ] , representing a paradigm shift from single-agent to multi-agent intelligence. RoboOS innovatively incorporates three groundbreaking components: (1) The Embodied Brain Model (RoboBrain [ 16 ] ) , a multimodal large language model (MLLM) that orchestrates global perception—including 3D scene reconstruction and historical state tracking—and high-level decision-making for multi-agent task decomposition and affordance-aware trajectory generation, while dynamically correcting errors through real-time replanning; (2) The Cerebellum Skill Library , a modular, plug-and-play toolkit that supports heterogeneous embodiments ( e.g. , single-arm, humanoid) with low-latency execution for manipulation (VLA-based tools [ 5 , 6 , 7 ] , Expert-based tools [ 17 , 18 ] ), navigation (VLN-based tools [ 19 , 20 ] , SLAM [ 21 , 22 ] ), and specialized skills [ 23 , 24 , 25 ] ; and (3) Real-Time Shared Memory , a spatiotemporal synchronization hub that maintains spatial memory ( e.g. , spatial relationship, locations of objects and robots), temporal memory ( e.g. , task feedback, tool-calling history), and embodiment memory ( e.g. , motion domain, joint states and battery levels) to enable fault prediction and load balancing across robots. Figure 1: Overview of RoboOS. Moreover, RoboOS optimizes scalability through edge-cloud communication and cloud-based distributed inference, ensuring high-frequency interactions and large-scale deployment of cloud inference with our Flagscale framework [ 26 ] . Extensive real-world validation in diverse scenarios—from industrial assembly to household services—demonstrates its versatility across heterogeneous robots, including dual-arm manipulators and wheeled platforms. For instance, to tackle a collaborative “apple-and-knife delivery” task, as shown in Fig. 1 , RoboOS dynamically allocates subtasks to three distinct robots ( Unitree Humanoid, AgileX Dual-arm, and RealMan Single-arm ) via shared memory, achieving seamless coordination through RoboBrain’s task decomposition and the Cerebellum’s skill execution. Our main contributions are summarized as follows: • We propose RoboOS , the pioneering open-source embodied system built on a Brain-Cerebellum Hierarchical Architecture , facilitating a transformative shift from single-agent systems to multi-agent intelligence. • We meticulously designed RoboOS with three core components: the Embodied Brain Model, the Cerebellum Skill Library, and Real-Time Shared Memory. Additionally, we optimized it for efficient edge-cloud communication and distributed inference, enhancing its overall performance and scalability. • Extensive real-world experiments across various scenarios—including restaurants, households, and supermarket settings —validate RoboOS’s adaptability and performance across diverse embodiments, such as single-arm, dual-arm, humanoid, and wheeled robots . This showcases its effectiveness in cross-embodiment collaboration and advances the capabilities of embodied intelligence, leading to practical and scalable solutions. 2 Related Work Multimodal Large Language Models Recent advancements in Vision-Language Models (VLMs) have demonstrated exceptional multimodal understanding. Proprietary models [ 27 , 28 , 29 , 30 ] and open-source alternatives [ 31 , 32 , 33 , 34 , 35 ] have set new benchmarks in visual question answering (VQA), image captioning, and multimodal dialogue through large-scale pretraining on image-text pairs. Reasoning-enhanced models like GPT-o1 [ 36 ] , DeepSeek-R1 [ 37 ] , and Kimi-1.5 [ 38 ] show that post-training reinforcement learning (RL) can significantly improve mathematical and coding abilities. RL-based reasoning MLLMs [ 39 , 40 ] also excel in multimodal reasoning tasks. However, transferring these capabilities to embodied intelligence systems is a challenge. Works like EmbodiedGPT [ 41 ] and RoboBrain [ 16 ] explore integrating visual-language understanding with robot-specific skills, including long-horizon task planning and trajectory synthesis. Extending reasoning-enhanced MLLMs to embodied scenarios [ 39 , 42 ] represents a promising research frontier. Vision-Language-Action (VLA) Models Building on the capabilities of VLMs [ 32 , 33 , 35 , 43 ] , researchers have developed vision-language-action (VLA) models for robotic manipulation tasks. Current VLAs are categorized as follows: End-to-End VLAs directly map visual-textual inputs to actions using unified architectures, employing regression-based [ 44 , 45 , 5 , 46 , 47 ] , diffusion-based [ 6 , 7 , 48 ] , and hybrid approaches [ 49 ] . While effective for short-term tasks, they struggle with long-horizon planning and generalization. Hierarchical VLAs address these limitations by using model-level [ 50 , 8 , 9 , 10 ] or task-level hierarchies [ 11 , 12 ] to decompose long-horizon tasks into subtasks. However, they still face challenges in cross-embodiment compatibility and multi-agent coordination. To tackle these issues, we introduce RoboOS, the first

Read the complete text