Robo-DM: Data Management For Large Robot Datasets
Kush Hari, Ashwin Balakrishna, Ted Xiao, Pannag R Sanketi, John Kubiatowicz, Ken Goldberg · 2025
Read Robo-DM: Data Management For Large Robot Datasets by Kush Hari, Ashwin Balakrishna, Ted Xiao, Pannag R Sanketi, John Kubiatowicz, Ken Goldberg (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15558.txt.
Introduction
Chen and co-authors build Robo-DM, a toolkit for storing and serving the huge data sets used to teach robots. Modern robot learning feeds on recordings of remote-controlled demonstrations, mixing video, text, and numbers from several cameras. Such data is bulky to curate, ship, and load. Robo-DM packs it into a self-contained format built on a standard media container.
Compression shrinks the data by up to seventy times with slight loss, or about three and a half times with none. Smart caching speeds up loading far beyond a rival framework. A model trained on the heavily compressed data still performed a pick-and-place task as well as before. The authors offer the open-source toolkit as plumbing for the era of giant robot data collections.
Catalog record
| CREATOR | Kush Hari, Ashwin Balakrishna, Ted Xiao, Pannag R Sanketi, John Kubiatowicz, Ken Goldberg |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-15558
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Robo-DM: Data Management For Large Robot Datasets Kush Hari, Ashwin Balakrishna, Ted Xiao, Pannag R Sanketi, John Kubiatowicz, Ken Goldberg Abstract Recent results suggest that very large datasets of teleoperated robot demonstrations can be used to train transformer-based models that have the potential to generalize to new scenes, robots, and tasks. However, curating, distributing, and loading large datasets of robot trajectories, which typically consist of video, textual, and numerical modalities - including streams from multiple cameras - remains challenging. We propose Robo-DM, an efficient open-source cloud-based data management toolkit for collecting, sharing, and learning with robot data. With Robo-DM, robot datasets are stored in a self-contained format with Extensible Binary Meta Language (EBML). Robo-DM can significantly reduce the size of robot trajectory data, transfer costs, and data load time during training. Compared to the RLDS format used in OXE datasets, Robo-DM’s compression saves space by up to 70x (lossy) and 3.5x (lossless). Robo-DM also accelerates data retrieval by load-balancing video decoding with memory-mapped decoding caches. Compared to LeRobot, a framework that also uses lossy video compression, Robo-DM is up to 50x faster when decoding sequentially. We physically evaluate a model trained by Robo-DM with lossy compression, a pick-and-place task, and In-Context Robot Transformer. Robo-DM uses 75x compression of the original dataset and does not suffer reduction in downstream task accuracy. Code and evaluation scripts can be found on website https://github.com/BerkeleyAutomation/fog_x . I Introduction Recent work [ 1 , 2 , 3 , 4 , 5 , 6 , 7 ] suggests Vision-Language-Action models [ 4 , 1 , 5 ] can enhance robot capabilities and generalization in handling multiple settings in diverse environments. A key ingredient for large model training is large and well-curated datasets of teleoperated robot demonstration trajectories such as the Open-X Embodiment (OXE) dataset [ 2 ] . However, the current management of robot data is inefficient [ 2 ] . Each robot demonstration consists of sequences of actions and observations, making the learning samples much larger and of diverse structrure compared to the images or text tokens in VLMs [ 8 , 9 , 10 , 11 , 12 ] and LLMs [ 13 , 14 ] . At tera or even penta-byte scale, which is sometimes characterized as Big Data [ 15 ] , existing robot data storage methods can be inefficient. We propose Robo-DM, an efficient data format with a toolkit for robot data collection, management, and training. Fig. 1 : Robo-DM can streamline robot data collection, management, and learning. (B) Robo-DM uses a unified format for vision, language, and action that does not rely on assumptions about timestamps and data, and supports plug-and-play data collection to integrate with existing setups. (C) Robo-DM can facilitate replay and visualization. (D) Existing training frameworks can load from Robo-DM efficiently with minimal modification. A typical robot dataset includes a number of episodes , a sequence of actions performed by an agent from a starting state to a terminal state. Each episode contains multiple sensor data streams in addition to language instructions and other metadata such as robot, task, environment, and control scheme specifications. The size of a typical episode ranges from 1 MB to 400 MB, depending on the episode length, compression level, number of cameras, and camera resolution. Data streams may be recorded at different sampling rates. Episodes are typically stored as a sequence of matrices; for example, data collection with DROID [ 6 ] automates data storage with Hierarchical Data Format 5 (HDF5) [ 16 ] , a format that supports hierarchical storage of matrices. OXE uses Reinforcement Learning Datasets (RLDS) [ 17 ] , an extension of Tensorflow Datasets (TFDS) to store reinforcement learning demonstrations. Storing image and sensor data directly in matrices is lossless but not space efficient. One emergent framework, LeRobot [ 18 ] , provides a platform to share robot models and datasets based on lossy video compression and HuggingFace datasets. However, its file structure is complex and loading is generally slower due to decoding. We observe the following challenges in robot data collection and usage: (A) Transmission Efficiency: Distributing robotics datasets is costly. Cloud service providers, such as Google Cloud Platform (GCP) and Amazon Web Services (AWS), charge the data host for both data storage and outbound data transfers. The cost of transferring data often exceeds the cost of storing it. For example, storing 8.9 TB of Open-X data on Google Cloud costs 172 US dollars per month, but every full download costs between 172 US Dollars and 1,540 US dollars. 1 1 1 The rate is calculated with the egress network traffic pricing in Google Cloud Platform (GCP), where the Open-X-Embodiment dataset is hosted. We use the size of Open-X v1.1 dataset with 8,964 GB in total. The rate differs by the downloading source and destination region. The rate does not consider retransmission of lost packets, so the actual cost is higher than the estimation. Directly training with cloud storage requires repeated downloads if the local storage cannot store the full dataset, further increasing network traffic and cost to the data host . Thus, improving data compression and transmission efficiency can reduce host costs and encourage public sharing of datasets. (B) Usability and Simplicity: Existing robot data frameworks impose restrictions on file structure, data layout, semantics, and alignment. In particular, hybrid approaches rely on framework-specific assumptions to handle multiple formats simultaneously. Extending the current framework or migrating between frameworks can be challenging, resulting in complex structure and file organization. Figure 2 shows a comparison of different storage formats. (C) Data Loading Performance: Large robot datasets are typically loaded into computationally training applications. In training, decoded frames are frequently reused and randomly accessed, and the decoded data is loaded on demand. Existing frameworks that use heavy compression sometimes lead to high computational resource utilization and interfere with the training performance. Thus, an efficient and perform ant data-loading framework should utilize available resources without contention. We introduce Robo-DM, an efficient cloud-based toolkit for collecting, sharing, and learning with robot data. Robo-DM streamlines storage for vision, language, and action data via a unified container format with Extensible Binary Meta Language (EBML). Robo-DM efficiently orchestrates heterogeneous data streams, supporting flexible lossless compression and lossy compression for enhanced transmission efficiency. In prior work, LeRobot [ 18 ] empirically evaluates how lossy video compression parameters in FFmpeg affect robot policy accuracy. Octo is also pre-trained by compressing image frames in OXE to lossy images [ 1 ] . Robo-DM improves the data loading performance for training workloads, which requires repetitive data access by using memory-mapped caching for faster data retrieval and loading. Loading from cache and decoding are load-balanced to maximize the utilization of compute, memory and storage resources. Robo-DM requires minimal integration effort with existing frameworks. It supports plug-and-play data collection, training, replay, and visualization with mainstream frameworks, and can also be easily exported to other formats such as HDF5 and RLDS. Experiments suggest that Robo-DM can reduce the size of data by up to 70 times with lossy compression compared to how Open-X-Embodiment currently shares the dataset, and up to 50x faster than LeRobot, a framework that also uses lossy video compression to encode vision data. We fine-tune Octo, a transformer-based robot policy