RoboCrowd: Scaling Robot Data Collection through Crowdsourcing

Suvir Mirchandani, David D. Yuan, Kaylee Burns, Md Sazzad Islam, Tony Z. Zhao, Chelsea Finn, Dorsa Sadigh · 2024

Read RoboCrowd: Scaling Robot Data Collection through Crowdsourcing by Suvir Mirchandani, David D. Yuan, Kaylee Burns, Md Sazzad Islam, Tony Z. Zhao, Chelsea Finn, Dorsa Sadigh (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01915.txt.

Description

Abstract In recent years, imitation learning from large-scale human demonstrations has emerged as a promising paradigm for training robot policies. However, the burden of collecting large quantities of human demonstrations is significant in terms of collection time and the need for access to expert operators.

Catalog record

CREATORSuvir Mirchandani, David D. Yuan, Kaylee Burns, Md Sazzad Islam, Tony Z. Zhao, Chelsea Finn, Dorsa Sadigh
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-01915

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RoboCrowd: Scaling Robot Data Collection through Crowdsourcing Suvir Mirchandani, David D. Yuan, Kaylee Burns, Md Sazzad Islam, Tony Z. Zhao, Chelsea Finn, Dorsa Sadigh Abstract In recent years, imitation learning from large-scale human demonstrations has emerged as a promising paradigm for training robot policies. However, the burden of collecting large quantities of human demonstrations is significant in terms of collection time and the need for access to expert operators. We introduce a new data collection paradigm, RoboCrowd, which distributes the workload by utilizing crowdsourcing principles and incentive design. RoboCrowd helps enable scalable data collection and facilitates more efficient learning of robot policies. We build RoboCrowd on top of ALOHA [ 1 ] —a bimanual platform that supports data collection via puppeteering—to explore the design space for crowdsourcing in-person demonstrations in a public environment. We propose three classes of incentive mechanisms to appeal to users’ varying sources of motivation for interacting with the system: material rewards, intrinsic interest, and social comparison. We instantiate these incentives through tasks that include physical rewards, engaging or challenging manipulations, as well as gamification elements such as a leaderboard. We conduct a large-scale, two-week field experiment in which the platform is situated in a university café. We observe significant engagement with the system—over 200 individuals independently volunteered to provide a total of over 800 interaction episodes. Our findings validate the proposed incentives as mechanisms for shaping users’ data quantity and quality. Further, we demonstrate that the crowdsourced data can serve as useful pre-training data for policies fine-tuned on expert demonstrations—boosting performance up to 20% compared to when this data is not available. These results suggest the potential for RoboCrowd to reduce the burden of robot data collection by carefully implementing crowdsourcing and incentive design principles. Videos are available at https://robocrowd.github.io . I Introduction With the success of pre-training large models on massive Internet-scale datasets in fields such as natural language processing and computer vision, imitation learning (IL) has become a popular paradigm for training robot policies [ 1 , 2 , 3 , 4 , 5 ] . However, modern IL algorithms continue to have significant data requirements especially as tasks increase in number and variety—on the order of hundreds to thousands of demonstrations. For example, OpenVLA [ 5 ] was trained on 970K trajectories from the Open-X Embodiment dataset [ 6 ] , much of which was collected by expert human operators over the course of thousands of hours. This underscores the need for scalable methods of collecting robot data. Prior efforts to scale up real-world data collection range from leveraging videos of human activity [ 7 , 8 ] to pooling demonstration data across different institutions [ 9 , 10 , 6 ] . While the former approach—tapping into internet scale videos—can provide useful visual representations [ 11 , 12 , 13 ] , such methods often struggle in tasks beyond pick-and-place without substantial real robot data. On the other hand, pooling datasets across many tasks and embodiments [ 6 , 4 ] has amortized the cost of real-robot data collection to a degree, but expert operators are still required to collect data especially when new embodiments or tasks are added. Other works focus on how to reduce the time burden on data collectors or guide the collection strategy [ 14 , 15 , 16 ] , but these methods do not address the fundamental problem that demonstrations are still solely collected by researchers or designated operators for the express purpose of training robot policies. This aspect of robot data collection drastically differs from other modalities such as text or images, where large volumes of data are organically produced by people in their daily activities and are readily available on the web. To explore ways to scale up robot data collection, we ask: Who can effectively collect robot data, and how might they be incentivized to do so? Fig. 1: Example of incentivizing demonstrations in RoboCrowd . The principal P consists of a robot teleoperation setup, a designer, and a scene they have designed. The scene contains tasks that an agent I (a crowd user) can attempt, guided by incentives put in place by the designer. For example, a material reward—e.g., a candy in a bin—can motivate I to produce a successful trajectory for a bin-picking task, which the designer can add to a dataset. To tackle this problem, we look to a large body of work outside of robotics which studies strategies for incentivizing people in crowdsourced data labeling tasks [ 17 , 18 , 19 , 20 , 21 , 22 ] . The goal of these works is to align the incentives of crowdworkers with researchers’ goals of labeling a given dataset—for example, gamifying the data labeling process [ 19 ] and aggregating data by tapping into the “wisdom of the crowd.” Our key idea is to build a system that leverages similar ideas for robot data collection—i.e., aligning human incentives to provide robot demonstration data . However, prior strategies in human-computer interaction are designed for applications that work well with web interfaces, and applying them to robotics introduces several challenges. First, robot teleoperation traditionally requires access to physical hardware which is not readily available to crowdworkers. Second, the robot platform must be capable of performing complex tasks—in order to be engaging to users, as well as to collect useful data. At the same time, the system must be intuitive to onboard, since the vast majority of potential data providers have no teleoperation experience. Further, the system must be safe for novice users to operate. To address these challenges, we propose RoboCrowd, a framework for incentive design in the context of crowdsourced robot data collection. Our framework centers five key properties: public accessibility, capability, intuitiveness, safety, and gamification. Diving deeper into the incentive design problem, we incorporate three classes of incentives to appeal to users’ varying sources of motivation for interacting with the system. These include material rewards (i.e., physical rewards from tasks), intrinsic interest (i.e., motivation from engaging tasks), and social comparison (i.e., comparison to other users). To instantiate the framework, we build upon ALOHA [ 1 ] —a bimanual platform for robot teleoperation—to satisfy our need for capable hardware, and situate the system in a public space to enable access to general users. We design hardware enhancements and a user interface to make teleoperation intuitive and safe for non-experts. Fig. 1 illustrates an example of how incentives might shape user interactions with the system into useful data: a scene contains a bin of candies, and when a user successfully acquires a candy via teleoperation, they simultaneously contribute a trajectory to a bin-picking dataset. We deploy the system in a field experiment in which the robot is situated near a university café, where users participate in a self-guided, gamified data collection experience. We observe significant engagement with the system—over 200 individuals independently volunteered to provide a total of over 800 interaction episodes. We compile the crowdsourced interactions into a dataset and annotate each trajectory with quality scores and task labels. We additionally validate material rewards, intrinsic interest, and social comparison as incentive types for shaping user interactions with the robot—observing up to 2 \times the amount of data collection time

Read the complete text