CLIP-RT: Learning Language-Conditioned Robotic Policies from Natural Language Supervision

Gi-Cheon Kang Junghyun Kim Kyuhwan Shim Jun Ki Lee Byoung-Tak Zhang · 2024

Read CLIP-RT: Learning Language-Conditioned Robotic Policies from Natural Language Supervision by Gi-Cheon Kang Junghyun Kim Kyuhwan Shim Jun Ki Lee Byoung-Tak Zhang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00508.txt.

Description

Abstract Teaching robots desired skills in real-world environments remains challenging, especially for non-experts. A key bottleneck is that collecting robotic data often requires expertise or specialized hardware, limiting accessibility and scalability. We posit that natural language offers an intuitive and accessible interface for robot learning.

Catalog record

CREATORGi-Cheon Kang Junghyun Kim Kyuhwan Shim Jun Ki Lee Byoung-Tak Zhang
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-00508

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CLIP-RT: Learning Language-Conditioned Robotic Policies from Natural Language Supervision Gi-Cheon Kang Junghyun Kim Kyuhwan Shim Jun Ki Lee Byoung-Tak Zhang Abstract Teaching robots desired skills in real-world environments remains challenging, especially for non-experts. A key bottleneck is that collecting robotic data often requires expertise or specialized hardware, limiting accessibility and scalability. We posit that natural language offers an intuitive and accessible interface for robot learning. To this end, we study two aspects: (1) enabling non-experts to collect robotic data through natural language supervision ( e.g., “move the arm to the right”) and (2) training robot policies directly from this supervision. Specifically, we introduce a data collection framework that collects robot demonstrations based on natural language supervision and further augments these demonstrations. We then present CLIP-RT, a new vision-language-action (VLA) model that learns language-conditioned visuomotor policies from this supervision. CLIP-RT adapts the pretrained CLIP model and learns to predict language-based motion primitives via contrastive imitation learning. We train CLIP-RT on the Open X-Embodiment dataset and finetune it on in-domain data collected by our framework. In real-world evaluations, CLIP-RT demonstrates strong capabilities in learning novel manipulation skills, outperforming OpenVLA (7B parameters) by 24% in average success rates, while using 7x fewer parameters (1B). We further assess CLIP-RT’s capabilities in few-shot generalization and collaborative scenarios involving large pretrained models or humans. In simulated environments, CLIP-RT also yields strong performance, achieving a 93.1% average success rate on the LIBERO benchmark with an inference throughput of 163 Hz. I Introduction Building robots that can understand natural language instructions and perform various real-world tasks is a long-standing goal of robotics and artificial intelligence. The research community has studied such robots in various domains, such as robotic manipulation [ 37 , 6 , 7 ] , navigation [ 2 , 11 , 54 , 32 ] , and other instructions-following tasks [ 50 , 42 ] . † † * equal contribution; \dagger equal advising. One key challenge for intelligent robots is grounding natural language to vision and action, bridging the abstraction gap between natural language instruction and visuomotor control in real-world tasks. Prior works on robotic manipulation have addressed this challenge by training language-conditioned policies, primarily through imitation learning [ 53 , 35 , 51 , 26 , 37 , 6 , 7 ] . This line of research has shown remarkable success as large amounts of robotic data become available [ 41 ] . However, even state-of-the-art models [ 7 , 41 , 3 , 29 ] trained in large-scale robot data struggle to easily expand their set of manipulation skills for a wide range of real-world tasks. We argue that a major bottleneck lies in how robot demonstrations are typically collected. Specifically, obtaining real-world robot demonstration data often requires expertise in robot control or access to specialized hardware, such as teleoperation or virtual reality (VR) systems [ 58 , 18 ] . This barrier severely limits accessibility, restricting the number of participants and environments from which data can be gathered. Consequently, this limited accessibility inherently hinders both the scalability (the volume of data) and the diversity (the range of scenarios and behaviors recorded) of the resulting datasets. We thus ask: how can non-experts train robotic policies without relying on specialized expertise or devices for data collection? Fig. 1: Overview of language-guided teleoperation. We argue that natural language is an intuitive and accessible interface for robot learning. We thus explore a method for training robotic skills through natural language. To this end, we propose a data collection framework that enables non-experts to collect in-domain robot data through natural language. It consists of two steps: language-based teleoperation and stochastic trajectory augmentation (STA). Figure 1 illustrates language-based teleoperation in which a human collects data for a skill described in the instruction ( e.g., “pour the dog food into the bowl”). The human first provides natural language supervision ( e.g., “move left a lot”) in each state. The large language model (LLM) [ 39 ] then translates this supervision into appropriate robotic behavior, which is ultimately executed by the robot. By repeating this process, we obtain a collection of robot demonstrations, where each state transition is associated with corresponding language supervision. After the language-based teleoperation, STA augments the demonstration into alternative trajectories. Specifically, it stochastically drives the robot into novel states that were not explicitly covered in the original demonstrations. STA then automatically labels the appropriate behavior at these novel states using a simple heuristic. In other words, STA generates new trajectory data, expanding the diversity of the training dataset beyond the original demonstrations. We introduce a vision-language-action (VLA) model that learns language-conditioned visuomotor policies from natural language supervision, which we call CLIP-RT (CLIP-based Robotics Transformer). A key idea is to leverage natural language as supervision to train visuomotor policies—inspired by CLIP [ 45 ] , which uses language as a training signal for visual representation learning. CLIP-RT employs CLIP models trained in Internet-scale data [ 47 , 17 ] and directly adapts them to predict language-based motion primitives ( e.g., “move the arm forward by 10cm”) through contrastive imitation learning. Specifically, our model learns to measure the pairwise similarity between language supervision and contextual information ( i.e., current scene and language instruction) for language-conditioned policies. We train CLIP-RT through a two-step process: pretraining and in-domain fine-tuning. In the pretraining stage, we train our model on the large-scale robot learning dataset—Open X-Embodiment [ 41 ] —to improve generalization capabilities. The dataset does not contain language supervision, so we transform existing low-level robotic actions into templated natural language supervision to train CLIP-RT. During in-domain fine-tuning, CLIP-RT learns diverse robotic skills using our collected data. Our contributions are fivefold. First, we propose CLIP-RT, a vision-language-action (VLA) model that learns language-conditioned policies from natural language supervision. Second, we propose a data collection framework that enables non-experts to collect robot data only through natural language and augment the human-collected demonstration data. Third, experiments demonstrate that CLIP-RT outperforms OpenVLA [ 29 ] by 24% in average success rates in 9 novel manipulation tasks. We further observe two important results: (1) language-based motion prediction and STA boost generalization capabilities of CLIP-RT and (2) CLIP-RT effectively learns shared structures across diverse robotic tasks, resulting in generalizable and transferable policies. Fourth, we demonstrate that CLIP-RT’s language-based motion prediction capability enables collaboration with humans and large pretrained models [ 24 ] , resulting in improved generalization. Fifth, to validate the generality of our method, we adapt CLIP-RT and evaluate it on the LIBERO simulation benchmark [ 33 ] that includes offline, human-teleoperated demonstrations. CLIP-RT achieves strong results, an average success rate of 92.8%, with an improved inference throughput of 163Hz. II Related Work Vision-Language-Action (VLA) Models. Vision-language models (VLM) trained on Internet-scale data have been widely studied in robotics, including high-level planning [ 13 , 22 ] , success detection [ 14 ] , and physical reasoning [ 19 ] . In particular, previous work [ 7 , 41 , 3 , 29 ] directly fine-tunes VLMs to predict robotic actions. This category of models is called vision-language-action (VLA) models. CLIP-RT belongs to this category. Current VLA models discretize continuous action values ( e.g., end-effector actions) into discrete action tokens and learn to generate a sequence of

Read the complete text