Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following

Vivek Myers \equal Bill Chunyuan Zheng \equal Anca Dragan Kuan Fang 1 Sergey Levine University of California, Berkeley 1 Cornell University · 2025

Read Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following by Vivek Myers \equal Bill Chunyuan Zheng \equal Anca Dragan Kuan Fang 1 Sergey Levine University of California, Berkeley 1 Cornell University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05454.txt.

Introduction

Vivek Myers, Bill Chunyuan Zheng, Anca Dragan, Kuan Fang, and Sergey Levine align robot representations in two thousand twenty five. Robots that learn basic tasks should compose them into longer ones. Good task representations make that composition natural. How to learn such representations was unclear.

Successor features with a temporal alignment loss do the job. Current and future state representations learn to associate over time. Compound tasks then emerge without explicit subtask planning. No reinforcement learning is needed for the gains shown. Tests span robot manipulation and simulation with language or image goals.

Catalog record

CREATORVivek Myers \equal Bill Chunyuan Zheng \equal Anca Dragan Kuan Fang 1 Sergey Levine University of California, Berkeley 1 Cornell University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-05454

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following Vivek Myers \equal Bill Chunyuan Zheng \equal Anca Dragan Kuan Fang 1 Sergey Levine University of California, Berkeley 1 Cornell University Abstract Effective task representations should facilitate compositionality, such that after learning a variety of basic tasks, an agent can perform compound tasks consisting of multiple steps simply by composing the representations of the constituent steps together. While this is conceptually simple and appealing, it is not clear how to automatically learn representations that enable this sort of compositionality. We show that learning to associate the representations of current and future states with a temporal alignment loss can improve compositional generalization, even in the absence of any explicit subtask planning or reinforcement learning. We evaluate our approach across diverse robotic manipulation tasks as well as in simulation, showing substantial improvements for tasks specified with either language or goal images. Temporal Representation Alignment Figure 1 : We show our Temporal Representation Alignment (TRA) method performing a language task, “put all food items in the bowl.” TRA adds a time-contrastive loss for learning task representations to use with a goal- and language-conditioned policy. While TRA can implicitly decompose the task into steps and execute them one by one, the behavioral cloning (BC) and offline RL (AWR) methods fail at this compositional task. The structured representations learned by TRA enable this compositional behavior without explicit planning or hierarchical structure. 1 Introduction Compositionality is a core aspect of intelligent behavior, describing the ability to sequence previously learned capabilities and solve new tasks ( Lashley,, 1951 ) . In domains involving long-horizon decision-making like robotics, various learning approaches have been proposed to enable this property, including hierarchical learning ( Kulkarni et al.,, 2016 ) , explicit subtask planning ( Schrittwieser et al.,, 2021 ; Fang et al., 2022b, ; Ahn et al.,, 2022 ) , and dynamic-programming-based “stitching” ( Ghugare et al.,, 2023 ; Kostrikov et al.,, 2022 ) . In practice, these techniques are often unstable or data-inefficient in real-world robotics settings, making them difficult to scale ( Laidlaw et al.,, 2024 ) . By contrast, humans and animals are adept at quickly composing behaviors to reach new goals ( Lashley,, 1951 ) . Possible explanations for these capabilities have been proposed, including the ability to perform transitive inference ( Ciranka et al.,, 2022 ) , learn successor representations and causal models ( Dayan, 1993b, ; Gopnik et al.,, 2017 ) , and plan with world models ( Vikbladh et al.,, 2019 ) . In common among these theories is the idea of learning structured representations of the world, which inference about which actions will lead to certain goals. How might these concepts translate to algorithms for robot learning? In this work, we study how adding an auxiliary successor representation learning objective affects compositional behavior in a real-world tabletop manipulation setting. We show that learning this representation structure improves the ability of the robot to perform long-horizon, compositionally-new tasks, specified either through goal images or natural language instructions. Perhaps surprisingly, we found that this temporal alignment does not need to be used for training the policy or test-time inference, as long as it is used as an auxiliary loss over the same representations used for the tasks ( Fig. 1 ). We compare our method, T emporal R epresentation A lignment (TRA), against past imitation and reinforcement learning baselines across a set of challenging multi-step manipulation tasks in the BridgeData setup ( Walke et al.,, 2023 ) as well as the OGBench simulation benchmark ( Park et al.,, 2025 ) . Unlike prior work in setup, we focus on the compositional capabilities of the robot policies: as a whole, the tasks are out-of-distribution, but each distinct subtask can be described through a goal image that lies in the training distribution. Adding a simple time-contrastive alignment loss improves compositional performance by > 40% across 13 tasks in 4 evaluation scenes, and simulation results show better performance compared to behavioral cloning (i.e., no structured representation learning), and comparable performance to offline RL methods that explicitly use a learned value function. 2 Related Work Our approach builds upon prior work on goal- and language-conditioned control, focusing particularly on the problem of compositional generalization. Robot manipulation with language and goals. Recent improvements in robot learning datasets have enabled the development of robot policies that can be commanded with image goals and language instructions ( Ahn et al.,, 2022 ; Walke et al.,, 2023 ; Shridhar et al.,, 2021 ) . These policies can be trained with goal- and language-conditioned imitation learning from human demonstrations ( Chowdhery et al.,, 2023 ; Jiang et al.,, 2023 ; Lynch and Sermanet,, 2021 ; Lynch et al.,, 2023 ; Brohan et al.,, 2023 ) , reinforcement learning ( Chebotar et al.,, 2023 ; Chen et al.,, 2021 ) , or other forms of supervision ( Bobu et al.,, 2023 ; Cui et al.,, 2023 ) . When trained to reach goals, methods can additionally use hindsight relabeling ( Andrychowicz et al.,, 2017 ; Kaelbling,, 1993 ) to improve performance ( Walke et al.,, 2023 ; Myers et al.,, 2023 ; Dehaene et al.,, 2022 ; Ding et al.,, 2019 ) . Our work shows how the benefits of goal-conditioned and language-conditioned supervised learning can be combined with temporal representation alignment to enable compositionality that would otherwise require planning or reinforcement learning. Compositional generalization in sequential decision making. In the context of decision making, compositional generalization refers to the ability to generalize to new behaviors that are composed of known sub-behaviors ( Rubino et al.,, 2023 ; Steedman,, 2004 ) . Biological learning systems show strong compositional generalization abilities ( Ciranka et al.,, 2022 ; Dehaene et al.,, 2022 ; Dickins,, 2011 ; Lake et al.,, 2019 ) , and recent work has explored how similar capabilities can be achieved in artificial systems ( Akyürek et al.,, 2021 ; Ito et al.,, 2022 ; Lewis et al.,, 2024 ) . In the context of policy learning, exploiting the compositionality of the behaviors can lead to generalization to unseen and temporarily extended tasks ( Ghugare et al.,, 2023 ; Kumar et al.,, 2023 ; Fang et al.,, 2019 ; Fang et al., 2022b, ; Mandlekar et al.,, 2021 ; Nasiriany et al.,, 2019 ) . Hierarchical and planning-based approaches also aim to enable compositional behavior by explicitly partitioning a task into its components ( Fang et al., 2022a, ; Myers et al., 2024a, ; Zhang et al.,, 2022 ; Park et al.,, 2023 ) . With improvements in vision-language models (VLMs), many recent works have explored using a pre-trained VLM to decompose a task into subtasks that are more attainable for the low-level manipulation policy ( Ahn et al.,, 2022 ; Attarian et al.,, 2022 ; Belkhale et al.,, 2024 ; Kwon et al.,, 2023 ; Myers et al., 2024a, ; Singh et al.,, 2023 ; Zhang et al.,, 2023 ) . These approaches are limited by the need for robust pre-trained models that can be fine-tuned and prompted for embodied tasks. Our contribution is to show compositional properties can be achieved without any explicit hierarchical structure or planning, by learning a

Read the complete text