STAR: Learning Diverse Robot Skill Abstractions through Rotation-Augmented Vector Quantization

Hao Li, Qi Lv, Rui Shao, Xiang Deng, Yinchuan Li, Jianye Hao, Liqiang Nie · 2025

Read STAR: Learning Diverse Robot Skill Abstractions through Rotation-Augmented Vector Quantization by Hao Li, Qi Lv, Rui Shao, Xiang Deng, Yinchuan Li, Jianye Hao, Liqiang Nie (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-03863.txt.

Introduction

Hao Li and co-authors teach robots a vocabulary of skills that neither collapses nor forgets cause and effect. Turning smooth action into discrete skill codes promises reusable behavior. But the codebooks in standard models often collapse, with most codes going unused. The sequence of skills is also modeled without its causal thread. Their 2025 paper, STAR, repairs both faults.

A rotation trick in the quantization step encodes the angles between the encoder's outputs into the learning signal. Codes in a group are forced to spread apart or pull together as the gradients demand, which keeps the codebook alive. A causal transformer then models how each skill follows from the last, so composed behaviors stay coherent. On the LIBERO benchmark and on real robot tasks, STAR beat its baselines by about twelve percent.

Catalog record

CREATORHao Li, Qi Lv, Rui Shao, Xiang Deng, Yinchuan Li, Jianye Hao, Liqiang Nie
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-03863

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

STAR: Learning Diverse Robot Skill Abstractions through Rotation-Augmented Vector Quantization Hao Li, Qi Lv, Rui Shao, Xiang Deng, Yinchuan Li, Jianye Hao, Liqiang Nie Abstract Transforming complex actions into discrete skill abstractions has demonstrated strong potential for robotic manipulation. Existing approaches mainly leverage latent variable models, e.g., VQ-VAE, to learn skill abstractions through learned vectors (codebooks), while they suffer from codebook collapse and modeling the causal relationship between learned skills. To address these limitations, we present S kill T raining with A ugmented R otation ( STAR ), a framework that advances both skill learning and composition to complete complex behaviors. Specifically, to prevent codebook collapse, we devise rotation-augmented residual skill quantization (RaRSQ). It encodes relative angles between encoder outputs into the gradient flow by rotation-based gradient mechanism. Points within the same skill code are forced to be either pushed apart or pulled closer together depending on gradient directions. Further, to capture the causal relationship between skills, we present causal skill transformer (CST) which explicitly models dependencies between skill representations through an autoregressive mechanism for coherent action generation. Extensive experiments demonstrate the superiority of STAR on both LIBERO benchmark and realworld tasks, with around 12% improvement over the baselines. † † affiliationnotice: † † corresponding-author: Corresponding author STAR.github.io Figure 1: Comparison between naive VQ and our RaRSQ approach in the skill learning process. Top: Overview of skill quantization framework. Bottom: Visualization of gradient flow and codebook updates across three stages (before update, during gradient flow, and after update), where RaRSQ maintains geometric relationships between embeddings, leading to more diverse skills. 1 Introduction The challenge of modeling multitask visuomotor policy has long been a central problem in robotic manipulation ( Levine et al. 2016 ; Zhu et al. 2018 ) . Individual manipulation tasks already pose significant challenges like multimodal action distributions ( Mandlekar et al. 2021 ) , while these challenges are substantially amplified in the multitask setting. This results in a highly entangled action space where characteristics of different tasks interact and overlap, making it challenging to learn complex manipulation behaviors ( Lv et al. 2024 ) . An intuitive approach to alleviate this problem is to learn structured representations of manipulation behaviors by decomposing complex actions into simpler, reusable skill abstractions ( Fu et al. 2024 ; Sharma et al. 2019 ) . This hierarchical framework reflects the compositional structure inherent to manipulation tasks, enabling the systematic decomposition of complex behaviors into sequences of skill abstractions. Recent studies ( Ju et al. 2024 ; Wu et al. 2024 ) have demonstrated promising results using latent variable models (LVM) to discretize continuous action spaces into learned skills. These methods enable a more efficient representation and composition of complex behaviors. However, while this discretization paradigm transforms continuous actions into skills and provides a structured representation for complex behaviors, existing LVM-based methods face two critical limitations in skill learning and composition ( Garg et al. 2022 ; Lee et al. 2024 ) . First, techniques like VQ-VAE suffer from codebook collapse ( Mentzer et al. 2023 ; Roy et al. 2018 ) . Most codebook vectors remain unused during training, with only a small subset being frequently utilized for encoding diverse manipulation skills. This severely limits the capacity to capture the rich variety of robot behaviors. We argue that this limitation stems from the straight-through gradient estimator (STE) in VQ-VAE ( Van Den Oord et al. 2017 ) . During training, as illustrated in the middle column of Fig. 1 , STE assigns identical gradients to all encoder embeddings (hollow circles) that are quantized to the same codebook vector (orange solid circles). This oversimplified gradient assignment ignores the inherent geometric relationships between different embeddings within the same partition (regions separated by curved blue decision boundaries), leading to suboptimal codebook updates and eventual collapse of the representation space. Second, existing approaches struggle with effective skill composition, particularly in complex, long-horizon tasks that require precise coordination of multiple skills ( Mete et al. 2024 ) . While some methods adopt residual quantization (RQ) ( Zeghidour et al. 2021 ) to decompose skills into multiple levels for more precise representation, they fail to model the dependencies between different skill abstractions ( Lee et al. 2024 ) . This makes it difficult to generate coherent and temporally consistent actions for multi-stage manipulation tasks, where skills need to be carefully sequenced and composed. To address these fundamental challenges, we propose S kill T raining with A ugmented R otation ( STAR ), a novel framework that advances both skill learning and composition for robot manipulation. Our key insight is that encoding geometric relationships between action sequences into the residual quantization process is crucial for learning diverse and reusable skills, rather than relying on the oversimplified gradient assignment of straight-through estimation. Specifically, (1) to prevent codebook collapse , we devise rotation-augmented residual skill quantization (RaRSQ), which combines multi-level residual encoding with rotation-based gradient mechanisms. The residual structure progressively captures skills at different abstraction levels, while the rotation-augmented gradient flow enables points within the same skill code to be either pushed apart or pulled closer together based on their geometric relationships. Compared to naive VQ-VAE where similar embeddings are forced to have identical gradients, Fig. 1 demonstrates that RaRSQ prevents embeddings from collapsing to the same codebook vector, leading to more diverse skill representations; and (2) for effective skill composition , we present causal skill transformer (CST) which explicitly models dependencies between skill representations through an autoregressive mechanism for coherent action generation. By leveraging the hierarchical nature of residual quantization, CST sequentially predicts skill codes from coarse to fine levels and incorporates an offset prediction mechanism from BeT ( Shafiullah et al. 2022 ) to bridge the gap between discrete skills and continuous control. This combination of hierarchical prediction and continuous refinement enables precise control throughout extended sequences, making it particularly effective for long-horizon manipulation tasks. To summarize, our main contributions are as follows: • A rotation-augmented residual skill quantization (RaRSQ) mechanism that maintains diverse skill representations by encoding relative angular relationships in gradient updates, while achieving precise skill abstraction through hierarchical residual encoding. • A causal skill transformer (CST) that models skill dependencies through autoregressive prediction and enhances action precision via action refinement. • Comprehensive experimental validation across multiple benchmarks and real-world tasks, demonstrating substantial improvements in both skill learning efficiency and task performance. 2 Related Work Multi-task Imitation Learning. Multi-task robotic learning has been approached through various methods including supervised pre-training ( Sun et al. 2023 ; Wu et al. 2023 ; Li et al. ; Shen et al. 2024 ; Chen et al. 2024a ) , and large-scale demonstration learning ( Vuong et al. 2023 ; Brohan et al. 2023 ; Li et al. 2025 ) . Recent advances have explored the use of generative models, with frameworks like diffusion models ( Chi et al. 2023 ; Ze et al. 2024 ; Lv et al. 2025 ) and transformer-based approaches ( Pertsch et al. 2025 ; Bharadhwaj et al. 2024 ) showing promising results in handling multimodal action distributions. The Behavior Transformer (BeT) ( Shafiullah et al. 2022 ) demonstrated that policies operating in discretized action spaces can effectively model diverse behaviors, and introduced an

Read the complete text