Policy Contrastive Decoding for Robotic Foundation Models
Shihan Wu, Xu Luo, Ji Zhang, Junlin Xie, Jingkuan Song, Heng Tao Shen, Lianli Gao · 2025
Read Policy Contrastive Decoding for Robotic Foundation Models by Shihan Wu, Xu Luo, Ji Zhang, Junlin Xie, Jingkuan Song, Heng Tao Shen, Lianli Gao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13255.txt.
Introduction
Shihan Wu and co-authors fix a weakness of generalist robot policies: they pick up spurious correlations from their training demonstrations. A policy may watch the wrong visual cues and fail when the scene changes. The proposed fix, policy contrastive decoding, compares what the robot would do with the full image against what it would do with the key object masked out.
Contrasting the two action distributions steers the policy toward object-relevant clues. The method needs no retraining and no access to model weights, so it plugs into existing systems. The team tested it on three open robot policies, including one autoregressive model and two diffusion-based ones. Gains appeared in both simulation and the real world. On the strongest baseline, performance rose about 9 percent in simulation and more than doubled in real-world trials. Code and demos are public.
Catalog record
| CREATOR | Shihan Wu, Xu Luo, Ji Zhang, Junlin Xie, Jingkuan Song, Heng Tao Shen, Lianli Gao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13255
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Policy Contrastive Decoding for Robotic Foundation Models Shihan Wu, Xu Luo, Ji Zhang, Junlin Xie, Jingkuan Song, Heng Tao Shen, Lianli Gao Abstract Robotic foundation models, or generalist robot policies, hold immense potential to enable flexible, general-purpose and dexterous robotic systems. Despite their advancements, our empirical experiments reveal that existing robot policies are prone to learning spurious correlations from pre-training trajectories, adversely affecting their generalization capabilities beyond the training data. To tackle this, we propose a novel Policy Contrastive Decoding (PCD) approach, which redirects the robot policy’s focus toward object-relevant visual clues by contrasting action probability distributions derived from original and object-masked visual inputs. As a training-free method, our PCD can be used as a plugin to improve different types of robot policies without needing to finetune or access model weights. We conduct extensive experiments on top of three open-source robot policies, including the autoregressive policy OpenVLA and the diffusion-based policies Octo and π 0 \pi_{0} . The obtained results in both simulation and real-world environments prove PCD’s flexibility and effectiveness, e.g., PCD enhances the state-of-the-art policy π 0 \pi_{0} by 8.9 % in the simulation environment and by 108 % in the real-world environment. 1 Introduction Recent efforts in developing flexible, general-purpose, and dexterous robotic systems have largely focused on robotic foundation models, or generalist robot policies. The goal of such policies is to empower users to instruct robots to perform arbitrary tasks, enabling autonomous task execution with minimal human supervision. Specifically, the policy takes as input a visual observation of the robot’s state combined with a language instruction that defines the task, and outputs a robot action, such as end-effector displacement. Ongoing advancements in scaling real-world robotic data corpora have facilitated the development of numerous robot policies ( Brohan et al., 2022 ; Kim et al., 2024 ; M. et al., 2024 ; Pertsch et al., 2025 ) , which have demonstrated exceptional effectiveness in controlling various robots across diverse environments and acquiring a wide range of manipulation skills. Despite their notable achievements, existing robot policies are prone to learning spurious correlations ( Ye et al., 2024 ; Geirhos et al., 2020 ; Liu et al., 2023a ) from pre-training trajectories, adversely affecting their ability to generalize outside their training data. As illustrated in Fig. 1 , the robot policy predominantly relies on spurious features (e.g., background or textures) rather than object features within observations to predict actions (see (a)(d) ). Consequently, slight alterations to the visual observation background, like adjustments to the panning light region (i.e., (a) \xrightarrow{} (b) ) and the handle position (i.e., (a) \xrightarrow{} (c) ), lead to 36 % and 32 % drops in the policy’s action prediction performance, respectively. Prior work in other fields has demonstrated that models relying on spurious correlations often suffer significant performance degradation on test data when a distribution shift occurs between the training and test environments ( Lu et al., 2025 ; Varma et al., 2024 ; Izmailov et al., 2022 ) . This underscores the importance of redirecting robot policies’ focus from spurious features to object-relevant ones to enhance their generalization across different scenarios during inference. Therefore, we provide a first study on the following question: Can we propose a training-free, plug-and-play approach to mitigate the adverse effects of spurious correlations, thereby rectifying predicted actions for existing robot policies? In this work, we present a Policy Contrastive Decoding ( PCD ) approach to answer the question. The core idea of our PCD approach is to shift the robot policy’s attention from spurious features to object-relevant ones by contrasting action probability distributions derived from original and object-masked observation images during inference. To ensure both effectiveness and flexibility of the approach, a Tracking-to-Mask (Track2Mask) strategy is introduced, which first annotates target-specific objects in the initial visual observation using human expertise (e.g., Point and Box prompts) or off-the-shelf object detection models (e.g., Grounding DINO ( Liu et al., 2024b ) ), then employs the SAM2 ( Kirillov et al., 2023 ) model to track these objects across subsequent observations in the trajectory. This enables precise object masking with minimal or no human intervention. Moreover, unlike autoregressive policies, diffusion-based policies cannot directly generate action probability distributions to feed into our PCD. To cope with this limitation, we introduce a KDE-based Probabilistic Modeling (KDE-PM) scheme, which calculates action probability distributions for diffusion-based policies using the kernel density estimation (KDE) ( Wkeglarczyk, 2018 ) technique. This enables our PCD approach to be compatible with both autoregressive and diffusion-based policies. Figure 1: Robot policies tend to spuriously correlate task-irrelevant features with actions, compromising their ability to generalize to unseen scenarios. As observed, changing the light position from (a) to (b) and the drawer handle position from (a) to (c) results in 36 % and 32 % drops in the performance of the baseline policy OpenVLA ( Kim et al., 2024 ) , respectively. (d) Attention map. More results are in Section 4.4 and Appendix A . To the best of our knowledge, this is the first work to introduce a training-free approach for resolving the spurious correlation issue in robot policies. We conduct extensive evaluations in both simulation and real-world environments, across a total of 15 diverse tasks, and on top of three open-source robot policies—including the autoregressive policy OpenVLA ( Kim et al., 2024 ) and the diffusion-based policies Octo ( M. et al., 2024 ) and \pi_{0} ( Black et al., 2024 ) . The achieved results demonstrate the strong flexibility and effectiveness of our PCD approach. On simulation benchmarks, PCD enhances the performance of Octo , OpenVLA and \pi_{0} by 29.7 %, 50.6 % and 8.9 %, respectively. On real-world manipulation tasks, PCD outperforms the state-of-the-art robot policy \pi_{0} by 108 %. Contributions . Our contributions in this work are threefold. 1 ) We propose PCD, a simple, training-free, and easy-to-implement scheme to tackle the spurious correlation issue in robot policies. 2 ) PCD can be used as a plugin to enhance both autoregressive and diffusion-based policies, without requiring fine-tuning or access to pre-trained model weights. 3 ) Extensive experiments demonstrate PCD’s effectiveness and flexibility, consistently improving three state-of-the-art policies across 15 diverse tasks. 2 Related Work Robotic Foundation Models . The rapid development of robotic foundation models or generalist robot policies has been significantly inspired by the success of large visual and language models ( Karamcheti et al., 2024a ; Zhai et al., 2023a ; Touvron et al., 2023a ) . Robot policies can be categorized based on action prediction strategies into autoregressive models and diffusion-based policies. Autoregressive policies sequentially generate action tokens based on previous outputs ( Brohan et al., 2022 ; Brohan et al., 2023 ; Zawalski et al., 2024 ; Belkhale et al., 2024 ; Zhang et al., 2024 ) . OpenVLA ( Kim et al., 2024 ) equips with a Llama 2 ( Touvron et al., 2023a ) language model alongside a visual encoder that merges pretrained features from SigLIP ( Zhai et al., 2023a ) and DINO-v2 ( Oquab et al., 2024 ) . Diffusion-based policies simultaneously generate entire action dimensions by performing diffusion processes on noise ( M. et al., 2024 ; Liu et al., 2024a ) . Octo ( Team et al.,