PPGuide: Steering Diffusion Policies with Performance Predictive Guidance
Zixing Wang, Devesh K. Jha, Ahmed H. Qureshi, Diego Romeres · 2026
Read PPGuide: Steering Diffusion Policies with Performance Predictive Guidance by Zixing Wang, Devesh K. Jha, Ahmed H. Qureshi, Diego Romeres (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10980.txt.
Introduction
Zixing Wang and coauthors introduce PPGuide, a lightweight way to steer a pretrained diffusion policy away from failure while it acts. It needs no extra expert demonstrations or learned world model. Attention-based multiple instance learning first labels which observation-action chunks in the policy's own rollouts matter for success or failure. A performance predictor trains on that self-labeled data. At inference time, the predictor's gradient nudges the policy toward more robust actions. Results improve across Robomimic and MimicGen benchmark tasks.
Catalog record
| CREATOR | Zixing Wang, Devesh K. Jha, Ahmed H. Qureshi, Diego Romeres |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32514 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.10980
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PPGuide: Steering Diffusion Policies with Performance Predictive Guidance Zixing Wang, Devesh K. Jha, Ahmed H. Qureshi, Diego Romeres Abstract Diffusion policies have shown to be very efficient at learning complex, multi-modal behaviors for robotic manipulation. However, errors in generated action sequences can compound over time which can potentially lead to failure. Some approaches mitigate this by augmenting datasets with expert demonstrations or learning predictive world models which might be computationally expensive. We introduce Performance Predictive Guidance (PPGuide), a lightweight, classifier-based framework that steers a pre-trained diffusion policy away from failure modes at inference time. PPGuide makes use of a novel self-supervised process: it uses attention-based multiple instance learning to automatically estimate which observation-action chunks from the policy’s rollouts are relevant to success or failure. We then train a performance predictor on this self-labeled data. During inference, this predictor provides a real-time gradient to guide the policy toward more robust actions. We validated our proposed PPGuide across a diverse set of tasks from the Robomimic and MimicGen benchmarks, demonstrating consistent improvements in performance. I Introduction Diffusion policies [ 4 ] have been shown to be very powerful and efficient at learning complex, long-horizon multi-modal action distributions. Diffusion policies learn a conditional generative model using demonstration data and they can learn meaningful policies for a large variety of tasks. However, the stochastic nature of the underlying generative models can lead to compounding errors over time. Over long horizons, subtle errors in generated action chunks can compound, leading to catastrophic drift and task failure. This makes the learned diffusion policy brittle to slight variations during execution. In this paper, we present a guidance method that can steer the pre-trained denoising process to direct them towards more successful actions resulting in robust policies (see Figure ). It is noted that the proposed method does not require access to demonstration data and could be applied to pre-trained policy during inference. To improve robustness of diffusion policies, existing approaches typically fall into two categories. Data-centric methods [ 28 , 14 , 20 ] rely on dataset augmentation, such as increasing volume, enhancing diversity, or incorporating corrective demonstrations. While effective, these approaches require substantial human effort for data collection and annotation. Reward-based methods leverage dense task rewards when available, either through reinforcement learning fine-tuning [ 3 , 26 ] or residual action learning [ 37 , 2 , 9 , 34 ] . However, dense rewards are often unavailable or expensive to engineer in real-world scenarios. Recent advances in inference-time guidance [ 5 ] have opened new possibilities for policy steering. Several works [ 1 , 12 , 23 , 24 , 27 , 30 , 32 , 7 ] have demonstrated impressive performance gains by guiding diffusion policies at inference time by adjusting the denoising process. However, these methods typically require either dense reward signals [ 1 , 12 ] or accurate world models [ 27 , 7 ] , both of which may be unavailable or computationally prohibitive in practice. In this work, we present a classifier-based guidance-based framework that improves the robustness of pre-trained diffusion policies using only sparse, binary terminal rewards (e.g., success or failure). The core challenge is to estimate relevance using sparse success or failure signals: how can a sparse, trajectory-level outcome provide dense, actionable guidance for the policy at each timestep. Instead of relying on auxiliary models to estimate state value or out-of-distribution scores [ 30 , 7 ] , our approach directly learns to estimate which actions within a trajectory are most relevant to the final outcome. We draw inspiration from attention-based Multiple Instance Learning (MIL) in computer vision, where models learn to estimate specific regions of interest from weak image-level labels [ 16 , 11 , 15 ] . We hypothesize that a similar principle can estimate success-relevant and failure-relevant state-action subsequences within a robotic trajectory, using only a binary outcome label. Based on this insight, we present P erformance P redictive G uidance ( PPGuide ). We use a two-stage learning process: first, we use an attention-based MIL model to automatically discover and label which observation-action chunks are most predictive of task outcomes. Second, we train a lightweight relevance classifier on these self-generated labels. During inference, this classifier provides a dense guidance signal, a gradient with respect to the action, steering the diffusion policy away from actions associated with failure. This MIL-based labeling process creates a powerful self-supervised loop, solving the temporal relevance assignment/estimation problem without any manual annotation. PPGuide offers several advantages: (1) Data-efficient , it requires only sparse, binary success signals readily available in most robotic tasks; (2) Self-supervised , it is entirely self-supervised, learning from the policy’s own experiences without external supervision; (3) Lightweight , it adds minimal computational overhead during inference; and (4) Model-agnostic , applicable to any pre-trained diffusion model based policy without architectural changes. We validate PPGuide on a diverse suite of challenging manipulation tasks from the Robomimic benchmark [ 21 ] . Our results demonstrate that PPGuide substantially improves task success rates over the base diffusion policies, achieving consistent performance gain across different tasks. II Related Work II-A Steering Robot Control Policies Approaches for steering control policies can be broadly categorized by whether they modify the policy’s parameters through re-training or gradient-based guidance. Policy Re-training and Fine-tuning. One major family of approaches involves updating the policy’s weights. This includes interactive methods like DAgger [ 28 , 13 ] and DART [ 14 ] , which collect corrective data from human experts to enrich the training set. While effective, these methods require significant expert involvement. Another popular technique is to use reinforcement learning (RL) to fine-tune a policy’s parameters [ 26 , 3 ] or train a residual policy that corrects the base actions [ 37 , 2 , 9 ] . However, RL often requires extensive and potentially unstable training phases. Gradient-based Inference-Time Steering. To circumvent the costs of re-training, a second family of methods focuses on modifying the policy’s output at runtime with gradient-based guidance. The concept of steering a generative model with an auxiliary function has been highly successful, particularly in image synthesis. Classifier guidance [ 5 ] uses the gradient of a trained classifier p(y\mid x) to steer a generative model p(x) towards producing samples x that belong to a desired class y . This idea was central to large-scale diffusion models [ 10 ] . Gradient-based techniques include guided denoising, which injects gradient guidance from reward signals directly into the sampling process of diffusion policies [ 1 , 12 , 27 ] ; value-based filtering, which selects the best action from a set of proposals [ 24 ] ; and human-in-the-loop steering, where user constraints guide the sampler [ 32 , 35 ] . An important sub-category is predictive steering, which uses a world or dynamics model to anticipate future outcomes and steer the policy toward safe and robust actions [ 30 , 25 , 7 , 31 , 33 ] . PPGuide belongs to the inference-time steering category but introduces a distinct mechanism that requires neither dense rewards nor an explicit world model. The closest related work is Latent Policy Barrier (LPB) [ 30 ] , which uses a world model to predict future out-of-distribution