Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Corrections

Xiaomeng Xu, Yifan HouZeyi LiuShuran SongStanford University · 2025

Read Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Corrections by Xiaomeng Xu, Yifan HouZeyi LiuShuran SongStanford University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-16685.txt.

Introduction

Xiaomeng Xu and co-authors help robots learn touch-heavy tasks from human fixes in this 2025 paper. A common training loop lets a person correct a robot mid-task, then retrains on the fixes. In real settings, though, gathering clean corrections and using them well is hard.

Their system adds a soft-touch control that lets people nudge the robot gently without halting its motion. A residual policy then learns from those nudges while sensing force. Success rates rose sharply on four hard tasks, from flipping books to inserting gears. The method beat both fresh retraining and fine-tuning while using few corrections, the authors report.

Catalog record

CREATORXiaomeng Xu, Yifan HouZeyi LiuShuran SongStanford University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-16685

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Corrections Xiaomeng Xu, Yifan HouZeyi LiuShuran SongStanford University Abstract We address key challenges in Dataset Aggregation (DAgger) for real-world contact-rich manipulation: how to collect informative human correction data and how to effectively update policies with this new data. We introduce Compliant Residual DAgger (CR-DAgger), which contains two novel components: 1) a Compliant Intervention Interface that leverages compliance control, allowing humans to provide gentle, accurate delta action corrections without interrupting the ongoing robot policy execution; and 2) a Compliant Residual Policy formulation that learns from human corrections while incorporating force feedback and force control. Our system significantly enhances performance on precise contact-rich manipulation tasks using minimal correction data, improving base policy success rates by over 50% on two challenging tasks (book flipping and belt assembly) while outperforming both retraining-from-scratch and finetuning approaches. Through extensive real-world experiments, we provide practical guidance for implementing effective DAgger in real-world robot learning tasks. Result videos are available at: https://compliant-residual-dagger.github.io/ 1 1 footnotetext: Equal Contributions Figure 1 : CR-DAgger. To improve a robot manipulation policy, we propose a compliant intervention interface (a) for collecting human correction data, and use this data to update a compliant residual policy (b), and thoroughly study their effects by deploying the updated policy on two contact-rich manipulation tasks in the real world (c). 1 Introduction Learning from human demonstrations has seen many recent successes in real-world robotic tasks [ 8 , 19 , 37 , 42 ] . However, to obtain a successful policy, human demonstrators often have to repeatedly deploy a policy and observe its failure cases, then collect more data to update the policy until it succeeds. This process is broadly referred to as Dataset Aggregation (DAgger) [ 33 , 21 ] . However, doing DAgger effectively for real-world robotic problems still faces the following challenges: How to collect informative human correction data? DAgger is most effective when the correction data is within the original policy’s induced state distribution [ 33 ] . In practice, the common approach is either (1) collecting offline demonstrations that cover the policy’s typical failure scenarios [ 7 ] , or (2) human taking over robot control during policy deployment [ 34 , 30 ] . However, in both cases, it is difficult for human demonstrators to gain the intuition for how to execute the corrective actions without deviating excessively from the original distribution. Human taking over additionally introduces force discontinuity when they do not instantly reproduce the exact same robot force. This is partially due to the lack of effective correction interfaces that support precise and instantaneous intervention. How to effectively update the policy with new data? Prior methods for improving a pretrained policy with additional data include (1) retraining the policy from scratch with the aggregated dataset [ 21 ] , which can be computationally expensive; (2) finetuning the policy with only the additional data [ 37 , 15 , 6 ] , which is sensitive to the quality of the new data [ 45 ] , and (3) training a residual policy separately on top of the pretrained policy, which is typically done with Reinforcement Learning [ 1 , 45 ] or Imitation Learning [ 4 ] , both require a large number of samples. In this work, we address these questions by proposing an improved system Compliant Residual DAgger (CR-DAgger) consisting of two critical components: • Compliant Intervention Interface. We propose an on-policy correction system based on kinesthetic teaching to collect delta action without interrupting the current robot policy . Leveraging compliance control, the interface lets humans directly feel the magnitude of their instantaneous correction, so they can provide gentle adjustments. Unlike take-over corrections that may cause force discontinuity, our design allows smooth transition between correction/no correction mode, while maintaining distributional consistency with the original policy. • Compliant Residual Policy. Leveraging the force feedback from our Compliant Intervention Interface, we propose a residual policy formulation that takes in an extra force modality and predicts both residual motions and target forces , which can fully describe the human correction behavior. The Compliant Residual Policy is force-aware, even when the base policy is position-only. We show that our residual policy formulation learns effective correction strategies using the data collected from our Compliant Intervention Interface. Together, our system significantly improves the success rate of precise contact-rich robot manipulation tasks using a small amount of additional data. We demonstrate the efficacy of our method on two challenging tasks with long horizons and sequences of contacts: book flipping and belt assembly. We improve over the base policy success rate by over 50% using less than 50 intervention episodes, while also outperforming retrain-from-scratch and finetuning under the same data budgets. In summary, our contributions are: • A Compliant Intervention Interface , a system that allows human to provide accurate, gentle, and smooth corrections in both position and force to a running robot policy without interrupting it. • A Compliant Residual Policy , a policy formulation that seamlessly integrates additional force modality inputs and predicts residual motions and forces. • A practical guidance for efficient DAgger based on extensive real-world studies for critical but often overlooked design choices such as batch size and sampling strategy. Our hardware design, training data, and policy code will be made publicly available. 2 Related Work Human-in-the-Loop Corrections for Robot Policy Learning . The original DAgger work [ 33 ] requires the demonstrator to directly label actions generated by the policy. In robotics, a practical variation is to let the human take over the robot control and provide correct action directly [ 21 ] . Such human correction motion can be recorded with spacemouse [ 6 , 27 ] , joystick [ 34 ] , smartphone [ 30 ] , or arm-based teleoperation system [ 16 , 17 , 37 ] . We instead proposes a novel kinesthetic teaching system with compliance controller that allows the demonstrator to apply delta corrections while the robot policy is still running, and additionally records force feedback. Our results show that both the delta correction data and the force data are crucial to the success of the learned policy. Improving Pretrained Robot Policies with New Data . The most direct approach to improving a pretrained policy with new correction data is to retrain the policy on the aggregated dataset, combining prior demonstrations with new feedback [ 30 , 34 ] . Alternatively, Reinforcement Learning (RL) offers a framework to incorporate both offline and online data, either by warm-starting replay buffers [ 29 , 2 ] or by using offline data to guide online fine-tuning [ 43 , 38 ] . When policies are trained on large-scale human demonstration datasets [ 5 , 35 , 23 , 32 , 22 , 39 , 40 ] , retraining becomes impractical, especially when the original data is inaccessible. In such cases, fine-tuning with only the new data is a common solution, using either imitation learning [ 27 , 15 , 37 ] or RL [ 31 , 6 ] . Another line of work introduces an additional residual model on top of the original policy. These residual policies can be trained with RL in simulation [ 45 , 1 ,

Read the complete text