Dexterous Manipulation Based on Prior Dexterous Grasp Pose Knowledge

Hengxu Yan, Haoshu Fang, Cewu Lu · 2024

Read Dexterous Manipulation Based on Prior Dexterous Grasp Pose Knowledge by Hengxu Yan, Haoshu Fang, Cewu Lu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-15587.txt.

Introduction

Robot hands with many joints can handle tools like human hands, but teaching them each grip takes long training. Hengxu Yan, Haoshu Fang, and Cewu Lu split the task in two in a 2024 paper. First, their system finds a strong grip pose on the working part of the object, based on prior grip knowledge.

Then a learning agent explores moves from that grip, instead of starting blind. Most training time, the team found, goes to finding the right start pose and view. Starting from a good grip cut that waste. Across four tasks, learning ran faster and success rates rose. The split offers a clean way to teach nimble hands.

Catalog record

CREATORHengxu Yan, Haoshu Fang, Cewu Lu
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-15587

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dexterous Manipulation Based on Prior Dexterous Grasp Pose Knowledge Hengxu Yan, Haoshu Fang, Cewu Lu Abstract Dexterous manipulation has received considerable attention in recent research. Predominantly, existing studies have concentrated on reinforcement learning methods to address the substantial degrees of freedom in hand movements. Nonetheless, these methods typically suffer from low efficiency and accuracy. In this work, we introduce a novel reinforcement learning approach that leverages prior dexterous grasp pose knowledge to enhance both efficiency and accuracy. Unlike previous work, they always make the robotic hand go with a fixed dexterous grasp pose, We decouple the manipulation process into two distinct phases: initially, we generate a dexterous grasp pose targeting the functional part of the object; after that, we employ reinforcement learning to comprehensively explore the environment. Our findings suggest that the majority of learning time is expended in identifying the appropriate initial position and selecting the optimal manipulation viewpoint. Experimental results demonstrate significant improvements in learning efficiency and success rates across four distinct tasks. I INTRODUCTION Human interactions with the physical world are heavily reliant on hand movements. Observations of infant learning reveal that they initially form a preliminary manipulation position and viewpoint after brief observation. They then actively interact with an object’s functional parts, continuously collecting visual feedback, which accelerates their grasp of effective manipulation strategies. However, replicating this human-like manipulation in robots remains a significant challenge due to the high degrees of freedom in robotic hands. Recent studies [ 6 , 29 ] have made encouraging progress in addressing this issue, yet they typically initiate the manipulation process from a fixed, human-designed position tailored to specific tasks. This approach often leads to inefficiencies, as it wastes valuable learning time exploring an extensive environmental space to determine the optimal position and viewpoint, especially when the object is randomly placed within the workspace. [ 28 ] introduces an imitation learning method that replicates expert policies. However, this approach requires extensive data, typically collected through human teleoperation of the robotic hand, which is both time-consuming and costly. To address the challenges of data collection, [ 38 , 5 ] propose dexterous manipulation data collection systems. Building on this, [ 18 ] develops a rapid imitation learning approach capable of acquiring robust visual skills from less than a minute of human demonstrations. This method updates policies based on a weak initial policy that can only adapt to a limited range of robotic hand motions. Fig. 1: For the tasks of lifting the bucket and opening the laptop, we set the initial dexterous grasp pose to facilitate successful task completion. Inspired by the learning process observed in infants, we propose a dexterous manipulation method grounded in prior dexterous grasp pose knowledge, as illustrated in Fig. 1 . Our approach is structured into two phases. First, we segment the functional part of the object, which is then used to generate a set of two-finger grasp poses using Anygrasp [ 14 ] . These two-finger grasp poses are mapped into a dexterous grasp space, and a final dexterous grasp pose is selected through collision detection, serving as the initial grasp pose. This process mirrors how infants determine the optimal viewpoint and position for manipulating an object. In the second phase, we employ the Proximal Policy Optimization (PPO) algorithm [ 32 ] to iteratively interact with the object using partial-view point cloud inputs. The principal contributions of our study are as follows: • We introduce a two-stage framework that decouples dexterous manipulation into two phases: generating an initial dexterous grasp pose for the functional part of the object and training a policy to refine this dexterous grasp pose for effective object manipulation through reinforcement learning. • We decompose the dexterous manipulation reward in reinforcement learning into three distinct components: interaction reward, completion reward and restriction reward, integrating them into the learning process to facilitate task completion. • We conduct extensive experiments to validate that leveraging prior dexterous grasp pose knowledge significantly enhances the learning efficiency and success rate of reinforcement learning in simulation environments. Furthermore, we demonstrate the successful transfer of our algorithm to real-world applications, confirming its practical performance. II RELATED WORK II-A Dexterous manipulation Human dexterity is centered on the hand, and the advancement of human civilization is deeply intertwined with hands-on exploration of the surrounding environment. In this context, several studies [ 17 , 16 ] have employed reinforcement learning to teach robots how to manipulate objects within large spaces. However, these approaches often require training over millions of time steps to achieve optimal performance and typically rely on a fixed grasp pose. Other research [ 10 , 21 , 2 , 3 , 26 , 43 ] has focused on in-hand manipulation, where the object is reoriented to a default pose using visual or tactile information. While these methods demonstrate finger dexterity, they often limit the manipulation to objects held within the palm, restricting the range of potential applications. Several studies [ 33 , 31 , 13 , 35 , 9 , 1 ] have employed human video demonstrations to teach robots object manipulation. However, significant differences between robotic and human hands, along with the diversity of robotic hand designs, pose challenges for direct transfer of these techniques. In response, some works [ 20 , 44 , 28 , 38 ] have utilized teleoperation to collect data, generating expert-level datasets by having robots manipulate real objects. Additionally, [ 30 , 4 ] propose a virtual reality (VR) setup for collecting dexterous manipulation demonstrations, introducing the Demo Augmented Policy Gradient algorithm for imitation learning. Despite these advancements, such methods often face inefficiencies in data collection and exhibit discrepancies between the collected data and real-world applications. Moreover, imitation learning is inherently limited by the capabilities of the demonstrator, as it lacks an explicit understanding of task success. To address these shortcomings, some studies [ 30 , 19 , 11 ] have combined reinforcement learning with imitation learning to enhance learning and exploration efficiency. [ 40 ] further leverages human grasp affordances to teach dexterous manipulation. However, most existing methods either focus on learning a policy tailored to a single object or rely on access to object states provided by a perfect detector, which presents additional challenges for Sim2Real transfer. II-B Dexterous Grasping Grasping tasks are foundational to dexterous manipulation, as they provide crucial prior knowledge about the objects being manipulated by a robotic hand. Previous research [ 14 , 15 , 39 , 36 , 23 ] has predominantly focused on two-finger grasping tasks. However, with the increasing demand for precision manipulation, other studies [ 24 , 25 , 22 , 34 , 7 ] have utilized human grasp demonstrations or object affordances to teach robots how to grasp objects using dexterous hands. [ 42 , 37 ] employ reinforcement learning and distillation methods to develop dexterous grasping policies. However, there has yet to be an application that effectively leverages prior dexterous grasp pose knowledge for object manipulation. In this paper, we propose a dexterous manipulation method that utilizes prior dexterous grasp pose knowledge to determine the initial approach direction and grasp position, thereby enabling the robot to complete tasks more quickly and accurately. III METHOD In this section, we address the challenge

Read the complete text