BODex: Scalable and Efficient Robotic Dexterous Grasp Synthesis Using Bilevel Optimization
Jiayi Chen, Yubin Ke, He Wang · 2024
Read BODex: Scalable and Efficient Robotic Dexterous Grasp Synthesis Using Bilevel Optimization by Jiayi Chen, Yubin Ke, He Wang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-16490.txt.
Introduction
Jiayi Chen, Yubin Ke, and He Wang build a fast grasp maker for robot hands in a 2024 computing paper. Dexterous hands with many fingers need huge sets of sample grasps to train on. Older methods made those samples slowly and tested them on few objects. Their system, called BODex, frames grasp building as a two-level math problem and runs it on graphics chips.
The lower level solves hand posture as a tidy quadratic problem, while the upper level adjusts by gradient steps. Thousands of grasps are synthesized side by side, reaching over forty-nine finished grasps per second on one card. A shared bench test in a physics simulator lets rivals be compared fairly. Fast sample factories can unstick data-hungry robot learning.
Catalog record
| CREATOR | Jiayi Chen, Yubin Ke, He Wang |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-16490
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
BODex: Scalable and Efficient Robotic Dexterous Grasp Synthesis Using Bilevel Optimization Jiayi Chen, Yubin Ke, He Wang Abstract Robotic dexterous grasping is important for interacting with the environment. To unleash the potential of data-driven models for dexterous grasping, a large-scale, high-quality dataset is essential. While gradient-based optimization offers a promising way for constructing such datasets, previous works suffer from limitations, such as inefficiency, strong assumptions in the grasp quality energy, or limited object sets for experiments. Moreover, the lack of a standard benchmark for comparing different methods and datasets hinders progress in this field. To address these challenges, we develop a highly efficient synthesis system and a comprehensive benchmark with MuJoCo for dexterous grasping. We formulate grasp synthesis as a bilevel optimization problem, combining a novel lower-level quadratic programming (QP) with an upper-level gradient descent process. By leveraging recent advances in CUDA-accelerated robotic libraries and GPU-based QP solvers, our system can parallelize thousands of grasps and synthesize over 49 49 grasps per second on a single 3090 GPU. Our synthesized grasps for Shadow, Allegro, and Leap hands all achieve a success rate above 75 % 75\% in simulation, with a penetration depth under 1 1 mm, outperforming existing baselines on nearly all metrics. Compared to the previous large-scale dataset, DexGraspNet, our dataset significantly improves the performance of learning models, with a success rate from around 40 % 40\% to 80 % 80\% in simulation. Real-world testing of the trained model on the Shadow Hand achieves an 81 % 81\% success rate across 20 diverse objects. The codes and datasets are released on our project page: https://pku-epic.github.io/BODex . I Introduction Robotic dexterous grasping is foundational to interacting with the environment and thus an important research topic. While large-scale data collection and learning-based methods have achieved significant success in parallel gripper grasping [ 1 , 2 ] , their potential for dexterous hands remains largely unexplored, partly due to the increased difficulty of data collection. Unlike parallel grippers, dexterous hands often have over 20 degrees of freedom (DoF), thus greatly reducing the effectiveness of directly sampling grasp poses. Although gradient-based optimization has been explored recently as a promising approach to scaling up the grasp data for dexterous hands, previous methods face several limitations. Some [ 3 , 4 ] rely on strong assumptions in the grasp quality energy, such as equal contact forces and no friction, while others [ 5 , 6 ] only study on a limited set of objects. Moreover, the synthesis speed of previous works is quite slow. In addition, differences in robot hands, simulators, and evaluation metrics across studies make comparison difficult. To address these challenges, we develop an efficient grasp synthesis system and a comprehensive benchmark. We formulate grasp synthesis as a bilevel optimization problem: the lower-level quadratic programming (QP) determines the optimal force combination for each contact at the current hand pose to achieve a desired wrench, without any assumption, while the upper-level process performs gradient descent on the hand pose to minimize the difference between the desired wrench and the best-applied wrench, as determined by the lower-level QP. Our system can also synthesize pre-grasp poses that maintain a certain distance from the object, aiding in planning collision-free hand-arm trajectories and controlling the hand to apply force on the object. Fig. 1 : Comparison with analytic-based dexterous grasp synthesis baselines on Allegro Hand. Our pipeline significantly outperforms baselines on almost all metrics, especially on the most important two, simulation success rate and speed. To accelerate the system and enable large-scale parallelization, we leverage recent advances in the CUDA-accelerated robotics library, cuRobo [ 7 ] , and the GPU-based QP solver, ReLU-QP [ 8 ] . To integrate these tools into our system, we first propose a coarse-to-fine strategy to address imprecise contact issues caused by sphere approximation in cuRobo. We also implement a batched version of ReLU-QP to solve multiple QPs in parallel on a GPU, achieving a 10x speedup compared to CPU solvers like ProxQP [ 9 ] and OSQP [ 10 ] . Dataset Hand Sim./Real Objects Grasps Table Pre-grasps Collision-free trajectory Method DDGdata [ 11 ] Shadow Sim. 565 6.9k ✗ ✗ ✗ GraspIt! DexGraspNet [ 4 ] Shadow Sim. 5355 1.32M ✗ ✗ ✗ Optimization GenDexGrasp [ 12 ] Multiple Sim. 58 436k ✗ ✗ ✗ Optimization RealDex [ 13 ] Shadow Real 52 59k ✓ ✗ 2630 Teleoperation BODex-Floating Shadow, Allegro, Leap Sim. 2397 4.09M ✗ ✓ ✗ Optimization BODex-Tabletop UR10e+Shadow Sim. 2397 3.08M ✓ ✓ 3.08M Optimization TABLE I : Robotic dexterous grasping dataset statistic. All of our grasps and trajectories have been validated in MuJoCo. Finally, we establish a benchmark with MuJoCo [ 14 ] to compare various analytic-based grasp synthesis pipelines, grasp energy functions, and learning-based methods. As shown in Fig. 1 , our system significantly outperforms baselines across nearly all metrics, achieving a 50x speedup in synthesizing higher-quality grasps. Compared to previous grasp energies, our QP-based energy does not rely on assumptions about contact forces and friction, resulting in a higher success rate and better correlation with simulation outcomes (i.e. a grasp with lower energy is more likely to succeed in simulation). The learning-based method trained on our dataset also greatly outperforms the same model trained on the previous large-scale dataset, DexGraspNet, improving success rates from around 40\% to 80\% . Real-world testing of the trained model on the Shadow Hand achieves an 81\% success rate across 20 diverse objects. In summary, our contributions are: (1) a GPU-based efficient grasp synthesis system using bilevel optimization; (2) a large-scale, high-quality dexterous grasp dataset that enables a better learning model; and (3) a reproducible benchmark for grasp synthesis with the MuJoCo simulator. II Related Work Analytic-based dexterous grasp synthesis methods are often used for constructing datasets to train learning-based models, as analytic-based methods typically rely on complete object geometry, which is difficult to obtain in real-time but available in offline 3D assets. These methods commonly focus on synthesizing force-closure grasps that can resist any external wrench applied to the object. The most popular metric for evaluating force closure is the Q1 metric [ 15 ] , which measures the radius of the largest inscribed ball within the Grasp Wrench Space (GWS) — the set of all wrenches that the hand can apply to the object. Early approaches, e.g. GraspIt! [ 16 ] , use sampling-based methods to find grasps with a high Q1 metric but are inefficient for high-DoF hands. More recent work has explored gradient-based optimization for grasp synthesis. DFC [ 3 , 12 ] introduces a differentiable force closure energy that aims to include the origin within the GWS, under the assumptions of no friction and equal contact forces. DexGraspNet [ 4 ] accelerates the pipeline of DFC and generates a large-scale grasp dataset for over 5000 objects, but its speed and data quality still need improvement. Grasp’D [ 17 ] and Fast-Grasp’D [ 18 ] explore the use of differentiable simulators for grasp synthesis. FRoGGeR [ 5 ] and SpringGrasp [ 6 ] propose novel energies for optimization but study only a limited set of objects. TaskDexGrasp [ 19 ] extends the formulation to both force-closure and non-force-closure grasps. The grasp energy proposed in [ 20