Dexonomy: Synthesizing All Dexterous Grasp Types in a Grasp Taxonomy

Jiayi Chen 1,2∗ , Yubin Ke 1,2∗ , Lin Peng 2 , He Wang 1,2,3† · 2025

Read Dexonomy: Synthesizing All Dexterous Grasp Types in a Grasp Taxonomy by Jiayi Chen 1,2∗ , Yubin Ke 1,2∗ , Lin Peng 2 , He Wang 1,2,3† (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18829.txt.

Introduction

Jiayi Chen and three colleagues wrote a 2025 robotics paper on how robot hands grasp objects. A good robot hand must pick the right grip for each job, from a firm wrap to a fine pinch. Building data for all grip types by hand is far too slow.

Their system, called Dexonomy, makes grasp data on its own for any grip type, object, and jointed hand. It starts from one human-marked example per grip and hand. First it reshapes the object to fit the hand's pose, then it fine-tunes the hand in a physics test bed. The grasps it makes are rich in contact, with no overlap errors. The team also adds a check for grasp quality.

Catalog record

CREATORJiayi Chen 1,2∗ , Yubin Ke 1,2∗ , Lin Peng 2 , He Wang 1,2,3†
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-18829

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dexonomy: Synthesizing All Dexterous Grasp Types in a Grasp Taxonomy Jiayi Chen 1,2∗ , Yubin Ke 1,2∗ , Lin Peng 2 , He Wang 1,2,3† Abstract Generalizable dexterous grasping with suitable grasp types is a fundamental skill for intelligent robots. Developing such skills requires a large-scale and high-quality dataset that covers numerous grasp types (i.e., at least those categorized by the GRASP taxonomy), but collecting such data is extremely challenging. Existing automatic grasp synthesis methods are often limited to specific grasp types or object categories, hindering scalability. This work proposes an efficient pipeline capable of synthesizing contact-rich, penetration-free, and physically plausible grasps for any grasp type, object, and articulated hand. Starting from a single human-annotated template for each hand and grasp type, our pipeline tackles the complicated synthesis problem with two stages: optimize the object to fit the hand template first, and then locally refine the hand to fit the object in simulation. To validate the synthesized grasps, we introduce a contact-aware control strategy that allows the hand to apply the appropriate force at each contact point to the object. Those validated grasps can also be used as new grasp templates to facilitate future synthesis. Experiments show that our method significantly outperforms previous type-unaware grasp synthesis baselines in simulation. Using our algorithm, we construct a dataset containing 10.7k objects and 9.5M grasps, covering 31 grasp types in the GRASP taxonomy. Finally, we train a type-conditional generative model that successfully performs the desired grasp type from single-view object point clouds, achieving an 82.3 % 82.3\% success rate in real-world experiments. Project page: https://pku-epic.github.io/Dexonomy . Fig. 1 : For any grasp type in the GRASP taxonomy [ 20 ] , any object , and any articulated hand , our pipeline efficiently synthesizes contact-rich, penetration-free, and physically plausible dexterous grasps, starting from only one human-annotated grasp template to specify an initial hand pose and contact information per hand and grasp type. I Introduction Dexterous grasping is a fundamental skill for intelligent robots, enabling flexible interaction with the environment. However, this potential remains largely under-explored. Although dexterous grasping has received increasing attention, most prior work focuses on whether a dexterous hand can successfully grasp an object, rather than considering how to grasp it. As a result, the dexterous hand loses its dexterity and becomes functionally similar to a large parallel gripper. True dexterous grasping is not merely about “grasping with dexterous hands”, but about “grasping dexterously with appropriate grasp types based on the task requirement”. For example, when a robot needs to securely grasp an apple or hold a knife to cut, it should use a power grasp to envelop the object. Conversely, when grasping a lightweight or flat object from the table, a precision grasp using the fingertips would be more suitable. To develop such intelligent skills, there are two key challenges: (1) selecting the appropriate grasp type based on the task and (2) generating high-quality grasps for specified types and objects. The first challenge is a high-level decision-making problem and can take advantage of recent advances in large vision-language models, e.g., GPT-4o [ 22 ] , as a temporary solution. However, the second challenge is less studied and represents a significant bottleneck, which is the main focus of this paper. To address the problem of type-aware grasp synthesis with data-driven methods, the first step is to build a large-scale grasp dataset that at least includes most of the grasp types in the GRASP taxonomy [ 20 ] . However, collecting and annotating grasp data, particularly for multi-fingered hands in contact-rich scenarios, remains a big challenge. Several approaches have been explored for automatically synthesizing a large-scale dexterous grasp dataset, but most of them suffer from various limitations. Analytical grasp synthesis methods [ 47 , 44 , 25 , 8 , 7 ] are often applicable to any object, but most of them are type-unaware and the synthesized grasps only belong to limited types. This is because specifying flexible grasp types solely through analytical metrics is challenging. Moreover, these methods often produce unnatural hand poses, as they prioritize force closure , which does not always align with human habits. Another line of research [ 55 , 48 , 52 ] focuses on transferring functional dexterous grasps by mapping object contact regions. While these methods generate more human-like grasps and support a wider range of grasp types, they are limited to objects that are geometrically similar or axis-aligned with the initial demonstration, making them less scalable. In this work, we propose a novel pipeline based on sampling and optimization to address these challenges. As shown in Figure 1 , our algorithm can efficiently synthesize high-quality dexterous grasps for any grasp type, object, and articulated hand, requiring only one human-annotated grasp template per hand and grasp type. Our synthesized grasps achieve rich hand-object contact (e.g., >10 hand links within 2 mm of the object for power grasps), guarantee penetration-free poses via collision mesh verification, and satisfy force closure under six-axis testing in MuJoCo [ 42 ] — all with shared hyperparameters across grasp types, objects, and hands. Our key insight is that grasping can be framed as a geometric matching problem, where the hand and object should align through contact points. We begin by introducing a human-annotated grasp template that specifies the initial hand pose and contact information (i.e., points and normals). Unlike previous methods that directly adjust the hand pose to fit the object, we first use a lightweight stage that samples and optimizes the object pose to match the hand contacts defined in the grasp template. This stage supports hundreds of thousands of initial samples processed in parallel on a single GPU and leaves only a small number of promising results for the next stage. The combination of dense sampling and optimization helps avoid local optima without sacrificing efficiency. After aligning the object pose, the hand only needs a slight refinement to get a good grasp. This dual-stage design not only eases the hand refinement stage, but also ensures that the final grasp remains similar to the initial grasp template and thus remains natural. In contrast to most prior work [ 24 , 47 , 8 , 7 ] that develops custom objective functions and optimizers to refine the hand pose, we propose a novel method based on the transposed Jacobian control in MuJoCo. This approach is key to achieving rich contacts while ensuring no penetration, with minimal coding effort and parameter tuning. Next, we evaluate the synthesized grasps in MuJoCo to assess their ability to withstand external forces applied to the object. Unlike previous work [ 47 , 60 ] that designs heuristics to squeeze the hand for applying force on the object, which is not suitable for all grasp types, we design a contact-aware control strategy that computes the desired forces for each contact point and controls the hand to apply them approximately, also based on the transposed Jacobian control. Finally, high-quality grasps that pass the simulation tests can be used to construct new grasp templates, reducing the need for human annotations and broadening the range of objects that can be grasped. Experiments show that our method greatly outperforms previous type-unaware grasp synthesis baselines in simulation, especially under more challenging testing conditions

Read the complete text