Cross-Embodied Co-Design for Dexterous Hands
Kehlani Fay, Darin Djapri, Anya Zorin, James Clinton, Ali El Lahib, Hao Su, Michael T. Tolley, Sha Yi, Xiaolong Wang · 2025
Read Cross-Embodied Co-Design for Dexterous Hands by Kehlani Fay, Darin Djapri, Anya Zorin, James Clinton, Ali El Lahib, Hao Su, Michael T. Tolley, Sha Yi, Xiaolong Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-03743.txt.
Introduction
A robot hand that can rotate an object between its fingers needs the right shape as much as the right software. Kehlani Fay and colleagues present House of Dextra in a 2025 robotics preprint, a framework that designs the hand and its controller together. The field lacks agreement on which designs best support dexterous tasks, so the authors let the task decide.
Their search space covers joints, fingers, and palms, generating many candidate morphologies. A shared control approach, conditioned on each design, makes it feasible to evaluate that wide space at scale. Winning designs are built from accessible parts for real-world testing, including in-hand rotation trials in simulation and on hardware. The full pipeline, from design to deployed hand, runs in under a day. The framework and the hands it produced are released openly for other groups to reuse.
Catalog record
| CREATOR | Kehlani Fay, Darin Djapri, Anya Zorin, James Clinton, Ali El Lahib, Hao Su, Michael T. Tolley, Sha Yi, Xiaolong Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-03743
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Cross-Embodied Co-Design for Dexterous Hands Kehlani Fay, Darin Djapri, Anya Zorin, James Clinton, Ali El Lahib, Hao Su, Michael T. Tolley, Sha Yi, Xiaolong Wang Abstract Dexterous manipulation is limited by both control and design, without consensus as to what makes manipulators best for performing dexterous tasks. This raises a fundamental challenge: how should we design and control robot manipulators that are optimized for dexterity? We present a co-design framework that learns task-specific hand morphology and complementary dexterous control policies. The framework supports 1) an expansive morphology search space including joint, finger, and palm generation, 2) scalable evaluation across the wide design space via morphology-conditioned cross-embodied control, and 3) real-world fabrication with accessible components. We evaluate the approach across multiple dexterous tasks, including in-hand rotation with simulation and real deployment. Our framework enables an end-to-end pipeline that can design, train, fabricate, and deploy a new robotic hand in under 24 hours. Results and build guide are available at: https://an-axolotl.github.io/co-design-for-dexterity.github.io/ Figure 1: We present our cross-embodied co-design framework for dexterous hands that jointly optimize control policies and hardware design. Four co-design hands are deployed sim-to-real. 1 Introduction Endowing robots with human-level dexterity critically requires both the development of fine control ( Wang et al. 2025 ; Qi et al. 2023 ; Zeng et al. 2019 ) and dexterous hardware design ( Zorin et al. 2025 ; Shaw et al. 2023 ; Unitree Robotics ; Allegro Hand ; Christoph et al. 2025 ) . Traditionally, robot manipulation has been developed by decoupling mechanical design from control, with reinforcement learning and control frameworks developed on fixed hardware architectures. This separation can limit dexterity - as optimal control policies may be fundamentally constrained by the hand’s morphology, degrees of freedom, and sensing capabilities ( Yin et al. 2025 ; Calandra et al. 2018 ) . Co-design aims to overcome this limitation by simultaneously adapting design and control ( Cheney et al. 2013 ; Pathak et al. 2019 ; Bai et al. 2025 ) . Co-design for manipulation especially suffers from a two fold curse of dimensionality, with a massive search space in both complex hardware design and fine dexterous control. As a result, jointly creating both requires efficiently iterating on hardware designs and quickly developing fine control. To achieve speed without loss of design complexity, prior works have often leveraged fast sampling based controllers ( Williams et al. 2016 ; Zhao et al. 2020a ; Xu et al. 2021c ) . However, there are challenges in using these controllers for manipulation due to sparse rewards, high contact solution space, and difficulty finding solutions farther away from initial conditions (Appendix Fig. 1) Rozzi et al. 2024 ; Cheng et al. 2021 . As a result, co-design methods for manipulation have a limited search space ( Mannam et al. 2023 ; Yi et al. 2025a ; Schaff et al. 2022 ) or avoid sparse, long horizon rewards altogether by addressing locomotion only ( Dinev et al. 2022 ; Yuan et al. 2021 ; Zhao et al. 2020a ) . We solve the scalability issue of complex design and fine control for dexterity by leveraging pre-training with a morphology-conditioned cross-embodied policy applied to generated robot hands. Additionally, previous works ( Yuan et al. 2021 ; Lu et al. 2025 ; Carlone & Pinciroli 2019 ; Wang et al. 2023a ; Cheney et al. 2013 ) have often been deployed only in simulation. The sim-to-real gap remains for many works due to over simplifications in hardware manufacturing constraints Aljalbout et al. 2025 , need for generalization of simulated control to real world dynamics Huang et al. 2023 , and modeling of physical properties Zhao et al. 2020b . To overcome this, we create a modular hardware platform with realistic components in simulation, enabling a feasible search with designs for sim-to-real. Cross-embodiment co-design framework: We enable quick and efficient evaluation of co-designed robot hands through incorporating a cross-embodiment control policy into our co-design framework, allowing design to sim-to-real in under 24 hours. Modular Robot Hand We create a robot hand platform with modular mechanical and electrical structure. This allows exploration of variable kinematics, degrees of freedom, and morphology. We show this by deploying four different robot hand embodiments sim-to-real. Accurate Robot Hand Generation We create modular and physically realistic grammars and rule structures that generate robot hands that can be built and deployed sim-to-real. Design Analysis: We expand co-design beyond primitive parameters for manipulation and identify design parameters of impact for future dexterity research. The result is a cross-embodied co-design framework with efficient evaluation and generation of robot hands. We demonstrate the effectiveness of this approach through sim-to-real with in-hand rotation across multiple generated designs. More result videos can be found at: https://an-axolotl.github.io/co-design-for-dexterity.github.io/ 2 Problem Setting and Notations We aim to jointly optimize robot embodiments and control, capturing their fundamental interdependence and enabling the discovery of designs where structure and control are naturally aligned. We denote the design space as \mathcal{G} and the control policy as \Pi . Specifically, our goal is to co-design both the robotic hand morphology and the policy (G\in\mathcal{G},\pi\in\Pi) such that the reward of a given manipulation task is maximized. Morphology Representation. We formulate a robot hand design as a graph G=(V,E,X_{v},X_{e}) . The nodes v\in V represent finger segment links. The edges e\in E represent kinematic joints between finger components. The maximum number of fingers used in this work is five, though the method could be expanded beyond that. We denote the number of fingers to be the width of the root node, i.e. f=\mathrm{width}(v_{root})=\{1,\cdots,5\} , and the length of each finger to be l=\mathrm{depth}(v_{root,i})=\{1,\cdots,4\} . Each finger may be of a different length, depending on the depth of its respective branch. Graph Neural Networks. To consider the isomorphism of different graphs, we encode each G with a message-passing GNN f_{\phi} . Given (V,E,X_{v},X_{e}) , f_{\phi} performs K rounds of message passing to produce a fixed-dimensional embedding y(G)=f_{\phi}(G)\in\mathbb{R}^{d}. This output y(G) is then used as a design condition for subsequent control policies and evaluations. Design-Conditioned Control Policy. We model control of our dexterous manipulation setting as a Markov Decision Process (MDP) \mathcal{M}_{G}=(\mathcal{S},\mathcal{A},T,R,\gamma), where the dynamics T_{G} depend on the chosen morphology G . The state space s\in\mathcal{S} is defined as s=\big(q,\,p_{o},\,y(G)\big), where q\in\mathbb{R}^{|E|} are the robot joint states, p_{o}\in\mathbb{R}^{6} is the object pose, and y(G)\in\mathbb{R}^{d} is the graph encoding of the design G , where d=F_{\max}\times(1+2L_{\max}) , corresponding to up to F_{\max}=5 fingers, each with at most L_{\max}=3 links. The action space \mathcal{A} consists of actuator commands of the joint positions. Since not all actuators are valid for all morphologies, we incorporate an action mask m(G)\in\{0,1\}^{|\mathcal{A}|} and apply it to the raw policy output: a_{t}=\pi_{\theta}(s_{t})\odot m(G), where \odot denotes element-wise multiplication. This ensures that the design-conditioned policy \pi_{\theta}:s_{t}\rightarrow a_{t}\in\Pi can be shared across designs without producing invalid actions. The transition function T(s_{t+1}\mid s_{t},a_{t}) represents the physics integration in simulation. The reward R(s_{t},a_{t}) is task-specific and includes object states, joint efforts, object proximity to the hand, and more, depending on the tasks. The objective of policy learning for a fixed G is to maximize expected discounted return: \displaystyle J(\pi,G)\;=\;\mathbb{E}_{\tau\sim\pi,\mathcal{M}_{G}}\Big[\sum_{t=0}^{K}\gamma^{t}R(s_{t},a_{t})\Big] (1) where \tau=(s_{0},a_{0},s_{1},\dots) denotes a trajectory induced by \pi in \mathcal{M}_{G} . Co-Design as Bi-Level Optimization.