HannesImitation: Grasping with the Hannes Prosthetic Hand via Imitation Learning

Carlo Alessi, Federico Vasile, Federico Ceola, Giulia Pasquale, Nicolò Boccardo, Lorenzo Natale · 2025

Read HannesImitation: Grasping with the Hannes Prosthetic Hand via Imitation Learning by Carlo Alessi, Federico Vasile, Federico Ceola, Giulia Pasquale, Nicolò Boccardo, Lorenzo Natale (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00491.txt.

Introduction

Carlo Alessi and co-authors teach a prosthetic hand to grasp by watching demonstrations in this 2025 paper. Modern prosthetic hands add cameras and sensors to share control with the user, easing the mental burden. Imitation learning, strong in robotics, had barely been tried for prostheses.

Their system, HannesImitation, targets the Hannes prosthetic hand. Instead of hand-coded rules or labeled drills, the hand learns grasping from shown examples. That route could restore finer dexterity and free users in less controlled settings. The paper bridges robot learning and assistive devices, two fields that rarely meet.

Catalog record

CREATORCarlo Alessi, Federico Vasile, Federico Ceola, Giulia Pasquale, Nicolò Boccardo, Lorenzo Natale
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-00491

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HannesImitation: Grasping with the Hannes Prosthetic Hand via Imitation Learning Carlo Alessi, Federico Vasile, Federico Ceola, Giulia Pasquale, Nicolò Boccardo, Lorenzo Natale Abstract Recent advancements in control of prosthetic hands have focused on increasing autonomy through the use of cameras and other sensory inputs. These systems aim to reduce the cognitive load on the user by automatically controlling certain degrees of freedom. In robotics, imitation learning has emerged as a promising approach for learning grasping and complex manipulation tasks while simplifying data collection. Its application to the control of prosthetic hands remains, however, largely unexplored. Bridging this gap could enhance dexterity restoration and enable prosthetic devices to operate in more unconstrained scenarios, where tasks are learned from demonstrations rather than relying on manually annotated sequences. To this end, we present HannesImitationPolicy , an imitation learning-based method to control the Hannes prosthetic hand, enabling object grasping in unstructured environments. Moreover, we introduce the HannesImitationDataset comprising grasping demonstrations in table, shelf, and human-to-prosthesis handover scenarios. We leverage such data to train a single diffusion policy and deploy it on the prosthetic hand to predict the wrist orientation and hand closure for grasping. Experimental evaluation demonstrates successful grasps across diverse objects and conditions. Finally, we show that the policy outperforms a segmentation-based visual servo controller in unstructured scenarios. Additional material is provided on our project page: https://hsp-iit.github.io/HannesImitation . IL imitation learning DP Diffusion Policy DoF degree of freedom I Introduction Learning-based robot control is emerging as a dominant paradigm for solving complex grasping and manipulation tasks across diverse robotic platforms [ 1 , 2 ] . In parallel, upper limb prostheses have evolved into sophisticated robotic devices with multiple DoF [ 3 ] . These advancements introduce new challenges and opportunities for learning-based methods in prosthetics. Fig. 1: We propose HannesImitation , an imitation learning-based approach that trains a single grasping policy across diverse objects and environments. The learned policy is deployed on the Hannes hand [ 4 ] , enabling control of wrist orientation and fingers closure from an eye-in-hand camera. Most commercial prostheses leverage electromyography (EMG) or mechanomyography (MMG) signals from residual muscle activity to decode intended movements [ 5 , 6 ] . Generally, two surface EMG electrodes are placed on antagonist muscles to decode intended movements. However, this approach limits intuitive control and dexterity, as users can typically control only one joint at a time. As the number of DoF increases, dexterous manipulation becomes more challenging, leading to higher cognitive load [ 7 ] . This limitation contributes to user dissatisfaction and, ultimately, device abandonment [ 8 ] . Other strategies include threshold-based, proportional control and pattern recognition methods [ 9 ] or incremental learning [ 10 ] . Conversely, machine learning models to predict hand motions were investigated to reduce the cognitive burden and enable precise myocontrol [ 11 ] . Despite the effectiveness of these techniques in laboratory settings, pattern recognition remains unstable due to the non-stationarity of EMG signals caused by muscle fatigue, electrode displacement and difference in arm posture [ 12 ] . Leveraging alternative input modalities, such as images, presents a viable solution to these challenges [ 13 ] . Specifically, additional input sources can either complement the user commands or be utilized by a semi-autonomous system to execute certain stages of the grasping action, aligning with the shared-autonomy framework [ 14 , 15 , 16 ] . However, these approaches typically learn from labeled data, that are difficult to acquire. Instead, we believe that learning a policy from demonstrations could unlock new possibilities. First, demonstrations inherently encapsulate all the necessary actions to accomplish a task, allowing semi-autonomous control of both wrist orientation—similar to prior work [ 17 , 18 , 19 ] —and hand closure, which could potentially reduce the cognitive burden on the user. Second, this approach facilitates deployment in tasks which require sophisticated labels. For instance, consider a human-to-prosthesis handover for a mug: if the human holds the mug by the handle, the prosthesis should grasp it from the body. Manually specifying this contextual information, e.g., with affordance segmentation masks or grasping poses, is impractical. Instead, learning from demonstration (also referred to as behavior cloning in the robotics literature) allows to autonomously learn from sequences of paired action-images demonstrating the task. This paper introduces a novel IL ( IL )-based control pipeline for prosthetic hands equipped with a camera embedded into the palm. By leveraging the generative capabilities of diffusion models, the proposed approach enables robust grasping across diverse objects and scenarios (Fig. 1 ). The paper is organized as follows. First, we review related work on IL for robotics and vision-based prosthetic control, highlighting the potential advantages in applying IL -based methods to control prosthetic devices (Sec. II ). Next, we introduce the HannesImitationDataset and describe how we adapt DP ( DP ) [ 20 ] to achieve high-frequency grasping with the Hannes prosthesis (Sec. III ). We release the dataset to foster further developments on IL for prosthetics. In Sec. IV , we present an extensive experimental validation, offering insights into learning-based prosthetic grasping from demonstrations. Finally, we discuss future research directions and summarize our key findings (Sec. V ). In summary, the major contributions of HannesImitation are: 1. HannesImitationPolicy . A novel DP-based approach for prosthetic grasping across a variety of objects and environments. The learned policy is deployed on the Hannes prosthesis to control wrist orientation and hand closure from visual input. 2. HannesImitationDataset . A collection of grasping demonstrations using the Hannes hand in both structured and unstructured environments. To the best of our knowledge, this is the first dataset for IL with prosthetic hands. II Related Work II-A Robot Manipulation with Behavior Cloning The latest vision-language-conditioned IL models have shown promising generalization capabilities over different manipulation tasks [ 21 , 22 , 23 , 24 ] . Conversely, diffusion-based approaches like DP [ 20 ] and its Flow Matching [ 25 ] variant achieve state-of-the-art performance for single-task learning. DP [ 20 ] is a widely adopted IL method for robotic manipulation tasks. Its policy formulation as a Denoising Diffusion Probabilistic Model (DDPM) [ 26 ] brings several advantages, including handling multi-modal actions, scalability to high-dimensional action spaces, and training stability, outperforming classical regression models for action prediction [ 27 ] . While behavior cloning is an efficient method to map robot observations to actions, data collection for policy learning could be time-consuming as it often requires teleoperation and specific hardware [ 22 ] . Some works overcome this limitation by simplifying data collection using hand-held grippers [ 28 , 29 ] and transfer policies trained on those data zero-shot on different robots. In this work, we adapt DP to prosthetic control, specifically to solve grasping and human-to-prosthesis handover tasks with the Hannes prosthesis, leveraging an eye-in-hand camera and proprioception with an approach similar to [ 29 ] . II-B Vision-based Prosthetic Control Early vision-based prosthetic systems employed image-processing techniques to estimate the object size and distance, automatically determining the most appropriate hand aperture and grasp type [ 30 , 31 ] . More recent approaches rely on advanced computer vision algorithms—such as object detection and segmentation [ 32 , 33 ] , grasp type

Read the complete text