Imitation Learning for Adaptive Control of a Virtual Soft Exoglove

Shirui Lyu, Vittorio Caggiano, Matteo Leonetti, Dario Farina, Letizia Gionfrida · 2025

Read Imitation Learning for Adaptive Control of a Virtual Soft Exoglove by Shirui Lyu, Vittorio Caggiano, Matteo Leonetti, Dario Farina, Letizia Gionfrida (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-09099.txt.

Introduction

Lyu and four co-authors teach a soft robotic glove to help weakened hands, in this two thousand twenty-five paper. Wearable hand robots can aid rehabilitation, but each patient's muscle loss is different, and standard controllers ignore that.

Working entirely in simulation, the team trained a manipulation model by imitation from videos of a person grasping objects, then fine-tuned it for specific tasks. They weakened muscles in the virtual hand to mimic neurological impairment and let a virtual exoglove supply the missing force. The learned controller restored about ninety point five percent of the original manipulation skill. The approach could one day tailor real therapy gloves to each patient's exact deficits.

Catalog record

CREATORShirui Lyu, Vittorio Caggiano, Matteo Leonetti, Dario Farina, Letizia Gionfrida
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-09099

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Imitation Learning for Adaptive Control of a Virtual Soft Exoglove Shirui Lyu, Vittorio Caggiano, Matteo Leonetti, Dario Farina, Letizia Gionfrida Abstract The use of wearable robots has been widely adopted in rehabilitation training for patients with hand motor impairments. However, the uniqueness of patients’ muscle loss is often overlooked. Leveraging reinforcement learning and a biologically accurate musculoskeletal model in simulation, we propose a customized wearable robotic controller that is able to address specific muscle deficits and to provide compensation for hand-object manipulation tasks. Video data of a same subject performing human grasping tasks is used to train a manipulation model through learning from demonstration. This manipulation model is subsequently fine-tuned to perform object-specific interaction tasks. The muscle forces in the musculoskeletal manipulation model are then weakened to simulate neurological motor impairments, which are later compensated by the actuation of a virtual wearable robotics glove. Results shows that integrating the virtual wearable robotic glove provides shared assistance to support the hand manipulator with weakened muscle forces. The learned exoglove controller achieved an average of 90.5% of the original manipulation proficiency. I INTRODUCTION The use of wearable robots has shown promising potential both in neurological rehabilitation protocols [ 1 ] and in passive supports to boost neural motor performance [ 2 ] . Patients with upper limb neurological dysfunctions or impairments often experience varying degrees of dexterity loss [ 3 ] . A personalized rehabilitation strategy is essential to enhance their independence, thereby improving their quality of life [ 4 ] . However, designing a tailored rehabilitative intervention that accounts for rapidly changing, patient-specific clinical indicators [ 5 ] , particularly through wearable robotic devices, remains a significant challenge. One of the challenges in realizing an intelligent controller for assistive wearable robots is the translation of the unique medical conditions of individual patient into adaptable controllers. For example, patients recovering from ischemic or hemorrhagic strokes, often experience significant limitations in their range of motion, characterized by symptoms such as reduced muscle strength or limitation of range of motion [ 6 ] . In clinical settings, such diverse medical conditions are typically managed through ad-hoc manual rehabilitation programs administered under the supervision of trained professionals [ 4 ] . However, in the context of wearable robotic rehabilitation, the device controller needs to be timely adjusted to provide clinical support [ 7 , 8 ] . The development of automated methods for adapting wearable device control to individual patient needs is crucial for enabling personalized and accessible rehabilitation interventions. Data-driven control approaches for wearable robotics have become widely adopted, enabling more general and accessible solutions for wearable robotic control [ 9 , 10 , 11 , 12 , 13 , 14 ] . These approaches learn biological features of the user such as kinematics, and muscle dynamics, from sufficient data to control the assistive devices. However, they focus on general rather than specific user behaviours, thus leaving a gap in addressing practical individual-based rehabilitation needs. The development of personalized control strategies for wearable robots has been employing musculoskeletal models to facilitate adaptable controller training through Reinforcement Learning (RL) [ 15 , 16 ] . Despite their utility, these approaches often yield non-intuitive behaviours [ 17 ] attributable to the absence of reliable human-reference data. To address these limitations, the integration of vision and imitation learning within RL training frameworks offers a promising solution. These methods enhance the fidelity of natural human behaviour modelling by providing enhanced biomechanical insights. In this work, we propose a framework that uses imitation learning to derive user-specific reference trajectories from a healthy subject’s video dataset [ 18 ] . These trajectories are used to reconstruct kinematics and muscle dynamics in silico, where hand impairments are simulated. Finally, a tendon-driven exoskeleton glove (exoglove) control strategy that adapts to the simulated impairments is tested using the healthy reference as a basis for optimization reference. The proposed approach consists of three stages (Fig. 1 ). First, reference trajectories are extracted from video data and learned by imitating the behaviour of a healthy subject through MyoHand [ 19 ] , a biologically accurate musculoskeletal hand model, to replicate the user’s natural behaviour. Second, RL-driven manipulation is applied to simulate hand-object manipulation tasks within the musculoskeletal model, and then a simulated impairment is incorporated to reflect reduced functionality in the hand. Finally, a virtual exoglove, modeled after a real device [ 20 ] , is implemented in simulation, where an adaptive control strategy is developed to compensate for the impaired hand, optimizing hand-object manipulation and improving functionality for the impaired hand. Fig. 1: Pipeline overview illustrating the three main stages: (1) extraction of user behavior via imitation learning from healthy subject video data, (2) training a biologically accurate musculoskeletal hand model for object-centric manipulation and simulating muscle impairments, and (3) development and testing of a tendon-driven exo-glove controller to compensate for impaired hand functionality. II Related Work II-A Learning manipulations from human videos and RL As human video inherently captures the intricate kinematics of hand-object interactions, including the complexity and precision required for dexterous manipulation, extensive research has focused on understanding and modeling these interactions [ 21 , 22 , 18 , 23 ] . Those recorded interactions provide sufficient context for the understanding of dexterous manipulation and facilitate other robotics applications such as handling previously unseen manipulation tasks [ 24 ] . Nevertheless, the learned model, referring to computational models trained on human video data to predict or replicate manipulation strategies, depends heavily on human video data and is less robust in handling unseen objects. RL-based manipulation, on the other hand, is often data-inefficient and explores solutions with computational resources over data. Such an approach gives rise to manipulation understanding that is less relevant to human demonstrations. Object-centric approaches [ 25 , 26 ] , for example, mainly match the object in hand with a reference trajectory, and learn a manipulation strategy even with a high Degree of Freedom (DoF) musculoskeletal hand model. Recent work [ 27 , 28 , 29 ] has attempted to combine the advantage from both sides: by using human demonstration video to facilitate RL training, and by using RL to train robust grasping behaviours. In addition, Wan et al. [ 30 ] leveraged RL and computer vision to first generate multiple grasping gestures before applying conditioned policy training to achieve generalized grasping behaviour. Chen et al. [ 17 ] proposed to use of human demonstration video to guide the RL training while leaving space for RL exploration to handle robot-object interactions. However, most previous works generalize the grasping behaviour and few of the studies were conducted on model-based RL for rehabilitative robotics applications. II-B Model-based wearable robotics control Recent studies [ 14 , 10 , 31 , 12 ] , on wearable robotics systems, have focused on integrating high-level task control with low-level hardware control to enhance system performance. This combined approach allows wearable robots to execute tasks more effectively while adapting to the mechanical and dynamic constraints of the hardware. Introducing high-level perceptual information such as egocentric views [ 32 , 33 ] , or user-intent [ 15 ] , into the control architecture usually improves the system’s responsiveness and adaptability. However, deriving accurate and reliable perceptual

Read the complete text