Neural Control and Learning of Simulated Hand Movements With an EMG-Based Closed-Loop Interface
Balint K. Hodossy, Dario Farina · 2026
Read Neural Control and Learning of Simulated Hand Movements With an EMG-Based Closed-Loop Interface by Balint K. Hodossy, Dario Farina (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-07364.txt.
Introduction
Hodossy and Farina build a virtual human that learns hand movements through a muscle-driven simulation plus a neural interface. Simply replaying recorded signals cannot capture the flexibility and causal structure of real control, so their model couples forward musculoskeletal simulation with reinforcement learning and online muscle-signal synthesis. The virtual user adapts its policy through feedback and feedforward control. A gesturing task shows the approach can test neural controllers and grow training data.
Catalog record
| CREATOR | Balint K. Hodossy, Dario Farina |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52981 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.07364
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Neural Control and Learning of Simulated Hand Movements With an EMG-Based Closed-Loop Interface Balint K. Hodossy, Dario Farina Abstract The standard engineering approach when facing uncertainty is modelling. Mixing data from a well-calibrated model with real recordings has led to breakthroughs in many applications of AI, from computer vision to autonomous driving. This type of model-based data augmentation is now beginning to show promising results in biosignal processing as well. However, while these simulated data are necessary, they are not sufficient for virtual neurophysiological experiments. Simply generating neural signals that reproduce a predetermined motor behaviour does not capture the flexibility, variability, and causal structure required to probe neural mechanisms during control tasks. In this study, we present an in silico neuromechanical model that combines a fully forward musculoskeletal simulation, reinforcement learning, and sequential, online electromyography synthesis. This framework provides not only synchronised kinematics, dynamics, and corresponding neural activity, but also explicitly models feedback and feedforward control in a virtual participant. In this way, online control problems can be represented, as the simulated human adapts its behaviour via a learned RL policy in response to a neural interface. For example, the virtual user can learn hand movements robust to perturbations or the control of a virtual gesture decoder. We illustrate the approach using a gesturing task within a biomechanical hand model, and lay the groundwork for using this technique to evaluate neural controllers, augment training datasets, and generate synthetic data for neurological conditions. 1 Introduction The planning and execution of motion form a fundamental part of human intelligence. We explore and influence our environment, communicate with each other and express ourselves with movement. Through neural interfacing, we aim to integrate artificial systems into this process, coordinating and synchronising the behaviour of assistive devices with our own neuromechanics. Yet, each of us has a unique body and an evolving “language” of biosignals, which may be further altered by neurological conditions. How can a neural interface learn to interpret these individual patterns and their adaptations when they are disrupted? One approach is to collect large-scale datasets, built on carefully designed protocols, appropriate instrumentation, and broad, inclusive recruitment, so that AI models can learn to accommodate this inter-individual diversity [ 29 ] . This is a time-consuming process that requires a significant amount of planning and resources. It is further hindered by the need to use Human Machine Interface (HMI) algorithms and hardware actively under development, particularly when involving closed-loop online control scenarios. This leads to a slow design iteration loop, which must be repeated whenever a new signal modality, control task or user population is considered. For this reason, promising concepts and techniques may fail when deployed in real conditions, if they reach that stage. Because the development of next generation of neural interfaces requires numerous design iterations and extensive user involvement, simulation has become an increasingly important tool. Simulation of wearable assistive devices, for example, has been validated by transferring to real-world environments, users and devices [ 25 , 32 ] . This approach could similarly be used to catalyse development of prototype HMI hardware and control algorithms. When using simulated neural signals to train and develop HMIs , we face an apparent contradiction. On the one hand, a real participant cannot “wear” a virtual HMI device, particularly when it is meant to provide mechanical assistance such as an exoskeleton. On the other hand, if we introduce a virtual human model to restore the necessary mechanical interactions, a simple one-way, open-loop generation of biosignals is no longer sufficient: the virtual user must be able to adapt its behaviour to the ongoing dynamics of the simulated body and its interaction with the HMI . To bridge this gap, we present a framework that embeds biosignal synthesis within a closed-loop, online simulation of a HMI and its user. As a proof of concept, we model an electromyography (EMG) -driven gesture decoding task using a musculoskeletal hand model. The central novelty is that the motor control policy of the virtual user is not fixed, but it continuously reacts to the gesture decoder’s output, adapting the simulated behaviour and thereby altering the generated sEMG signals in order to optimise the HMI output. These updated signals are then fed back into the decoder, forming a bidirectional, adaptive closed-loop between the virtual human and the HMI . This is in contrast to all current simulation approaches that are based on one-way, open-loop signal generators. In addition to virtual prototyping, our methods are relevant for applications such as data augmentation for data-driven HMIs . For example, integrating models of neurological conditions could generate pathological signals, reducing the participation burden on the patient population in preliminary studies. 1.1 Background 1.1.1 Electromyography Surface electromyography ( EMG ) provides an indirect measure of motor commands and is widely used both as a control modality and as a tool to monitor neurological status [ 38 , 5 , 56 ] . However, EMG signals are highly sensitive to the neurophysiological state and recording conditions (e.g., electrode placement, tissue composition, perspiration), leading to substantial inter- and intra-subject variability. This variability hampers the generalisation of pattern recognition and AI-based decoding methods to new users and sessions [ 2 , 62 ] . Recent work has addressed robustness via domain adaptation, meta-learning, and large multi-participant datasets [ 65 , 29 ] . A complementary strategy is to augment real recordings with synthetic EMG . When synthesis spans sufficiently diverse physiological and recording conditions, it can improve controller robustness and even enable transfer from simulation to real-world applications [ 1 ] . Advances in modelling motor unit action potentials and motor unit discharge patterns now permit efficient generation of plausible EMG for prescribed motions [ 39 , 36 , 34 , 33 , 50 ] . However, these inverse approaches rely on predetermined behaviour and therefore cannot support predictive, closed-loop virtual experiments with HMIs . This motivates the development of forward neuromechanical models that couple neural drive to muscles, muscle dynamics, and physics-based interaction with the environment. 1.1.2 Musculoskeletal Simulation Modelling motor tasks with the human musculoskeletal (MSk) system is commonly done by representing body segments as rigid bodies and their joints as a system of articulations. Their configuration is best described in either general or anatomical coordinates, depending on the application [ 20 , 61 , 26 ] . Physics engines provide a validated way to construct and integrate the mechanics of this simulation state. The open-source gold standard physics engine for biomechanics simulation is Simbody [ 49 ] , usually accessed through the OpenSim development environment. However, it faces severe limitations when scaling up to data-heavy requirements, such as those of reinforcement learning (RL) [ 9 ] . MuJoCo is an alternative open-source choice of physics engine, optimised for efficient and scalable execution and versatile and robust contact models relevant for manipulation tasks [ 55 ] . Thanks to these properties, it has been increasingly adopted for biomechanical modelling research with heavy computational costs [ 21 , 51 , 58 ] , although there are many examples that use options such as Simbody [ 13 ] or DART [ 32 , 41 , 30 ] for similar applications. Of particular interest in MuJoCo is its