Robust Immersive Bilateral Teleoperation of Beyond-Human-Scale Systems with Enhanced Transparency and Sense of Embodiment

Mahdi Hejrati, Pauli Mustalahti, Jouni Mattila · 2025

Read Robust Immersive Bilateral Teleoperation of Beyond-Human-Scale Systems with Enhanced Transparency and Sense of Embodiment by Mahdi Hejrati, Pauli Mustalahti, Jouni Mattila (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14486.txt.

Introduction

Mahdi Hejrati, Pauli Mustalahti, and Jouni Mattila build an immersive teleoperation system for machines far bigger than the human body, in a 2025 paper. Their platform pairs a full-scale industrial hydraulic arm with a seven-joint haptic exoskeleton and head-tracked virtual reality. The goal is remote work that feels direct, with motion and force faithfully mirrored at giant scale.

A force-sensorless adaptive controller handles the mismatch between operator and machine, along with delays up to a fixed fraction of a second. Full-scale trials showed precise tracking with motion scaled down hugely and force scaled down far more. A ten-person user study found embodiment scores about half again higher with virtual reality than with a plain monitor.

Catalog record

CREATORMahdi Hejrati, Pauli Mustalahti, Jouni Mattila
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE90 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-14486

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Robust Immersive Bilateral Teleoperation of Beyond-Human-Scale Systems with Enhanced Transparency and Sense of Embodiment Mahdi Hejrati, Pauli Mustalahti, Jouni Mattila Abstract In human-in-the-loop systems such as teleoperation, especially those involving heavy-duty manipulators, achieving high task performance requires both robust control and strong human engagement. This paper presents a bilateral teleoperation framework for beyond-human-scale robotic systems that enhances the transparency and the operator’s sense of embodiment (SoE), specifically, the senses of agency and self-location, through an immersive virtual reality interface and distributed haptic feedback. To support this embodiment and establish high level of motion and force transparency, we develop a force-sensorless, robust control architecture that tackles input nonlinearities, master-surrogate asymmetries, unknown uncertainties, and arbitrary time delays. A human-robot augmented dynamic model is integrated into the control loop to enhance human-adaptability of the controller. Theoretical analysis confirms semi-global uniform ultimate boundedness of the closed-loop system, guaranteeing the robustness to the real-world uncertainties. Extensive real-world experiments demonstrate high accuracy tracking under up to 1:13 motion scaling and 1:1000 force scaling, showcasing the significance of the results. Additionally, the stability-transparency tradeoff for motion tracking and force reflection and tracking is established up to 150 ms of one-way fix and time-varying communication delays. The results of user study with 10 participants (9 male and 1 female) demonstrate that the system can imply a good level of SoE (76.4%), at the same time is very user friendly with no gender limitation. These results are significant given the scale and weight of the heavy-duty manipulators. ©2025 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, including reprinting/republishing this material for advertising or promotional purposes, collecting new collected works for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. I Introduction Teleoperated robotic systems, first introduced by Goertz in the 1940s [ 1 ] , integrate human cognitive abilities with control-theoretic approaches to enable remote operation of robots in environments that are hazardous, unstructured, or otherwise unsuitable for direct human presence. In scenarios where use of fully autonomous systems remain impractical or unsafe, teleoperation offers a robust and flexible alternative, with proven effectiveness across a wide range of domains [ 2 , 3 , 4 ] . One field with significant untapped potential for teleoperation is heavy-duty robotic operation, particularly through the use of heavy-duty hydraulic manipulators (HHMs). These machines are already widely deployed in industries such as mining, construction, and forestry, where they handle physically intensive tasks in dynamic and often dangerous environments. By integrating human sensorimotor skills with the power and scale of HHMs, teleoperation equips HHMs with beyond human capabilities, while providing a reliable medium for intuitive human-to-HHM skill transfer [ 5 ] , addressing a key bottleneck in their automation [ 6 ] . Despite the advantages of teleoperation, task performance often deteriorates during remote operation due to factors such as master-surrogate mismatches, communication delays, and non-intuitive human-machine interfaces that can cause cyber sicknesses [ 7 ] . These challenges become even more pronounced when teleoperating HHMs, whose scale, dynamics, and mechanical complexity make them significantly less tractable than lightweight robotic systems. Such issues disrupt the operator’s natural sensorimotor loop and can impair proprioceptive perception, especially with visuo-haptic feedback, leading to a loss of intuitiveness in the teleoperation experience. One effective approach to mitigate these effects is enhancing the operator’s sense of presence [ 8 ] , defined as the subjective experience of being physically situated at the remote site, which, in the context of teleoperation, is more precisely referred to as the sense of telepresence (SoT) [ 9 , 10 ] . Nonetheless, enhancing telepresence alone is not sufficient to ensure high-quality task execution in teleoperation by a human operator. Slater (2009) [ 11 ] argued that Place Illusion—his proposed refinement of the term ”presence”—is not sufficient on its own for users to behave realistically in virtual environments. He introduced the concept of Plausibility Illusion, defined as the illusion that events occurring in the virtual environment are actually happening. According to Slater, both Place Illusion and Plausibility Illusion must co-occur for users to generate realistic responses within the virtual context. Building on this framework, Kilteni et al. (2012) [ 12 ] emphasized that realistic actions in virtual environments are closely linked to the sense of embodiment (SoE). Within the concept of teleoperation, SoE is defined as a psychological state in which the operator experiences the surrogate as an extension of their own body. According to [ 12 , 13 ] , SoE encompasses three critical components: (1) body ownership —feeling the surrogate as part of oneself; (2) self-location —a strong spatial telepresence at the remote site; and (3) agency —the confidence that one’s intentions directly produce corresponding actions in the remote environment. In many real-world applications—particularly industrial and field scenarios—the most relevant embodiment components are agency and self-location [ 13 ] , which directly influence the operator’s control quality and task performance. Thus, a strong SoE is crucial for successful teleoperation. When the SoE is strong, the operator no longer perceives the surrogate,the teleoperated robot, as an external tool, but rather as an extension of their own body and sensorimotor system [ 14 ] . This shift minimizes cognitive load and enhances the fluidity of task execution. Immersion is one key to evoking this SoE. Through the combination of visual feedback, delivered via a virtual reality (VR) headset, and distributed haptic feedback from force-reflective exoskeletons, immersive bilateral teleoperation creates more realistic task operation. But for immersion to truly support embodiment, it must be grounded in two essential pillars, each addressing relevant components of SoE: precise control, which reinforces the operator’s sense of agency, and appropriate equipment, which anchors their sense of self-location. This study addresses both dimensions in the context of industrial surrogates, focusing on HHMs. By developing a robust, high-accuracy control framework and pairing it with immersive, well-integrated hardware, we aim to strengthen both agency and self-location. This enables HHMs to combine human-like precision with mechanical strength, empowering operators to perform complex tasks efficiently and safely in hazardous or unstructured environments. II Related Works II-A Sense of Self-Location Immersive bilateral teleoperation relies on the integration of multi-modal sensory feedback, including visual, auditory, and haptic cues, to foster a stronger connection between the human operator and the remote environment. This immersive experience enhances the SoT, which in turn reinforces the sense of self-location (SoSL), as the two are closely interlinked. It is important to note, however, that not all sensory inputs contribute equally to immersion. Visual feedback, in particular, plays a foundational role in shaping the immersive experience during teleoperation [ 15 ] . Yet, immersion itself is a qualitative and often comparative concept [ 11 ] ; one system can be considered as more immersive than another with the given visual feedback, if, for instance, it provides real-time egocentric head-tracking compared to one with static camera. Even within the same system configuration, enhancements such as higher display resolution or the addition of haptic feedback can further elevate

Read the complete text