Human-Machine Interfaces for Subsea Telerobotics: From Soda-straw to Natural Language Interactions

Adnan Abdullah, David Blow, Ruo Chen, Thanakon Uthai, Eric Jing Du, Md Jahidul Islam · 2024

Read Human-Machine Interfaces for Subsea Telerobotics: From Soda-straw to Natural Language Interactions by Adnan Abdullah, David Blow, Ruo Chen, Thanakon Uthai, Eric Jing Du, Md Jahidul Islam (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01753.txt.

Introduction

Adnan Abdullah and co-authors review how humans steer robots under the sea. Early consoles gave a soda-straw view through one narrow camera feed. Modern setups add gesture control, virtual reality, and plain language commands. The review reads the field through operator feel, robot autonomy, and two-way communication.

Weak spots persist in feedback realism and in controls that feel natural. No shared standard links rival platforms yet. Simulators and digital twins earn praise as cheap training and testing grounds. Shared autonomy, where human and robot split judgment, marks the frontier. A roadmap closes with open questions for smarter, kinder interfaces.

Catalog record

CREATORAdnan Abdullah, David Blow, Ruo Chen, Thanakon Uthai, Eric Jing Du, Md Jahidul Islam
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE86 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-01753

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Human-Machine Interfaces for Subsea Telerobotics: From Soda-straw to Natural Language Interactions Adnan Abdullah, David Blow, Ruo Chen, Thanakon Uthai, Eric Jing Du, Md Jahidul Islam Abstract This review explores the evolution of human-machine interfaces (HMIs) in subsea telerobotics, charting the progression from traditional first-person “soda-straw" consoles– characterized by narrow field-of-view camera feeds– to contemporary interfaces leveraging gesture recognition, virtual reality, and natural language processing. We systematically analyze the state-of-the-art literature through three interrelated perspectives: operator experience (including immersive feedback, cognitive workload, and ergonomic design), robotic autonomy (contextual understanding and task execution), and the quality of bidirectional communication between human and machine. Emphasis is placed on interface features to highlight persistent limitations in current systems, notably in immersive feedback fidelity, intuitive control mechanisms, and the lack of cross-platform standardization. Additionally, we assess the role of simulators and digital twins as scalable tools for operator training and system prototyping. The review extends beyond classical teleoperation paradigms to examine modern shared autonomy frameworks that facilitate seamless human-robot collaboration. By synthesizing insights from robotics, marine engineering, artificial intelligence, and human factors– this work provides a comprehensive overview of the current landscape and emerging trajectories in subsea HMI development. Finally, we identify key challenges and open research questions and outline a forward-looking roadmap for advancing intelligent and user-centric HMI technologies in subsea telerobotics. ]Email correspondence: [email protected] , [email protected] This pre-print is currently under review; project page: https://robopi.ece.ufl.edu/subsea_tr.html . keywords Human-machine interface; subsea telerobotics. † † firstpage: 1 † † address: RoboPI Laboratory † † address: ICIC Laboratory. University of Florida, Gainesville, \state FL, USA † † corresponding: [ 1 1. Introduction Subsea telerobotic technologies have advanced significantly over the past few decades, driven by the need for remote inspection, maintenance, and research expeditions in underwater environments [ 1 , 2 ] . Remotely operated vehicles (ROVs) and autonomous underwater vehicles (AUVs) are typically deployed to perform remote tasks in subsea environments that are beyond the reach of human scuba divers [ 3 ] . The robots are generally teleoperated from a surface vessel or a base station [ 4 ] for applications such as underwater infrastructure inspection [ 5 , 6 , 7 ] , seabed mapping [ 8 , 9 ] , remote surveillance [ 10 , 11 ] , environmental monitoring [ 12 ] , scientific expeditions, and more. Despite growing interest in autonomy, teleoperation continues to dominate subsea missions, comprising over 75% of deployments [ 13 ] . This trend highlights the immense potential of advancing telerobotic interfaces for economic growth and scientific discovery. Early teleoperated systems were limited in terms of the feedback they provided, often relying on basic visual data from low-resolution cameras. The operator had to rely heavily on experience and intuition to navigate and perform tasks, which increased the cognitive load and risk of errors. While effective for simpler tasks, studies indicate that the limited field-of-view (FOV), appearing as looking through a “ soda straw ” [ 14 ] , results in reduced situational awareness in more complex and dynamic environments, leading to increased cognitive load and reduced performance. The 2D visual feedback provides only a partial understanding of the remote workspace, requiring operators to frequently switch between different viewpoints if available [ 15 ] , and manually integrate information from various perspectives. This challenge is especially pronounced in confined or cluttered spaces, a common scenario in underwater pipelines, caves, and shipwreck exploration. In larger workspaces such as coral reefs, offshore rigs, and mining fields– traditional monocular camera-based interfaces fail to provide a comprehensive spatial view, leading to significant cognitive overload on the teleoperators [ 16 , 17 ] . Figure 1: An outline of the human-robot sensorimotor interaction flow for subsea telerobotics is shown. The interactions occur via control and feedback devices such as visual console, haptic gloves and suit, HMD, as well as by natural languages. A virtual simulation engine is often integrated with the interface to project the remote environment and ROV operation as a digital twin . Simulators with integrated twin models help teleoperation training by mission rehearsal/replay, which is essential for analyzing teleoperators’ cognitive load as well as for prototyping human-machine embodied/shared autonomy features. (FPV: first person view; NL: natural language; XR: augmented/virtual/mixed reality) These limitations have led to the development of more interactive systems, including 3D scene rendering with augmented/virtual/mixed reality (XR), haptic cues, and natural language (NL) controls for complex teleoperation tasks. Recent research highlights the importance of human-robot interface design in improving the operator’s awareness, cognitive load, and stress factors [ 18 , 19 ] – as well as improving the ROVs’ physical capabilities [ 20 ] . These developments have allowed operators to engage in human-machine shared autonomy, significantly enhancing the precision and safety of subsea missions [ 21 , 22 ] . Fig. 1 presents a general outline of human-machine sensorimotor interaction for subsea telerobotics. As shown, the interface elements bridge the human and machine entities with an interactive flow of teleoperation (teleop) control and feedback commands. It increases the sense of telepresence via intuitive feedback, integrates intelligent machine capabilities, and offers virtual engines to render a physics-simulated world and digital twin (DT) model of the real ROV [ 23 ] . More advanced interfaces offer task-sharing between the operator and the robot, thereby reducing cognitive load and enhancing overall mission efficiency [ 24 ] . Knowledge gaps and open questions . The interdisciplinary nature of modern HMIs with haptics, XR, and NL features has led to a broad but fragmented body of research literature. The existing systems and interfaces are built for a diverse set of platforms and applications without any consistent guidelines. There is no comprehensive technical review that compares the interfaces, compiles the findings, and identifies integration opportunities. To this end, we pose the following research questions, highlighting key gaps in HMI design, communication modalities, interaction strategies, and autonomy frameworks. 1. What design principles and standard guidelines are necessary to ensure modularity and cross-platform compatibility of HMIs in subsea telerobotics? 2. What command and feedback modalities ( e.g . , language, gesture, haptics, camera vision) should be prioritized or fused for different tasks according to complexity, environmental constraints, and mission objectives? 3. To what extent can digital replicas (simulators and twins) mimic a real-world scenario, thereby improving operator skills, reducing mission logistics, and supporting algorithm development for subsea operations? 4. How various aspects of shared autonomy can be integrated into modern interfaces and adapted according to the operator’s skill level and cognitive load? An in-depth understanding of these aspects is essential for driving innovation and advancing the HMI frontiers of subsea telerobotics. This paper aims to contribute toward that objective by providing a comprehensive technical review of the key HMI dimensions relevant to marine robotic systems. Specific contributions . In this paper, we present a structured analysis of subsea HMI systems based on their functional roles, interaction modalities, and levels of autonomy as observed in the literature. To facilitate comparative evaluation, we introduce the concept of “ technology clusters " that delineate how human-machine interaction happens across different system architectures. First, we categorize and evaluate the state-of-the-art (SOTA) subsea telerobotics interfaces and simulators based on a set of guidelines such as their choice of sensory

Read the complete text