SafeSpect: Safety-First Augmented Reality Heads-up Display for Drone Inspections Conference: CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26-May 1, 2025, Yokohama, Japan DOI: 10.1145/

Peisen Xu, Jérémie Garcia, Wei Tsang Ooi, Christophe Jouffrais · 2025

Read SafeSpect: Safety-First Augmented Reality Heads-up Display for Drone Inspections Conference: CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26-May 1, 2025, Yokohama, Japan DOI: 10.1145/ by Peisen Xu, Jérémie Garcia, Wei Tsang Ooi, Christophe Jouffrais (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16533.txt.

Introduction

Peisen Xu and co-authors design in 2025 a heads-up display that helps drone pilots inspect buildings safely. Tablet controls force pilots to split their gaze between screen and sky, which dulls awareness and loads the mind. Their augmented reality design lays key data over the pilot's real view.

Workshops with professional pilots shaped the features. The display swaps between a task view and a safety view to avoid overload. In a study with fifteen users on a mock inspection, the plain AR version eased access to safety data. The adaptive version also cut mental load and raised awareness, with no loss in task scores.

Catalog record

CREATORPeisen Xu, Jérémie Garcia, Wei Tsang Ooi, Christophe Jouffrais
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-16533

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SafeSpect: Safety-First Augmented Reality Heads-up Display for Drone Inspections Conference: CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26-May 1, 2025, Yokohama, Japan DOI: 10.1145/3706598.3714283 ISBN: 979-8-4007-1394-1/25/04 CCS: Human-centered computing Mixed / augmented reality CCS: Human-centered computing Participatory design Peisen Xu, Jérémie Garcia, Wei Tsang Ooi, Christophe Jouffrais Abstract . Current tablet-based interfaces for drone operations often impose a heavy cognitive load on pilots and reduce situational awareness by dividing attention between the video feed and the real world. To address these challenges, we designed a heads-up augmented reality (AR) interface that overlays in-situ information to support drone pilots in safety-critical tasks. Through participatory design workshops with professional pilots, we identified key features and developed an adaptive AR interface that dynamically switches between task and safety views to prevent information overload. We evaluated our prototype by creating a realistic building inspection task and comparing three interfaces: a 2D tablet, a static AR, and our adaptive AR design. A user study with 15 participants showed that the AR interface improved access to safety information, while the adaptive AR interface reduced cognitive load and enhanced situational awareness without compromising task performance. We offer design insights for developing safety-first heads-up AR interfaces. Figure 1. An adaptive AR drone control interface for building inspection task balances mission and safety information to reduce cognitive load. (A) shows a simplified interface for data collection. (B) helps track the drone’s position and mission progress. (C) illustrates a safety view for attention switching between camera view and the line-of-sight. (D) displays the critical data such as heading, collision status and positional uncertainty near the drone for enhanced situational awareness. This is the teaser figure that illustrates an adaptive AR drone control interface for building inspection task. The pilot is wearing an Optical Head-Mounted Display (OHMDs) for drone control and data collection. On the left, a mission view is illustrated which simplifies the interface to help the pilot focus on collecting data and monitoring the progress. Details of the AR waypoints and path visualization are shown. On the right, a safety view is displayed with flight data visualized in 2D and AR to make the pilot aware of the safety issues. This view helps the pilot switch attention between a 2D camera view and the direct line-of-sight for controlling the drone. The situational-awareness-enhancing AR visualizations, such as the drone heading, collision status, and positional uncertainty are shown. † † cc-license: by 1. Introduction Drone inspections have become commonplace for maintaining hard-to-reach areas in urban environments. In addition to structure condition monitoring, drones are used for data collection for digital twins and tracking construction progress ( Anwar et al. 2018 ; Halder and Afsari 2023 ) . However, the current reliance on 2D handheld devices for flight control and data monitoring presents significant challenges for pilots ( Ljungblad et al. 2021 ) . Pilots often experience "tunnel vision", focusing on the screen rather than the drone’s physical surroundings, which increases the risk of collisions, especially in safety-critical scenarios ( Borowik et al. 2022 ) . Head-up Displays (HUDs) and augmented reality (AR) interfaces provide an opportunity to integrate drone data directly into the pilot’s visual line of sight (VLOS), potentially reducing cognitive load ( Ram and Zhao 2021 ; Buchner et al. 2022 ) and improving situational awareness ( Kim and Dey 2009 ) . A key challenge lies in designing an AR interface that effectively displays multiple critical elements such as remaining battery, GPS status, speed or orientation without overwhelming the pilot who is already facing cognitive burden ( Ljungblad et al. 2021 ) . Previous research has often focused on individual elements of the interface, such as camera feed ( Hedayati et al. 2018 ) , waypoints (for semi-autonomous flight) ( Walker et al. 2019 ) , altitude ( Zollmann et al. 2014 ) or heading ( Szafir et al. 2015 ) . Rebensky et al. ( Rebensky et al. 2022 ) suggest that directly adapting conventional 2D interfaces to heads-up displays might overwhelm new pilots, recommending AR HUDs to prioritize information based on task urgency and context. Another challenge, as with every aeronautical activity, is to ensure safety. According to Rahmani et al. ( Rahmani and Weckman 2023 ) , in addition to GPS signal loss or battery failures, situational awareness, decision-based, and skill-based errors are the main issues leading to drone accidents. While previous research has improved the operational efficiency of drone flight with AR, such as camera alignment ( Chen et al. 2021 ; Hedayati et al. 2018 ) , less attention has been given to critical safety scenarios, which are difficult to simulate in real-world experiments. Our goal is to address these two gaps: (1) the need to manage information processing during an inspection task with AR interfaces, without overwhelming the pilot, and (2) the lack of research on AR interfaces in safety-critical situations. We investigate the following research questions: • RQ1 . How can 3D heads-up AR interfaces reduce the cognitive load of drone pilots performing facade inspection? • RQ2 . What design factors on the heads-up AR interface can ensure safety and situational awareness during critical scenarios of drone flight? To answer these questions, we adopted a research-through-design approach ( Zimmerman et al. 2007 ) , aiming at transforming drone interfaces "from their current state to a preferred state" that enhances both safety and efficiency during drone inspections. Through this process, we sought to create an AR prototype that could improve drone piloting under challenging conditions while minimizing cognitive overload. In this paper, we first describe the participatory design workshops with five professional pilots. These workshops helped identify the potential benefits of using AR for monitoring mission progress and ensuring safety during facade inspection, as well as the need to balance the information presented to avoid overwhelming users. We then describe the iterative design process with three pilots of a VR-based drone inspection simulator. During this process, we explored how dynamic adaptation between safety and mission in AR could reduce cognitive load and enhance situational awareness. The Unity simulator allowed us to simulate the demanding inspection task and challenging scenarios such as GPS signal loss and obstacles without the risks associated with real-world testing. We then describe a user study with 15 experienced drone users comparing the adaptive AR interface ( adapt-ar ) with the conventional 2D interface ( 2d-only ) and a static, non-adaptive AR interface ( full-ar ) during a facade inspection task. The study was followed by semi-structured interviews. We conclude with design insights for developing safety-first heads-up AR interfaces. 2. Related Work 2.1. Risks and Challenges of Urban Drone Flight Commercial drone operations, especially those conducted in densely populated urban areas, require rigorous safety assessment procedures. Through interviews with professional pilots, Ljungblad et al. ( Ljungblad et al. 2021 ) identified safety as the foremost concern of the pilots. Rakotonarivo et al. ( Rakotonarivo et al. 2023 ) discovered three key factors that impact flight safety: (1) environmental factors like weather condition, geographical elements, airspace characteristics; (2) drone-specific parameters like automation, range, speed, and (3) mission-specific parameters such as payload, trajectories,

Read the complete text