HORUS: A Mixed Reality Interface for Managing Teams of Mobile Robots
Omotoye Shamsudeen Adekoya*, Antonio Sgorbissa, Carmine Tommaso Recchiuto · 2025
Read HORUS: A Mixed Reality Interface for Managing Teams of Mobile Robots by Omotoye Shamsudeen Adekoya*, Antonio Sgorbissa, Carmine Tommaso Recchiuto (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02622.txt.
Introduction
Omotoye Shamsudeen Adekoya, Antonio Sgorbissa, and Carmine Tommaso Recchiuto present in 2025 HORUS, a mixed reality headset interface for running whole teams of mobile robots. Mixed reality has been tried for steering one robot at a time, but supervising many at once, the real shape of warehouse and rescue work, was largely unexplored.
Through the headset, an operator watches each robot's status and sees live sensor data projected into the room. Tasks can go to one robot, a subset, or the whole team from a mini-map station. The design treats the fleet, not the single machine, as the thing being driven.
Catalog record
| CREATOR | Omotoye Shamsudeen Adekoya*, Antonio Sgorbissa, Carmine Tommaso Recchiuto |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-02622
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HORUS: A Mixed Reality Interface for Managing Teams of Mobile Robots Omotoye Shamsudeen Adekoya*, Antonio Sgorbissa, Carmine Tommaso Recchiuto Abstract Mixed Reality (MR) interfaces have been extensively explored for controlling mobile robots, but there is limited research on their application to managing teams of robots. This paper presents HORUS: Holistic Operational Reality for Unified Systems, a Mixed Reality interface offering a comprehensive set of tools for managing multiple mobile robots simultaneously. HORUS enables operators to monitor individual robot statuses, visualize sensor data projected in real time, and assign tasks to single robots, subsets of the team, or the entire group, all from a Mini-Map (Ground Station). The interface also provides different teleoperation modes: a mini-map mode that allows teleoperation while observing the robot model and its transform on the mini-map, and a semi-immersive mode that offers a flat, screen-like view in either single or stereo view (3D). We conducted a user study in which participants used HORUS to manage a team of mobile robots tasked with finding clues in an environment, simulating search and rescue tasks. This study compared HORUS’s full-team management capabilities with individual robot teleoperation. The experiments validated the versatility and effectiveness of HORUS in multi-robot coordination, demonstrating its potential to advance human-robot collaboration in dynamic, team-based environments. I INTRODUCTION Multi-robot systems are increasingly deployed in critical applications such as search and rescue [ 1 ] , industrial inspection, and exploration, where rapid decision-making and robust situational awareness can be life-saving. However, managing a team of robots in dynamic, complex environments remains challenging due to the limitations of traditional control interfaces. Conventional 2D displays require operators to mentally fuse disparate information—from robot cameras, sensors, and maps—leading to increased cognitive load and potential errors. Mixed Reality (MR) interfaces, which blend real and virtual elements in real time, are emerging as powerful tools in human-robot interaction (HRI). MR extends Augmented Reality (AR) by anchoring virtual content to the real world and enabling two-way interaction between physical and digital objects. In robotics, MR interfaces allow human operators to visualize robots and their data overlaid on the real environment, offering intuitive means to monitor and control them. This capability is especially important for managing teams of mobile robots, where coordinating multiple agents demands rapid assimilation of varied data sources and clear visualization of the mission status [ 2 ] . Overlaying mission data, such as robot locations, paths, obstacles, and sensor feeds, onto real-world views has been shown to improve operator situational awareness, enabling users to detect mission-critical elements more accurately [ 2 ] . Fig. 1: The minimap displays two robots: Robot One, which has a projected camera view, and Robot Two, which has an overhead camera view. The Robot Manager Panel for Robot Two is activated, allowing the operator to check the robot’s status, toggle sensor data visualization on or off, assign a task, and initiate teleoperation. Laser scan data visualization is enabled for both robots. Building on these insights and the demonstrated benefits of MR interfaces for single-robot data visualization and control, this paper introduces HORUS (Holistic Operational Reality for Unified Systems), a novel MR interface designed specifically for managing teams of mobile robots. In our implementation, the Husarion ROSBot 2.0 platform serves as the testbed for HORUS. The system leverages a collaborative mini-map that integrates real-time data from each robot to form a unified operational picture. Through this interface, operators can monitor individual robot statuses, visualize sensor data (such as camera feeds, laser-scan outputs, navigation paths, and tf frames), and execute a variety of task commands, including setting goal points, defining waypoints, labeling poses, and drawing navigation plans. HORUS is organized around a Robot Manager panel that presents dedicated tabs for Robot Status, Robot Data Visualization, Tasks Assignment, and Single Robot Teleoperation for every single robot. Operators interact with each robot’s holographic representation on the mini-map, enabling them to inspect critical parameters and initiate control actions. In scenarios where an operator may need to teleoperate one robot in the team (e.g., manually drive one robot through a tricky area), two distinct teleoperation modes are supported: (i) mini-map teleoperation, where commands are issued by interacting with the robot’s model directly on the map, and (ii) semi-immersive teleoperation, which offers a detailed, flat, screen-like view (in both single and stereo 3D) for precise control while the mini-map remains in the operator’s view. By fusing data from all robots into a common, spatially registered map, HORUS not only enhances situational awareness but also allows fluid switching between high-level task assignment and direct robot control. This work addresses a significant gap in the literature, where most MR interfaces have predominantly focused on single-robot scenarios or have been confined to simulated environments. In contrast, HORUS has been validated on a real team of wheeled mobile robots, demonstrating its effectiveness in handling the complexities of multi-robot coordination in an operational setting. II RELATED WORKS Mixed Reality (MR) interfaces have gained momentum in robotics research due to their potential to overlay rich, spatially registered information onto the physical world. Although numerous single-robot systems have demonstrated the benefits of MR-based teleoperation, data visualization, and task management, extending these capabilities to multi-robot teams introduces greater complexity and new opportunities. This section reviews key developments, ranging from single-robot control to advanced multi-robot coordination, and discusses the existing challenges that our work seeks to address. Extensive work has shown that Mixed Reality (MR) can reduce cognitive load and improve operator performance in single-robot scenarios, typically by overlaying sensor data, paths, and status indicators within an immersive display. Systems such as ARviz [ 3 ] and iViz [ 4 ] highlight how headsets or mobile devices augment real-world views with robot information (e.g., camera feeds or 3D trajectories). This intuitive overlay fosters egocentric command inputs, enabling operators to “grab” or redirect holographic representations rather than manipulate abstract menus, thereby promoting more natural interactions [ 5 , 6 ] . Building on single-robot successes, researchers have explored drag-and-drop or gesture-based MR interfaces for high-level coordination of multiple robots. Chen et al. [ 7 ] demonstrated an immersive 3D map where each robot’s avatar is co-localized via multi-agent SLAM; tasks are assigned simply by moving holographic agents to the desired target. Similarly, Kennel-Maushart et al. [ 8 ] applied a board-game metaphor for multi-robot planning on a HoloLens 2, enabling non-experts to perform task allocation in robot teams with gesture-based interactions and minimal training. For teams operating in complex or partially occluded spaces, MR interfaces offer rich situational awareness by merging sensor feeds into a single, spatially aligned display. Users can effectively “see through walls” [ 7 ] , monitoring a swarm’s positions, planned paths, and camera streams in real time. This unified view supersedes traditional interfaces that require shifting between multiple 2D panels, reducing oversight and enabling timely interventions [ 2 ] . Finally, some tasks require finer control of specific robots. Immersive systems have shown how an operator can pivot from supervisory modes to direct teleoperation for navigation or manipulation tasks [ 6 , 8 ] . By embedding holographic cues (e.g., collision warnings or desired end-effector locations), MR can streamline complex telemanipulation for one robot while still tracking the rest of the team’s mission progress. Despite notable gains, MR