MR-SLAM: Immersive Spatial Supervision for Multi-Robot Mapping via Mixed Reality
Prakash Aryan 1 , Cem Erdogdu 1 , Kavinaya Kumarchokkappan 1 , Timo Kehrer 1 , Sebastiano Panichella 1 2 · 2026
Read MR-SLAM: Immersive Spatial Supervision for Multi-Robot Mapping via Mixed Reality by Prakash Aryan 1 , Cem Erdogdu 1 , Kavinaya Kumarchokkappan 1 , Timo Kehrer 1 , Sebastiano Panichella 1 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16432.txt.
Introduction
Aryan and four colleagues put the supervisor inside the map: the operator, wearing a Meta Quest 3 headset, steers three simulated TurtleBot3 robots. A see-through view lets real objects block the virtual scene, while space-fixed panels report mapping progress on the spot. Each robot builds its own map with a SLAM Toolbox copy, and a ROS 2 back end fuses the grids as they arrive. In trials, agreement between robot map copies reached 94.7 percent across robot pairs, while merged maps covered 17.9 square metres and scans ran at 8.83 Hz.
Catalog record
| CREATOR | Prakash Aryan 1 , Cem Erdogdu 1 , Kavinaya Kumarchokkappan 1 , Timo Kehrer 1 , Sebastiano Panichella 1 2 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23522 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.16432
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MR-SLAM: Immersive Spatial Supervision for Multi-Robot Mapping via Mixed Reality Thanks: This work was supported by Horizon 2020 (EU Commission) project InnoGuard, Marie Skłodowska-Curie Actions Doctoral Networks (HORIZON-MSCA-2023-DN), the SNSF SwarmOps project (No. 200021_219732), and the Hasler Foundation project on UAV Reliability and Societal Trust (No. 2025-02-27-311). Prakash Aryan 1 , Cem Erdogdu 1 , Kavinaya Kumarchokkappan 1 , Timo Kehrer 1 , Sebastiano Panichella 1 2 Abstract Operating a multi-robot fleet for simultaneous localization and mapping (SLAM) in applications such as building inspection or warehouse-aisle monitoring requires the operator to maintain spatial awareness of each robot’s position and mapping state, a task that scales poorly on conventional 2D interfaces. We present MR-SLAM, a mixed reality (MR) system in which an operator wearing a Meta Quest 3 headset teleoperates three simulated TurtleBot3 robots through a passthrough view with real-world occlusion, while spatially anchored dashboard panels report mapping progress in situ. Each robot runs an independent SLAM Toolbox instance whose occupancy grid is merged in real time on a Robot Operating System 2 (ROS 2) back-end. Across five 9-minute evaluation sessions, the system delivered scans at 8.83 ± 0.16 8.83\pm 0.16 Hz, mapped 17.9 ± 0.8 17.9\pm 0.8 m 2 of merged occupancy, and reached 94.7 ± 0.5 % 94.7\pm 0.5\% cross-instance occupancy consistency across robot pairs. An additional session recorded 6.3 ms median transform jitter and 26.7 m 2 coverage of a 41 m 2 grid. We position MR-SLAM as a reference implementation for combining passthrough mixed reality supervision with multi-robot SLAM on consumer hardware. I Introduction Simultaneous localization and mapping (SLAM) allows mobile robots to operate without external positioning infrastructure [ 7 , 12 ] . The problem was first formulated as a joint state estimation task [ 39 ] and later grounded in probabilistic robotics [ 41 ] . Extending SLAM to multi-robot teams enables faster area coverage and fault tolerance, but introduces challenges in communication, map fusion, and operator supervision [ 23 ] . These challenges are seen in time-critical applications such as building inspection and warehouse inventory, where an on-site operator must coordinate multiple robots from a single station. The SLAM back-end has matured from graph-based optimizers [ 16 ] to distributed systems [ 22 , 21 ] , yet the operator interface for multi-robot supervision during mapping has received limited attention. Robot teleoperation typically relies on desktop workstations with 2D displays and joystick or keyboard input [ 38 , 20 ] . These interfaces require the operator to mentally reconstruct 3D spatial relationships from flat views, and this cognitive burden grows with the number of robots [ 49 ] . Virtual Reality (VR), augmented reality (AR), and mixed reality (MR) head-mounted displays (HMDs) have been shown to improve situational awareness in single-robot teleoperation [ 40 , 46 , 17 ] as well as facilitate safety assessment of autonomous systems [ 5 ] . Walker et al. [ 45 ] reported a 28% improvement in operator effectiveness using an MR interface for single-robot navigation. However, none addresses the coordination challenges of supervising multiple robots during active SLAM. In prior work, we developed SimNav-XR [ 4 ] , an extended reality platform for single-robot simulation integrating Unity, ROS 2, and the Meta Quest 3. SimNav-XR demonstrated MR-based robot control with LiDAR simulation and Nav2 integration but was limited to a single robot without real-time SLAM. This paper presents MR-SLAM, which extends SimNav-XR to multi-robot SLAM with three contributions: (1) a passthrough MR interface on the Meta Quest 3 rendering three simulated TurtleBot3 robots with raycasted LiDAR, spatial occlusion, and joystick teleoperation; (2) a multi-robot SLAM pipeline with independent SLAM Toolbox [ 25 ] instances and real-time map merging via multirobot_map_merge [ 6 , 9 ] ; and (3) a spatially anchored dashboard design displaying coverage, overlap, scan rate, and latency metrics. The system targets scenarios where an operator supervises a small robot fleet from a fixed location, such as inspecting a building wing or monitoring warehouse aisles. II Related Work II-A Multi-Robot SLAM Multi-robot SLAM architectures range from centralized to fully decentralized [ 23 , 3 ] . Centralized systems such as CCM-SLAM [ 37 ] aggregate data at a server for joint optimization; this simplifies map fusion but introduces a communication bottleneck and a single point of failure. Distributed systems such as DOOR-SLAM [ 22 ] and Kimera-Multi [ 11 , 42 ] perform peer-to-peer pose-graph optimization with outlier rejection, improving scalability at the cost of increased synchronization complexity. Swarm-SLAM [ 21 ] provided a ROS 2 decentralized framework supporting multiple sensor modalities, and LAMP [ 13 ] and LAMP 2.0 [ 10 ] addressed subterranean multi-robot SLAM during the DARPA SubT Challenge. Occupancy grid mapping [ 14 , 28 ] remains standard for 2D LiDAR. ORB-SLAM2 [ 29 ] established feature-based visual SLAM, and Google Cartographer [ 18 ] demonstrated real-time 2D LiDAR SLAM with branch-and-bound loop closure. For ROS 2 specifically, Macenski et al. developed SLAM Toolbox [ 25 ] as a lifelong 2D SLAM solution, and benchmarked modern visual SLAM approaches for Nav2 integration [ 26 ] . Map merging for occupancy grids was formalized by Carpin and Birk [ 8 , 6 ] using spectral methods and stochastic search, and extended with Hough-transform acceleration [ 9 ] . The multirobot_map_merge ROS package has been applied in multi-TurtleBot3 deployments [ 30 , 44 ] . II-B Mixed Reality for Robot Teleoperation Surveys by Suzuki et al. [ 40 ] and Walker et al. [ 46 ] catalogued AR/MR strategies for human-robot interaction (HRI). Early work demonstrated intent communication via HMD overlays [ 36 , 17 ] . Whitney et al. [ 48 ] introduced ROS Reality for Unity-based virtual reality (VR) teleoperation. On the Meta Quest platform, Quest2ROS [ 47 ] measured 0.46 mm tracking accuracy with 82 ms latency, and Quest2ROS2 [ 24 ] extended this to ROS 2. Morando and Loianno [ 27 ] demonstrated MR-based human-drone collaboration with spatial awareness sharing. Roldán et al. [ 35 ] addressed multi-robot VR monitoring without SLAM, and the Cyber-Physical Control Room [ 45 ] provided MR perspectives for single-robot teleoperation. However, none addresses the cognitive and coordination challenges of supervising multiple robots during active SLAM. II-C ROS–Unity Integration The Unity Robotics Hub [ 43 ] provides ros_tcp_connector for Unity–ROS communication. Allspaw et al. [ 1 ] benchmarked Unity–ROS bridges and found ros_tcp_connector offers the best latency-integration trade-off. Platt and Ricks [ 32 ] showed that Unity matches or exceeds Gazebo for SLAM simulation with better scalability. These integration frameworks enable real-time bidirectional communication, yet none has been applied to multi-robot SLAM with an immersive operator interface. II-D Positioning of This Work Table I compares MR-SLAM with related systems. Prior work addresses either MR-based single-robot teleoperation [ 45 , 27 , 47 ] or multi-robot SLAM without an MR interface [ 21 , 30 ] . Roldán et al. [ 35 ] built a multi-robot VR interface but did not integrate SLAM. Patel et al. [ 31 ] explored MR for multi-robot interaction but without a SLAM pipeline. In contrast, MR-SLAM uniquely combines passthrough MR visualization, real-time multi-robot SLAM with occupancy grid merging, and spatial performance feedback into a unified system, enabling operators to