Multi-Agent Monocular Dense SLAM With 3D Reconstruction Priors

Yuchen Zhou, Haihang Wu* · 2025

Read Multi-Agent Monocular Dense SLAM With 3D Reconstruction Priors by Yuchen Zhou, Haihang Wu* (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-19031.txt.

Introduction

Yuchen Zhou and Haihang Wu extend a single camera SLAM system, which estimates a robot's pose while mapping an unknown space, to teams of agents working together. Dense single camera SLAM is accurate but costly because it optimizes iteratively; a recent system speeds it up with learned three dimensional reconstruction priors, yet works for one agent only.

Each agent in their system runs local SLAM with the same prior. Their maps then merge into one globally consistent map through loop closure based fusion. On real world data sets the multi agent system improves computational efficiency over leading methods while keeping similar mapping accuracy.

Catalog record

CREATORYuchen Zhou, Haihang Wu*
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE24 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-19031

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Agent Monocular Dense SLAM With 3D Reconstruction Priors Yuchen Zhou, Haihang Wu* Abstract Monocular Simultaneous Localization and Mapping (SLAM) aims to estimate a robot’s pose while simultaneously reconstructing an unknown 3D scene using a single camera. While existing monocular SLAM systems generate detailed 3D geometry through dense scene representations, they are computationally expensive due to the need for iterative optimization. To address this challenge, MASt3R-SLAM utilizes learned 3D reconstruction priors, enabling more efficient and accurate estimation of both 3D structures and camera poses. However, MASt3R-SLAM is limited to single-agent operation. In this paper, we extend MASt3R-SLAM to introduce the first multi-agent monocular dense SLAM system. Each agent performs local SLAM using a 3D reconstruction prior, and their individual maps are fused into a globally consistent map through a loop-closure-based map fusion mechanism. Our approach improves computational efficiency compared to state-of-the-art methods, while maintaining similar mapping accuracy when evaluated on real-world datasets. I Introduction I-A Single-Agent SLAM Visual Simultaneous Localization and Mapping (vSLAM) aims to estimate a mobile robot’s trajectory while reconstructing the 3D structure of an unknown environment using only camera input. [ 5 ] . vSLAM has found widespread application in robotic navigation and augmented reality, thanks to its ability to provide spatial awareness. Monocular SLAM, a subset of vSLAM, uses only a camera and has gained significant attention due to its simple configuration and low cost. Early monocular SLAM relied on sparse feature-based scene representations to reduce computational and memory demands [ 12 ] . Although such approaches enable accurate camera tracking with limited resources [ 2 ] , the resulting sparse maps lack geometric detail and are therefore unsuitable for tasks that require dense spatial understanding, such as robot navigation. With the advent of high-performance hardware such as GPUs, research has shifted toward dense representations—such as point clouds [ 15 , 6 ] and Truncated Signed Distance Functions (TSDFs) [ 14 ] . These representations capture detailed geometry but struggle with view-dependent appearance and suffer from high memory consumption in large scenes. To overcome this limitation, existing work has explored continuous, joint geometry–appearance representations. Neural Radiance Fields (NeRF) model the scene as a continuous volumetric function parameterized by a neural network, enabling photorealistic rendering and novel view synthesis. Despite their compactness and expressiveness, NeRF-based SLAM systems remain computationally expensive [ 21 , 27 ] , since the network must be evaluated hundreds of times per pixel during image rendering. Recent advances in 3D Gaussian splatting alleviate this problem by representing the scene as a set of Gaussian primitives, achieving comparable visual fidelity with substantially higher efficiency [ 11 , 17 ] . Nonetheless, these approaches still require iterative optimization for state estimation, including camera pose and scene reconstruction. To accelerate state estimation, studies have trained neural networks on large 3D datasets. These models can predict camera poses and scene structure in a single forward pass, significantly improving efficiency. DUST3R [ 24 ] introduces the pointmap representation to model 3D shape and trains a transformer to predict it from image pairs. MASt3R [ 10 ] improves image matching accuracy and efficiency by adding a network head that outputs dense local features. Subsequent work extends these ideas to handle hundreds of input views, enabling consistent multi-view 3D reconstruction [ 23 ] . However, these approaches are primarily designed for Structure-from-Motion (SfM), which reconstructs camera trajectories and 3D scene structure from a set of pre-captured images offline . Recent studies have adapted MASt3R for online and incremental SLAM. Building on MASt3R, MASt3R-SLAM [ 13 ] estimates pointmaps from image pairs and fuses them for local tracking (camera pose estimation) and mapping (3D structure reconstruction). Loop closures are employed to enhance pose accuracy and map consistency. Despite its strong performance, MASt3R-SLAM remains a single-agent system. While single-agent systems are sufficient for tasks like autonomous navigation in small environments, they are not enough for many applications, such as search and rescue or warehouse automation. In these scenarios, multiple robots must collaborate for efficient exploration, task execution, and data sharing, highlighting the need for multi-agent SLAM. I-B Multi-Agent SLAM Existing multi-agent SLAM systems generally adopt either a distributed or centralized architecture. In distributed approaches, each agent estimates its pose and constructs a local map independently, exchanging information only intermittently—an arrangement suitable for environments with limited communication capabilities [ 8 ] . Centralized systems, in contrast, depend on a central server that collects local maps and trajectories from all agents to perform global optimization [ 26 , 3 ] . In this study, we adopt a centralized system due to its superior 3D reconstruction quality Significant progress has been made in multi-agent SLAM. Most systems employing sensor configurations like monocular+IMU [ 19 ] , stereo [ 4 ] , or stereo+IMU [ 9 , 25 , 22 , 8 ] . While monocular cameras are more affordable and simpler, their use in multi-agent SLAM remains limited, primarily due to challenges such as arbitrary map scale and low feature density. To the best of our knowledge, CCM-SLAM [ 18 ] is one of the few multi-agent systems utilizing monocular cameras. However, it relies on sparse features for localization and mapping, which constrains its applicability in tasks that require dense spatial awareness. In this work, we extend MASt3R-SLAM [ 13 ] and propose the first multi-agent monocular dense SLAM framework. Each agent is equipped with MASt3R, a strong 3D reconstruction prior that enables accurate tracking and mapping. The resulting local trajectories and maps are then integrated into a consistent global map through a loop-closure-based fusion mechanism. The main contributions of this study are as follows: 1. We introduce the first multi-agent dense SLAM system leveraging 3D reconstruction priors. 2. We present a loop-closure-based local map fusion method that generates globally consistent maps. 3. We demonstrate that our system achieves competitive accuracy while significantly outperforming state-of-the-art methods in terms of computational speed. II Method Our framework is illustrated in Figure 1 . Each agent is equipped with an identical pre-trained 3D vision model, MASt3R (Section II-A ). Upon receiving an RGB image stream, each agent independently performs local tracking, mapping, and loop closure (Section II-B ) based on MASt3R’s learned 3D priors. After all agents complete their local SLAM processes, their submaps and trajectories are transmitted to a central server, which fuses them into a globally consistent map (Section II-C ). Fig. 1: System overview. Agent Side: Each agent processes its own RGBD stream, performing tasks including tracking, mapping, and local graph optimization. Upon completion, the agent sends its sub-map and trajectory to the central server. Server Side: The server performs both intra-agent and inter-agent loop closure detection, followed by global graph optimization. This process merges the submaps into a unified global map and updates the agent poses accordingly. II-A Background In this section, we revisit MASt3R. MASt3R is a pre-trained model that reconstructs the 3D structure from two-view input: ({{\mathbf{X}}}^{i}_{i},{{\mathbf{X}}}^{j}_{i},{{\mathbf{C}}}^{i}_{i},{{\mathbf{C}}}^{j}_{i},{{\mathbf{D}}}^{i}_{i},{{\mathbf{D}}}^{j}_{i},{{\mathbf{Q}}}^{i}_{i},{{\mathbf{Q}}}^{j}_{i})=\mathcal{F}_{M}(\mathcal{I}^{i},\mathcal{I}^{j}), (1) where \mathcal{I}^{i},\mathcal{I}^{j}\in\mathbb{R}^{H\times W\times 3} are input images, {{\mathbf{X}}}^{i}_{i},{{\mathbf{X}}}^{j}_{i}\in\mathbb{R}^{H\times W\times 3} are the corresponding pointmaps represented in the coordinate frame of camera i , {{\mathbf{C}}}^{i}_{i},{{\mathbf{C}}}^{j}_{i}\in\mathbb{R}^{H\times W\times 1} are their confidences, {{\mathbf{D}}}^{i}_{i},{{\mathbf{D}}}^{j}_{i}\in\mathbb{R}^{H\times W\times d} are d -dimensional matching features, and {{\mathbf{Q}}}^{i}_{i},{{\mathbf{Q}}}^{j}_{i}\in\mathbb{R}^{H\times W\times 1} are their confidences. To enable tracking and mapping in SLAM, pixel correspondences, {{\mathbf{m}}}_{i,j}=\mathcal{M}({{\mathbf{X}}}^{i}_{\smash{i}},{{\mathbf{X}}}^{j}_{\smash{i}},{{\mathbf{D}}}^{i}_{\smash{i}},{{\mathbf{D}}}^{j}_{\smash{i}}) ,

Read the complete text