UP-SLAM: Adaptively Structured Gaussian SLAM with Uncertainty Prediction in Dynamic Environments

Wancai Zheng, Linlin Ou, Jiajie He, Libo Zhou, Xinyi Yu, Yan Wei · 2025

Read UP-SLAM: Adaptively Structured Gaussian SLAM with Uncertainty Prediction in Dynamic Environments by Wancai Zheng, Linlin Ou, Jiajie He, Libo Zhou, Xinyi Yu, Yan Wei (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22335.txt.

Introduction

Wancai Zheng and five co-authors present UP-SLAM, a real-time mapping system for robots moving through scenes where things move too, in this 2025 arXiv paper. It builds the map from Gaussian splats and splits tracking from mapping so they run in parallel. A probabilistic octree manages the splats, adding and pruning them without hand-set thresholds.

A training-free estimator fuses several error signals to judge, pixel by pixel, what is likely moving, catching new object types without labels. A temporal encoder and borrowed image features enrich the map. Across hard test sets the system beat leading rivals on tracking accuracy and image quality, while keeping a clean, reusable map of the static world.

Catalog record

CREATORWancai Zheng, Linlin Ou, Jiajie He, Libo Zhou, Xinyi Yu, Yan Wei
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-22335

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

UP-SLAM: Adaptively Structured Gaussian SLAM with Uncertainty Prediction in Dynamic Environments Wancai Zheng, Linlin Ou, Jiajie He, Libo Zhou, Xinyi Yu, Yan Wei Abstract Recent 3D Gaussian Splatting (3DGS) techniques for Visual Simultaneous Localization and Mapping (SLAM) have significantly progressed in tracking and high-fidelity mapping. However, their sequential optimization framework and sensitivity to dynamic objects limit real-time performance and robustness in real-world scenarios. We present UP-SLAM, a real-time RGB-D SLAM system for dynamic environments that decouples tracking and mapping through a parallelized framework. A probabilistic octree is employed to manage Gaussian primitives adaptively, enabling efficient initialization and pruning without hand-crafted thresholds. To robustly filter dynamic regions during tracking, we propose a training-free uncertainty estimator that fuses multi-modal residuals to estimate per-pixel motion uncertainty, achieving open-set dynamic object handling without reliance on semantic labels. Furthermore, a temporal encoder is designed to enhance rendering quality. Concurrently, low-dimensional features are efficiently transformed via a shallow multilayer perceptron to construct DINO features, which are then employed to enrich the Gaussian field and improve the robustness of uncertainty prediction. Extensive experiments on multiple challenging datasets suggest that UP-SLAM outperforms state-of-the-art methods in both localization accuracy (by 59.8%) and rendering quality (by 4.57 dB PSNR), while maintaining real-time performance and producing reusable, artifact-free static maps in dynamic environments. The project: https://aczheng-cai.github.io/up_slam.github.io/ Figure 1: UP-SLAM compresses the 3DGS [ 1 ] representation into adaptive voxel, and constructs an artifact-free static feature map by leveraging uncertainty prediction to filter out transient objects. The dashed region on the right presents results from baseline methods. In contrast, our approach achieves high-fidelity static scene reconstructions in the presence of dynamic elements. 1 Introduction Visual Simultaneous Localization and Mapping (SLAM) is a core technology for embodied intelligence and virtual reality. Traditional SLAM algorithms typically assume static environments, which has facilitated the development of numerous effective systems [ 2 , 3 , 4 ] . However, this assumption restricts the applicability of SLAM in dynamic real-world environments, thereby impeding advancements in robotics and related fields. Recent SLAM approaches [ 5 , 6 , 7 ] leverage object detection and multiple-view geometry theory to reduce the impact of dynamic objects. While these approaches enhance system robustness in dynamic environments, they heavily depend on prior knowledge of dynamic objects and the reliability of detection algorithms. Advances in high-fidelity scene representations, such as Neural Radiance Fields [ 8 ] (NeRF) and 3D Gaussian Splatting [ 1 ] (3DGS), have motivated interest in introducing uncertainty modeling into 3D reconstruction. Recent studies [ 9 , 10 , 11 ] show that incorporating uncertainty prediction can significantly enhance robustness to transient scene elements. These uncertainty-aware models can achieve high-quality reconstructions even under intermittent occlusions. However, these methods depend on advantageous conditions, such as accurate camera poses and sparse viewpoints, which are challenging to achieve in SLAM systems using continuous frame inputs. To address these challenges, a real-time RGB-D SLAM system named UP-SLAM is presented for robust pose estimation and static scene reconstruction in dynamic environments. Our approach compresses 3DGS into structured anchors encoded by multiple shallow multilayer perceptrons (MLPs). A probabilistic octree is introduced to enable adaptive adjustment of anchors to delete redundant anchors caused by dynamic objects. Furthermore, by decoupling motion mask generation from map optimization, UP-SLAM enables parallel tracking and mapping, supporting real-time localization. In the tracking process, we propose a training-free, optimization-based multi-modal consistency estimation method that fuses geometric cues with DINO features for effective dynamic object recognition. In the mapping process, to further enhance reconstruction under dynamic conditions, a temporal encoder that leverages sinusoidal positional encoding is designed to embed inter-frame information into the MLP, thereby increasing the representational capacity. In addition, the inconsistent appearance and motion of dynamic objects across frames provide valuable cues for uncertainty prediction. Therefore, robust DINO features are fed into a shallow MLP for per-pixel uncertainty estimation, enabling continuous motion mask refinement and enhancing reconstruction robustness. Our primary contributions are as follows: ( i ) An uncertainty-aware parallel tracking and mapping framework is proposed to effectively mitigate dynamic disturbances without relying on predefined semantic annotations, thereby enabling the construction of high-quality, artifact-free static maps. ( ii ) We propose an adaptive structured 3DGS scene representation with a probabilistic octree, which supports automatic Gaussian primitive allocation or pruning in dynamic environments. This approach enhances localization accuracy and reduces model size. ( iii ) We integrate our approach into ORB-SLAM3 [ 2 ] and perform comprehensive evaluations on multiple datasets. Additionally, we introduce a protocol for assessing rendering quality in dynamic environments, and we will release our datasets to the public. 2 Related work 2.1 Traditional Visual SLAM Over the past few decades, visual SLAM has made remarkable progress, leading to the development of numerous outstanding algorithms [ 3 , 2 , 4 ] . However, these methods generally assume a static environment and neglect the velocity components in observations, making them susceptible to severe drift when dynamic objects are present. To address this issue, dynamic SLAM methods [ 12 , 7 ] based on object detection or semantic segmentation have been proposed. DynaSLAM [ 5 ] leverages predefined semantic labels from Mask R-CNN [ 13 ] , combined with geometric constraints, to reduce the influence of dynamic objects. However, this dependence on prior labels significantly limits the generalizability of the method. To reduce reliance on semantic priors, the method [ 14 ] takes advantage of spatial correlations between 3D points to eliminate dynamic features, while other methods [ 15 , 16 ] detect inconsistencies between point clouds to localize moving objects. Most existing methods emphasize tracking robustness but neglect the construction of semantically enriched and reusable maps, which are critical for high-level tasks such as navigation and scene understanding. To bridge this gap, we design to efficiently distill high-dimensional visual features from DINOv2 [ 17 ] into the 3DGS representation, enabling the construction of a dense, feature-rich map that serves as a reliable foundation for downstream robotic tasks. 2.2 NeRF and 3DGS SLAM NeRF SLAM. Neural implicit SLAM methods have recently attracted increasing attention for their capability to reconstruct high-fidelity scenes with continuous representations. iMAP [ 18 ] is the first to introduce volumetric NeRF scene representations into SLAM, encoding geometry through a single MLP and feature grid for implicit mapping. However, the long-term localization performance is hindered by the catastrophic forgetting issue associated with single-network representations. To address this, NICE-SLAM [ 19 ] proposes a hierarchical grid-based feature structure, significantly improving scalability and real-time performance for large-scale scene reconstruction. Subsequent works such as Vox-Fusion [ 20 ] , Co-SLAM [ 21 ] , and ESLAM [ 22 ] incorporate signed distance fields into hybrid representations, achieving notable gains in reconstruction quality and computational efficiency. While these methods demonstrate impressive performance in static scenes, they tend to degrade under dynamic conditions. More recent approaches have sought to extend NeRF-based SLAM to dynamic environments. For example, RodynSLAM [ 23 ] combines semantic priors and an optical flow estimator to mask dynamic regions. The method [ 24 ] pre-trains a dynamic object classifier and incrementally updates it by feeding features from objects identified through residuals between the map and ground truth. This enables the system to learn new dynamic features over time. However,

Read the complete text