MOSAIC-GS: Monocular Scene Reconstruction via Advanced Initialization for Complex Dynamic Environments

Svitlana Morkva 1 Maximum Wilder-Smith 1 Michael Oechsle 2 Alessio Tonioni 2 Marco Hutter 1 Vaishakh Patil 1 1 ETH Zürich 2 Google {smorkva, mwilder, mahutter, patilv}@ethz.ch {michaeloechsle,alessiot}@google.com · 2026

Read MOSAIC-GS: Monocular Scene Reconstruction via Advanced Initialization for Complex Dynamic Environments by Svitlana Morkva 1 Maximum Wilder-Smith 1 Michael Oechsle 2 Alessio Tonioni 2 Marco Hutter 1 Vaishakh Patil 1 1 ETH Zürich 2 Google {smorkva, mwilder, mahutter, patilv}@ethz.ch {michaeloechsle,alessiot}@google.com (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05368.txt.

Introduction

Svitlana Morkva and 5 co-authors reconstruct dynamic scenes from a single video camera using Gaussian splatting in a 2026 paper. Monocular reconstruction is inherently ill-posed: one viewpoint lacks the multi-view constraints needed to pin down geometry and motion. Their method, MOSAIC-GS, is fully explicit and computationally efficient, aiming for high-fidelity reconstruction of complex dynamic environments from ordinary monocular video.

The approach combines several geometric cues, including depth, optical flow, dynamic object segmentation, and point tracking. Rigidity-based motion constraints then turn those cues into preliminary three-dimensional scene dynamics. Reliable single-camera reconstruction would help robotics, film, and augmented reality, where rigs of synchronized cameras are often impractical and the only record is one moving viewpoint.

Catalog record

CREATORSvitlana Morkva 1 Maximum Wilder-Smith 1 Michael Oechsle 2 Alessio Tonioni 2 Marco Hutter 1 Vaishakh Patil 1 1 ETH Zürich 2 Google {smorkva, mwilder, mahutter, patilv}@ethz.ch {michaeloechsle,alessiot}@google.com
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-05368

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MOSAIC-GS: Monocular Scene Reconstruction via Advanced Initialization for Complex Dynamic Environments Svitlana Morkva 1 Maximum Wilder-Smith 1 Michael Oechsle 2 Alessio Tonioni 2 Marco Hutter 1 Vaishakh Patil 1 1 ETH Zürich 2 Google {smorkva, mwilder, mahutter, patilv}@ethz.ch {michaeloechsle,alessiot}@google.com Abstract We present MOSAIC-GS, a novel, fully explicit, and computationally efficient approach for high-fidelity dynamic scene reconstruction from monocular videos using Gaussian Splatting. Monocular reconstruction is inherently ill-posed due to the lack of sufficient multiview constraints, making accurate recovery of object geometry and temporal coherence particularly challenging. To address this, we leverage multiple geometric cues, such as depth, optical flow, dynamic object segmentation, and point tracking. Combined with rigidity-based motion constraints, these cues allow us to estimate preliminary 3D scene dynamics during an initialization stage. Recovering scene dynamics prior to the photometric optimization reduces reliance on motion inference from visual appearance alone, which is often ambiguous in monocular settings. To enable compact representations, fast training, and real-time rendering while supporting non-rigid deformations, the scene is decomposed into static and dynamic components. Each Gaussian in the dynamic part of the scene is assigned a trajectory represented as time-dependent Poly-Fourier curve for parameter-efficient motion encoding. We demonstrate that MOSAIC-GS achieves substantially faster optimization and rendering compared to existing methods, while maintaining reconstruction quality on par with state-of-the-art approaches across standard monocular dynamic scene benchmarks. \captionof figure MOSAIC-GS produces high-quality reconstructions of dynamic scenes, capturing fine details in dynamic regions more accurately while significantly reducing training and rendering time compared to prior methods. Beyond reconstruction, our approach enables dynamic scene segmentation and editing, allowing objects to be removed, isolated, or manipulated without any additional computational cost. 1 Introduction Recent advances in Neural Radiance Fields (NeRFs) [ 36 ] and 3D Gaussian Splatting [ 20 ] have revolutionized 3D scene reconstruction, enabling highly accurate and photorealistic modeling of complex environments. However, these techniques are limited to static scenes, restricting their applicability in domains that involve dynamic environments, such as robotics, autonomous driving, or digital-twin generation for virtual and augmented reality (VR/AR) applications. Several recent methods extend these ideas to dynamic scenes [ 6 , 38 , 39 , 25 , 33 , 54 ] , but the problem remains largely unsolved. Existing approaches often require extensive training time, substantial memory and storage resources, and suffer from slow rendering, while still exhibiting noticeable artifacts in regions with complex motion. Many of these methods also prioritize visual realism over physical consistency, leading to distortions when observed from unseen viewpoints. This issue is particularly pronounced in settings with restricted camera viewpoints or monocular video acquisition, where the limited field of view further complicates physically consistent reconstruction. Such scenarios are common in robotics and embedded devices, where cameras typically provide only narrow or stereo views rather than wide multi-view coverage. Moreover, the limited computational and memory resources of these platforms make compact and efficient scene representations essential. To address these challenges, we present MOSAIC-GS, an explicit and computationally efficient framework for dynamic scene reconstruction from monocular videos. MOSAIC-GS leverages a 3D Gaussian Splatting representation [ 20 ] constructed from different viewpoints of a dynamic scene acquired over an interval of time. Each view is paired with a camera pose and a depth estimate, obtained either by dedicated sensors or learned models. A key insight of our work is that inferring scene dynamics from purely visual data during photometric optimization is both inefficient and unreliable. Instead, we emphasize the importance of high-quality initialization, which has been shown to strongly influence reconstruction fidelity [ 20 ] and convergence speed [ 34 ] . Our pre-processing pipeline first focuses on robust identification and tracking of dynamic regions, enabling decomposition of the scene into static and dynamic components. This separation improves parameter efficiency and simplifies the subsequent optimization. We then estimate preliminary Gaussian motions from all available geometric and physical cues in the video, leveraging recent advances in optical flow [ 48 ] , object segmentation [ 40 ] , and per-point tracking [ 4 , 3 ] to extract per-point trajectories and object associations. We further incorporate rigidity constraints during this stage to enable refinement of these motions and efficient inference even in unobserved regions of the scene. To compactly encode motion, we represent these trajectories using time-dependent Poly-Fourier curves, following Lin et al. [ 28 ] , instead of the per-frame deformation models commonly used in prior monocular methods [ 45 , 52 , 24 ] . These trajectories are then used to initialize the dynamic Gaussians. Combined with disentanglement into static and dynamic components, this representation enables efficient modeling, fast training, real-time rendering, and robust handling of non-rigid deformations. The proposed MOSAIC-GS framework is particularly effective in monocular settings, where limited viewpoints make motion inference from visual cues alone unreliable. The main contributions of our work can be summarized as follows: • An advanced pre-processing framework for dynamic Gaussian Splatting that improves reconstruction quality and significantly accelerates training. • A robust pipeline for dynamic object detection and tracking across monocular video sequences. • A novel trajectory refinement method that extracts and optimizes 3D motion using per-instance rigidity constraints. • State-of-the-art training efficiency and inference speed, achieved through scene disentanglement, compact deformation representation, and accurate motion initialization. 2 Related Works 2.1 Moving object detection Detection of moving objects in video remains a challenging problem, particularly under changing illumination and camera motion. However, advances in deep learning, particularly convolutional neural networks (CNNs) [ 22 , 21 , 14 ] , recurrent neural networks (RNNs) [ 47 ] , and more recently transformer-based architectures [ 51 , 5 ] , have significantly improved robustness and accuracy. Building on these developments, recent motion detection frameworks integrate instance segmentation [ 13 , 41 ] of common dynamic objects, dense optical flow [ 46 , 48 , 17 ] , and epipolar geometry constraints to reliably detect motion even in challenging scenarios such as moving cameras or complex illumination [ 27 , 32 , 58 , 24 , 15 ] . 2.2 Novel-View Synthesis of Dynamic Scenes Novel-view synthesis aims to generate realistic images of a scene from previously unseen viewpoints. A major breakthrough came with the introduction of Neural Radiance Fields (NeRFs) [ 36 ] , which enabled high-fidelity 3D scene representations and have since been extended to capture dynamic content [ 39 , 37 , 38 , 27 ] . Despite their impressive quality, NeRF-based methods are computationally expensive and slow to render. To address this, hybrid approaches combine implicit neural fields with explicit 3D structures such as voxels or planes [ 9 , 2 , 42 , 44 , 1 ] , which improve efficiency but still struggle with real-time photorealistic rendering. This limitation has motivated the development of explicit point-based models such as 3D Gaussian Splatting [ 20 ] , which enable fast and high-quality rendering. Building on this foundation, recent works have extended Gaussian Splatting to dynamic scenes by modeling temporal motion in two main ways. The first line of work applies per-frame deformations, storing Gaussian transformations for each frame, either for all Gaussians or a smaller skeletal subset [ 33 , 16 , 45 , 24 , 52 ] . These methods estimate per-point trajectories either during optimization or via

Read the complete text