MOSAIC: Generating Consistent, Privacy-Preserving Scenes from Multiple Depth Views in Multi-Room Environments
Zhixuan Liu, Haokun Zhu, Rui Chen, Jonathan Francis, Soonmin Hwang, Ji Zhang, Jean Oh · 2025
Read MOSAIC: Generating Consistent, Privacy-Preserving Scenes from Multiple Depth Views in Multi-Room Environments by Zhixuan Liu, Haokun Zhu, Rui Chen, Jonathan Francis, Soonmin Hwang, Ji Zhang, Jean Oh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13816.txt.
Introduction
Zhixuan Liu and co-authors build digital twins of multi-room interiors from depth images alone, in work from 2025. Depth images hide faces and text, so twins built from them protect privacy. Stitching many room views into one steady model is the hard part.
Their model weighs how views overlap while it generates the scene, rather than adding rooms one by one. That design stops errors from piling up across rooms. More overlap provably steadies the process and sharpens output. No extra training is needed as scenes grow more complex. Tests beat top baselines on image quality scores. Uses include home robots, virtual tours, and planning tools.
Catalog record
| CREATOR | Zhixuan Liu, Haokun Zhu, Rui Chen, Jonathan Francis, Soonmin Hwang, Ji Zhang, Jean Oh |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-13816
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MOSAIC: Generating Consistent, Privacy-Preserving Scenes from Multiple Depth Views in Multi-Room Environments Zhixuan Liu, Haokun Zhu, Rui Chen, Jonathan Francis, Soonmin Hwang, Ji Zhang, Jean Oh Abstract We introduce a diffusion-based approach for generating privacy-preserving digital twins of multi-room indoor environments from depth images only. Central to our approach is a novel M ulti-view O verlapped S cene A lignment with I mplicit C onsistency (MOSAIC) model that explicitly considers cross-view dependencies within the same scene in the probabilistic sense. MOSAIC operates through a multi-channel inference-time optimization that avoids error accumulation common in sequential or single-room constraints in panorama-based approaches. MOSAIC scales to complex scenes with zero extra training and provably reduces the variance during denoising process when more overlapping views are added, leading to improved generation quality. Experiments show that MOSAIC outperforms state-of-the-art baselines on image fidelity metrics in reconstructing complex multi-room environments. Resources and code are at https://mosaic-cmubig.github.io . Figure 1: For privacy-preserving scenarios where RGB collection is restricted, MOSAIC generates consistent RGB images from depth data captured along robot paths, guided by text prompts. These outputs further enable 3D reconstruction of multi-room environments. 1 Introduction Autonomous scene reconstruction is a crucial capability in robotics and computer vision [ 24 , 25 , 17 , 1 ] , with applications spanning virtual reality, architectural design, and AI-driven simulation. However, existing multi-view 3D reconstruction methods rely heavily on capturing RGB images, which poses significant privacy risks in sensitive environments such as hospitals, factories, elder care facilities, and private residences. Capturing RGB data in these settings can inadvertently expose personal information [ 32 , 47 ] , limiting the practicality of generative AI solutions for real-world deployments where visual privacy must be preserved. A promising solution for privacy-sensitive environments involves deploying mobile robots that collect only geometric data, such as depth images: this strategy preserves the scene’s structural layout while preventing the capture of sensitive texture information. In order to create digital replicas of these scenes, we need algorithms that are capable of generating high-quality, multiview-consistent sequential images that align with ground truth geometry, while maintaining visual coherence across viewpoints. Various approaches for scene-level multi-view image generation have been proposed, with autoregressive generation techniques [ 15 , 10 , 45 , 44 , 8 ] representing the current mainstream trend. These approaches leverage powerful inpainting models to iteratively render unseen parts of the scene from sequential camera perspectives. However, these approaches exhibit style drift, where images generated for initial viewpoints often differ stylistically from those produced when revisiting the same location from different angles, creating visual inconsistencies. Moreover, warp-and-inpainting methods frequently encounter depth misalignment issues, leading to compounding errors over sequential generations [ 36 , 23 ] . Unlike prior methods that perform diffusion synchronization tasks [ 2 , 20 , 11 , 22 , 18 ] in constrained settings (limited viewpoint variations), our task requires coherence between viewpoints based on arbitrary trajectory features, extensive perspective changes, and spatial discontinuities. This fundamental distinction makes existing methods ineffective for complex multi-room environments with widely-varying camera positions. To overcome these limitations, we present MOSAIC ( M ulti‑view O verlapped S cene A lignment with I mplicit C onsistency), a training‑free diffusion framework that converts privacy‑preserving depth sequences into photorealistic, geometry‑aligned RGB views across large, cluttered, multi‑room environments. Previous work [ 37 ] enforces consistency by fine‑tuning a modified architecture on trajectory‑specific data, limiting scalability and generalization; MOSAIC instead performs a lightweight multi‑view inference‑time optimization that scales to unseen environments and enforces cross‑view agreement at every denoising step, while preserving the intrinsic knowledge of a large pre‑trained model [ 28 ] . A depth‑weighted projection loss highlights the most reliable viewpoints, and a final pixel‑space refinement converts latent‑space coherence into precisely aligned images. As more overlapping views are added, MOSAIC reduces variance and tightens image–depth alignment, unlike autoregressive pipelines that compound errors, thus delivering stable reconstructions over arbitrarily long trajectories. Our experiments demonstrate superior performance over existing methods across multiple metrics, establishing a new state-of-the-art for scene level multi-view image generation with geometric priors. In summary, (1) we introduce MOSAIC, a training-free diffusion pipeline that turns arbitrary depth sequences into photorealistic geometry-aligned RGB views, (2) we devise an inference-time sampler, with depth-weighted projection loss and late pixel refinement, that enforces strict cross-view coherence, (3) we theoretically prove that adding overlapping views monotonically reduces denoising variance, curbing error accumulation, and (4) we demonstrate state-of-the-art results in fidelity, prompt-following, and geometry alignment in cluttered, multi-room environments. 2 Related Work Diffusion‑guided scene‑level multi‑view generation. Recent advances in 2D diffusion-based generative models [ 35 , 13 , 33 , 46 , 28 , 9 ] have catalyzed significant progress in scene-level multiview image generation, yet most pipelines rely on iterative warping and inpainting for view completion [ 15 , 10 , 45 , 44 , 8 ] , where style drift accumulates across iterations and produces artifacts. A panorama‑first strategy mitigates drift by re‑projecting a single generated panorama [ 36 , 21 , 34 , 40 ] , but cannot cope with multi‑room layouts or long, unconstrained trajectories. MVDiffusion [ 37 ] requires training an additional module for cross-view consistency, which limits the generalization to unseen environments and prompt following fidelity. SceneTex [ 7 ] assumes a pre‑meshed scene and performs score‑distillation texture optimization, requiring \sim 20 hours per scene and often seeking to a single appearance mode; our training‑free DDIM process finishes in few minutes while preserving generation diversity under single text prompt. Recent video‑diffusion models provide temporal coherence [ 4 , 14 , 43 , 48 ] , but lack depth conditioning and break under large viewpoint changes. Our method, on the other hand, explicitly models cross-view consistency by optimizing MOSAIC objective and generalizes to arbitrary viewpoints without imposing layout constraints. Diffusion Synchronization. Several works have explored synchronization techniques for diffusion models [ 2 , 20 , 11 , 22 , 18 ] to achieve consistency across multiple outputs. However, existing approaches primarily address specialized cases: panoramic view generation [ 2 , 20 , 18 ] , orthogonal transformations [ 11 ] , or object-level consistency with evenly distributed camera positions and well established UV map [ 18 , 22 ] . Our work addresses a fundamentally different challenge: maintaining consistent image generation across scene-level camera trajectories with arbitrary viewpoints and significant perspective variations. Diffusion Inference-Time Optimization. To circumvent the need for large-scale model fine-tuning, several approaches have explored test-time optimization with directional guidance. Diffusion-TTA [ 29 ] adapts discriminative models using pre-trained generative diffusion models, updating parameters through gradient backpropagation at inference time. Other methods [ 26 , 38 , 27 ] compute directional controls for single-instance generation (music, images) by calculating directional losses during diffusion and back-propagating to update denoising features. While these methods focus on intra-level control with easily quantifiable directions, our approach is the first to propose multi-channel test-time optimization for cross-view control using self-provided directional guidance, maintaining consistency across arbitrary viewpoints without requiring additional training. Figure 2: MOSAIC overview. (a) Multi-channel denoising. Each depth–text–conditioned view is assigned its own latent channel. A shared denoiser iteratively refines the latent set while a multi-channel inference-time optimizer keeps the channels synchronized. (b) Projection loss. At every step the predicted clean latents z_{0} guided