CasaMaestro: Multi-View Panoramas for House-Scale 3D Reconstruction

Yuzhou Ji *, Xiaotian Yang *, Zhipeng Zhang † \dagger · 2026

Read CasaMaestro: Multi-View Panoramas for House-Scale 3D Reconstruction by Yuzhou Ji *, Xiaotian Yang *, Zhipeng Zhang † \dagger (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31086.txt.

Introduction

Ji, Yang, and Zhang introduce CasaMaestro, a model that reconstructs an entire house in 3D from a small set of panoramic photos. Home robots need fast, metric reconstructions of residential spaces, but standard pipelines built for narrow-view cameras need thousands of images to cover several rooms and accumulate drift over long alignment chains. LiDAR scanners solve this, at high cost.

CasaMaestro takes only twenty to fifty sparse multi-view panoramas and directly predicts metric depth and camera poses in one forward pass, yielding a full point cloud of the house. The authors report it is the first model to support house-scale reconstruction from panoramas. Tests on real and synthetic homes show robust, high-quality results, which they propose as a foundation for building indoor simulation environments for embodied AI.

Catalog record

CREATORYuzhou Ji *, Xiaotian Yang *, Zhipeng Zhang † \dagger
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.31086

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CasaMaestro: Multi-View Panoramas for House-Scale 3D Reconstruction Yuzhou Ji *, Xiaotian Yang *, Zhipeng Zhang † \dagger Abstract The rise of home-deployed embodied AI systems is driving a growing need for fast, metric 3D reconstruction of residential spaces to support navigation, interaction, and long-horizon task execution. However, the commonly used pinhole-camera 3D reconstruction pipelines struggle to model large indoor residences efficiently due to their limited field of view, to which achieving full coverage across multiple rooms often requires thousands of images and incurs drift from long chains of incremental alignment. In this work, we present CasaMaestro ( Spanish words meaning “house” and “master”), a feedforward model that can take only twenty to fifty sparse multi-view indoor panoramas as input and directly predicts metric depth along with camera poses, allowing fast point-cloud reconstruction of the entire house with full coverage. CasaMaestro is the first model that supports house-scale reconstruction with multi-view panoramas. Experiments show that CasaMaestro can robustly provide high quality results in both real-world and synthetic scenes, which can serve as a strong foundation for acquiring house-scale 3D indoor assets to be applied in close-loop simulation. 1 1 footnotetext: Equal contribution. \dagger Corresponding author. Figure 1 : Demonstration of CasaMaestro. The existing pinhole vision foundation models (VFM) require dense video streams for scene-level reconstruction, which is consuming in both human effort and computing resources, while also facing severe errors in overly long and short input sequences. We propose CasaMaestro, which takes only sparse indoor panoramas and directly completes house-scale (with multiple rooms) metric reconstruction, providing efficiency along with accuracy. 1 Introduction Figure 2 : Illustration of existing problems. Left visualization shows pinhole models either face limited FoV in sparse capture or accumulative error in dense sequence. Right figure shows pose accuracy and processing time under different input density. With embodied AI shifting toward in-the-wild operation in everyday homes, the construction of realistic and metrically consistent closed-loop simulation environments becomes increasingly important. In order to bridge the sim-to-real gap, reconstructing virtual 3D residential scenes from real-world capture has served as a crucial role for both training and testing, which, however, still faces many difficulties in practice. Although LiDAR scanning remains a reliable solution for acquiring high-quality 3D assets of homes, it is costly and hardware-dependent. In contrast, recent 3D vision foundation models [ 15 , 41 , 37 , 11 ] suggest a compelling alternative by reconstructing 3D from unconstrained image sequences, enabling 3D lifting for existing and even synthetic imagery without specialized sensors. However, today’s large-scale multi-view methods predominantly assume pinhole cameras, whose narrow field of view makes residential capture inefficient and fragile, where dense imaging is required for multi-room coverage and long trajectories exacerbate drift and error accumulation. As shown in Fig. 2 , pinhole methods suffer from severe displacement with sparse capture viewpoints that can not be solved (and may even worsen) by increasing single viewpoint scanning density due to accumulative error, while the time and GPU memory consumption are already extremely high. This fact substantially prevents the practical deployment of pure-vision reconstruction for real-home data acquisition. To improve reconstruction efficiency, several researches have explored using panoramic cameras, yet they are mostly restricted to single-view depth prediction [ 16 , 13 ] or to small-scale, short-range multi-view settings. For example, PanoSplatt3R [ 27 ] provides feed-forward 3DGS reconstruction from only panoramas, but can only handle 2 input views. PanoPose [ 32 ] uses separate pose network dedicated for relative pose estimation of panorama pairs to progressively predict long sequences, yet requires large view overlap and consequently a long image sequence for house-level scan. Although SPR [ 46 ] supports relatively larger view displacement comparing with PanoPose, it still demands video streams and is solely trained on 5 views setting, leaving the challenge of house-scale multi-view panoramic reconstruction unresolved. In this work, we present CasaMaestro, the first feedforward model that achieves extrinsic-free multi-view panoramic 3D reconstruction with sparse house-scale capture. Embracing a minimalist design philosophy [ 15 ] , CasaMaestro is simply built upon a DINOv2 [ 19 ] backbone that processes multiple views, paired with dedicated heads for depth and pose prediction. To explicitly tackle the severe view displacements inherent in sparse residential captures, we introduce a lightweight panoramic camera pose decoder. By incorporating a second stage attention mechanism across views, this decoder achieves superior pose estimation accuracy. Moreover, recognizing that existing datasets offer a restricted variety of scenes and viewpoints, we propose a novel panoramic data augmentation strategy via ERP (Equirectangular Projection) remapping, which generates abundant pairs of poses and views, significantly boosting model robustness. Ultimately, CasaMaestro takes only raw panoramas as input to directly predict metric depth and camera poses. This elegant pipeline is remarkably simple yet surprisingly effective, which enables the immediate point cloud reconstruction of entire homes, delivering full coverage and exceptional geometric consistency. Experiments show that our model is superior than existing methods with specifically 84% and 119% improvements of the AUC30 metric in real-world and synthetic scenes with balanced rotation and translation error, and also an average 21.98% decrease of AbsRel on unseen datasets comparing with the best previous results. In conclusion, we provide the following contribution: • We present CasaMaestro, the first feedforward model for metric house-scale panoramic reconstruction. • We design a lightweight camera pose decoder for stronger pose estimation with large displacement. • We propose a ERP panoramic data augmentation method for more pose-view pairs with existing data. 2 Related Work 2.1 3D Foundation Models Feed-forward models have advanced significantly in point cloud reconstruction while predicting multiple 3D attributes, and become 3D Foundation Models for many downstream vision tasks. The early DUSt3R [ 40 ] and MASt3R [ 12 ] predict a coupled scene representation but require further post-processing. The following works expand them towards more pipelines [ 8 , 18 , 9 , 20 ] and support extra input views [ 3 , 36 , 39 ] , but with limited quality compared to traditional optimization. Recently, multi-view models such as VGGT [ 37 ] have made excellent progress in 3D prediction, with many valuable efforts in faster reconstruction [ 28 ] , more accurate localization [ 31 , 7 ] and longer sequences [ 43 , 6 , 44 ] . Notably, \pi^{3} [ 41 ] employs a fully permutation-equivariant architecture and achieves higher robustness, while MapAnything [ 11 ] further enables a broad range of 3D vision tasks in a single feed-forward pass, pushing such models to real-world applications. Later, Depth Anything 3 [ 15 ] shows that a single plain transformer is sufficient as a backbone, achieving minimal modeling. Despite strong performance, the existing 3D foundation models are commonly based on pinhole cameras, leaving the challenge of panoramic 3D yet to be conquered. In this paper, we aim to provide a panoramic 3D foundation model for future applications. While many researches choose to implement special design for panoramas, we also hold the perspective [ 15 ] that architectural specialization is not really necessary and uses a vanilla DINO as our backbone. 2.2 Panoramic Depth Estimation With much larger view range than pinhole cameras, panoramic depth estimation serves as a promising technique for cost-efficient depth prediction method that may be used in autonomous driving and embodied

Read the complete text