Surf3R: Rapid Surface Reconstruction from Sparse RGB Views in Seconds

Haodong Zhu, Changbai Li, Yangyang Ren, Zichao Feng, Xuhui Liu, Hanlin Chen, Xiantong Zhen, Baochang Zhang · 2025

Read Surf3R: Rapid Surface Reconstruction from Sparse RGB Views in Seconds by Haodong Zhu, Changbai Li, Yangyang Ren, Zichao Feng, Xuhui Liu, Hanlin Chen, Xiantong Zhen, Baochang Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-04508.txt.

Introduction

Eight authors present Surf3R, a system that rebuilds 3D surfaces from a few ordinary photos in under ten seconds. Standard methods first calibrate cameras and estimate their poses, a slow and fiddly pre-step that blocks casual use. Surf3R skips that stage entirely and works in a single feedforward pass. Several reference views guide the rebuild together through a multi-branch design with cross-view attention, so each view fills gaps left by the others.

A second piece, a normal-based regularizer built on an explicit 3D Gaussian form, ties surface orientation to the other shape values during training. That coupling sharpens fine detail and keeps the surface consistent across views. On the ScanNet++ and Replica benchmarks, the system reached top scores on standard surface measures. The authors stress its speed and its ability to generalize to new scenes.

Catalog record

CREATORHaodong Zhu, Changbai Li, Yangyang Ren, Zichao Feng, Xuhui Liu, Hanlin Chen, Xiantong Zhen, Baochang Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-04508

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Surf3R: Rapid Surface Reconstruction from Sparse RGB Views in Seconds Haodong Zhu, Changbai Li, Yangyang Ren, Zichao Feng, Xuhui Liu, Hanlin Chen, Xiantong Zhen, Baochang Zhang Abstract Current multi-view 3D reconstruction methods rely on accurate camera calibration and pose estimation, requiring complex and time-intensive pre-processing that hinders their practical deployment. To address this challenge, we introduce Surf3R , an end-to-end feedforward approach that reconstructs 3D surfaces from sparse views without estimating camera poses and completes an entire scene in under 10 seconds . Our method employs a multi-branch and multi-view decoding architecture in which multiple reference views jointly guide the reconstruction process. Through the proposed branch-wise processing, cross-view attention, and inter-branch fusion, the model effectively captures complementary geometric cues without requiring camera calibration. Moreover, we introduce a D-Normal regularizer based on an explicit 3D Gaussian representation for surface reconstruction. It couples surface normals with other geometric parameters to jointly optimize the 3D geometry, significantly improving 3D consistency and surface detail accuracy. Experimental results demonstrate that Surf3R achieves state-of-the-art performance on multiple surface reconstruction metrics on ScanNet++ and Replica datasets, exhibiting excellent generalization and efficiency. 1 1 footnotetext: These authors contributed equally. 2 2 footnotetext: Corresponding author and Project lead. E-mail: [email protected] 1 Introduction 3D surface reconstruction is a long-standing problem that aims to create 3D surfaces of an object or scene captured from multiple viewpoints ( Broadhurst, Drummond, and Cipolla 2001 ; Kutulakos and Seitz 1999 ; Seitz and Dyer 1999 ) . This technique has wide applications in robotics, graphics, virtual reality, and other fields. Traditional 3D surface reconstruction methods typically include two main approaches: Structure-from-Motion (SfM) combined with Multi-View Stereo (MVS) ( Chen, Li, and Lee 2023 ; Huang et al. 2024 ; Chen et al. 2024a ) or volumetric methods ( Zak Murez and Rabinovich 2020 ; Sun et al. 2021 ) . The SfM+MVS pipeline involves first estimating camera poses and generating sparse 3D point clouds using SfM ( Crandall et al. 2013 ; Charatan et al. 2024 ; Schönberger and Frahm 2016 ) . This is followed by computing per-view depth maps and fusing them into the final 3D surface using MVS techniques ( Huang et al. 2024 ; Guédon and Lepetit 2024 ; Chen et al. 2024a ) . On the other hand, volumetric methods, such as Atlas ( Zak Murez and Rabinovich 2020 ) and NeuralRecon ( Sun et al. 2021 ) , predict 3D volumes like Truncated Signed Distance Function (TSDF) from multiple views, often avoiding the explicit depth map computation. Although these two kinds of methods achieve high-quality surface reconstruction, they often rely on prior knowledge or require nontrivial pre-processing steps, such as SfM to estimate camera intrinsics and extrinsics. These pre-processing steps often require heavy GPU computation and are time-consuming (typically taking 1–2 hours per scene on a modern GPU), making real-time inference challenging and reducing their practical usability. Figure 1: Comparison Between Traditional Methods and Our Approach. Traditional methods rely on SfM for sparse point clouds and calibrated poses, followed by different 3D Reconstruction Methods (RMs). In contrast, our method directly reconstructs the scene from uncalibrated images in under 10 s , eliminating the need for calibration or iterative refinement. To address the aforementioned limitations, inspired by DUSt3R ( Wang et al. 2024 ) , we propose Surf3R , the first feed-forward network that performs pose-free surface reconstruction from sparse RGB inputs in a single pass. Specifically, we first encode all input views using a shared encoder to extract multi-scale visual features. To effectively model cross-view information interactions, we introduce Feature-Refine (FR) blocks that jointly learn not only the pairwise relationships between a selected reference view and all other source views, but also the interactions among source views themselves. When reconstructing a large scene from sparse multi-view images, the geometric correspondence between a selected reference view and certain source views could be insufficient. This is because substantial changes in camera poses make it difficult to directly infer the relation between the reference view and those source views. To mitigate this issue, we further introduce a cross-reference fusion mechanism, implemented via a multi-branch design where multiple reference views are independently selected. Each branch processes the input views through its own FR blocks and integrates information using dedicated Cross-Reference Fusion (CRF) blocks, enabling effective propagation of long-range and complementary information across views. Based on the fused multi-view features, we first generate a sparse 3D point cloud for reconstruction. While directly converting this point cloud into a mesh using NKSR ( Huang et al. 2023 ) is feasible, our experiments (see Sec. 4.2 ) show that this naive approach yields poor reconstruction quality. The underlying limitation is that point-cloud supervision is applied in a view-separated manner and thus lacks global 3D consistency. We therefore adopt a Gaussian representation: each Gaussian resides in a unified 3D space and is projected into every view during rendering, so the per-view loss implicitly regularizes the entire scene and yields smoother, more accurate surfaces. The final per-pixel Gaussian primitives are derived from specifically designed Gaussian heads. To further facilitate accurate surface reconstruction from the predicted Gaussian parameters, we introduce a Depth-Normal Regularization strategy ( Chen et al. 2024b ) designed to enhance the geometric fidelity of the reconstructed surfaces. Specifically, we first apply a flattening operation to the Gaussian primitives to better align them with the local surface geometry. Subsequently, we introduce a D-Normal formulation, in which surface normals are not directly blended from 3D Gaussians, but instead derived from the gradient of the rendered depth map. As a result, it allows the Gaussian parameters to be directly supervised by surface normals, jointly optimizing the geometry and markedly enhancing 3D consistency and surface detail. Extensive experiments on the ScanNet++ datasets demonstrate that Surf3R achieves state-of-the-art surface reconstruction performance. It significantly outperforms both optimization-based and feed-forward baselines in terms of accuracy and completeness. Moreover, when evaluated on the unseen Replica dataset in a zero-shot setting, our model maintains competitive accuracy, demonstrating robust generalization to novel scenes. And it also delivers strong performance on novel view synthesis tasks. Notably, Surf3R reconstructs an entire scene in under 10 seconds , making it highly efficient for real-time or interactive applications. We summarize our contributions as follows: • We present Surf3R , the first feed-forward network for pose-free surface reconstruction from sparse multi-view RGB inputs. It achieves real-time surface reconstruction in under 10 seconds , offering both high efficiency and scalability. • We employ a multi-branch architecture where multiple reference views are jointly leveraged to capture long-range cross-view interactions. Furthermore, we introduce a Depth-Normal Regularization strategy to enhance geometric fidelity. • Extensive experiments on ScanNet++ and Replica datasets demonstrate that Surf3R achieves state-of-the-art surface reconstruction, generalizes zero-shot to new scenes, and remains competitive for novel-view synthesis. Figure 2: Overall Framework of Surf3R . The reference view branches are shown in orange, while the branches of other source views are shown in blue. Each model path uses a different reference view. For clarity, only one of the stacked FRBlock and CRFBlock is displayed. 2 Related Works Multi-View Surface Reconstruction. Multi-view surface reconstruction recovers dense geometry from images captured at multiple viewpoints. Classical pipelines fuse depth maps obtained by multi-view stereo ( Furukawa, Hernández

Read the complete text