Global Structure-from-Motion Meets Feedforward Reconstruction

Linfei Pan, Johannes Schönberger, Marc Pollefeys · 2026

Read Global Structure-from-Motion Meets Feedforward Reconstruction by Linfei Pan, Johannes Schönberger, Marc Pollefeys (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26103.txt.

Introduction

Pan, Schönberger and Pollefeys merge old and new methods for structure-from-motion, recovering camera poses and 3D scene shape from photos. Old methods fail on flat texture, scant overlap and symmetries, where feedforward rebuilding shines. Feedforward methods, though, trail old ones on scale, accuracy or toughness in standard scenes. The authors map those limits in detail and build a pipeline using each side's strengths. Tests plus an open-source release show gains where either method alone falters.

Catalog record

CREATORLinfei Pan, Johannes Schönberger, Marc Pollefeys
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79492
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26103

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Global Structure-from-Motion Meets Feedforward Reconstruction Linfei Pan, Johannes Schönberger, Marc Pollefeys Abstract Structure-from-Motion – the process of simultaneously estimating camera poses and 3D scene structure from a collection of images – remains a central challenge in computer vision, with many open problems yet to be solved. Recent advances in feedforward 3D reconstruction have made significant strides in overcoming persistent failure cases of classical SfM methods, particularly in scenarios characterized by low texture, limited overlap, and symmetries. However, while feedforward approaches excel in these challenging conditions, they often face limitations regarding scalability, accuracy, or robustness, and typically fall short of classical methods in standard reconstruction settings. In this work, we systematically analyze these limitations and propose a new Structure-from-Motion pipeline by combining the respective strengths of classical and feedforward methods. Extensive experiments across multiple datasets show the benefits of our approach, achieving state-of-the-art results across a wide range of scenarios. We share our system as an open-source implementation at https://github.com/colmap/gluemap . 1 Introduction Structure-from-Motion (SfM) tackles the problem of reconstructing 3D scene structure and cameras given a set of images. It is a fundamental technique in computer vision and serves as a critical building block for numerous applications like localization 35 , multi-view stereo 38 , novel-view-synthesis 20 , or 3D training data generation 45 . Throughout its long history 44 , progress in the field has been primarily driven by optimization-based algorithms 51 ; 41 ; 28 ; 36 ; 30 ; 24 , which we refer to as classical methods in the remainder of this text. While these approaches differ in their specific formulations, they generally share a common structure: correspondence search using pairwise (local) feature matching and (global or incremental) reconstruction using robust optimization. To this day, SIFT 25 remains the standard choice for correspondence search. The global reconstruction paradigm 30 sets itself apart from the incremental 36 one by estimating camera poses and scene structure in a single, unified global optimization step, rather than incrementally building up a partial reconstruction using repeated local and global optimizations. The global approach generally results in improved runtime scalability and better robustness against symmetry issues. The state-of-the-art classical SfM systems 36 ; 30 achieve high levels of reliability 31 for sufficiently overlapping input images with enough parallax and discriminative scene texture. In contrast, these methods frequently struggle when the input images are sparse, have limited parallax, or do not contain enough distinctive features to match. Figure 1 : We evaluate on 5 datasets, featuring different challenges. The average rank on each dataset is reported. Classical and feedforward methods struggle under different settings, while ours consistently maintains best performance across the board. To address the limitations of classical approaches, the research community has increasingly focused on replacing hand-crafted modules in the 3D reconstruction pipeline with learned feedforward components, while still leveraging classical optimization techniques. For example, learned features like ALIKED 56 and learned matchers like LightGlue 23 enable more reliable matching under challenging appearance changes. More recently, fueled by significant advances in large-scale model training, the community has achieved breakthrough results by solving the multi-view 3D reconstruction problem end-to-end with a single feedforward architecture, effectively eliminating the need for explicit geometric solvers and optimization. After a long period of incremental progress, these methods have enabled a substantial leap forward, resolving several persistent failure cases that challenged traditional approaches. However, these methods are no silver bullet. In many cases, they still lag behind classical techniques in terms of scalability, accuracy, and robustness. The reliance on heavy transformer-based architectures leads to high memory requirements, limiting scalability to only several hundred input views at relatively low image resolutions, thus limiting accuracy. Recent efforts 53 ; 9 have sought to address these scalability issues, but they still struggle to process beyond a thousand images. Moreover, as we will demonstrate later, these approaches can exhibit counterintuitive behavior in which adding more input images does not necessarily improve the outputs. For scenarios where classical SfM methods succeed, feedforward approaches typically lag significantly behind in terms of accuracy. When feedforward methods do approach classical performance in these nominal cases, they often do so by incorporating optimization-based bundle adjustment (BA), which only partially bridges the gap. In terms of robustness, none of the existing methods can reliably handle multiple connected components, resolve symmetric structures, or systematically reject outliers such as irrelevant input images. Our key contributions are an analysis of the limitations of classical and feedforward 3D reconstruction. We then use these insights to propose a novel pipeline by combining their respective strengths. Extensive experiments across multiple datasets show the benefits of our approach, achieving state-of-the-art results across a wide range of scenarios. 2 Related Work 2.1 Classical Methods Classical SfM algorithms broadly categorize into incremental and global methods. Both types begin with correspondence search, which typically involves extracting local image features 25 ; 10 ; 56 and matching them across images 23 . To establish the view graph, images can be paired exhaustively or, for greater scalability, by employing image retrieval techniques 1 ; 37 to identify overlapping image pairs. Correspondence search concludes by estimating two-view geometries 16 using robust estimation techniques. The primary distinction between incremental and global pipelines lies in how they estimate cameras and structure. Incremental SfM pipelines – such as Bundler 41 or COLMAP 36 – build the reconstruction by adding images one at a time. At each step, they perform robust BA to jointly refine cameras and 3D points, interleaving this optimization as new images are incorporated. Global SfM pipelines, in contrast, estimate all cameras and 3D points simultaneously from the entire set of images. Historically, incremental methods have been regarded as more robust, particularly in challenging scenarios. However, recent methods like GLOMAP 30 demonstrate that global SfM pipelines can now achieve comparable robustness and accuracy but with better efficiency and scalability characteristics. In our work, we build upon the global SfM paradigm. In global SfM, camera intrinsics, rotations, and translations are usually recovered in different stages. For camera intrinsics, Sweeney et al . 42 proposed to perform view-graph calibration. The estimated intrinsics are then used to decompose two-view geometries into relative camera poses before estimating global camera rotations using rotation averaging 15 ; 27 ; 5 . Next, many systems use translation averaging 57 ; 29 ; 2 to solve for global camera translations, but since pairwise translations are only up-to-scale, the formulation is often ill-posed. GLOMAP 30 addresses this with global positioning to simultaneously recover camera poses and 3D points, but it can get stuck in local minima when tracks are insufficient. Similarity averaging 7 takes a different approach by using depth-derived scale constraints but remains sensitive to noise. All approaches finalize the reconstruction with global BA to improve accuracy. Classical SfM methods face significant challenges in certain types of scenes, particularly those that are texture-less, have low image overlap, or exhibit low parallax and symmetries. In areas with low texture , local feature matching becomes unreliable or even impossible. Without sufficient correspondences, accurate two-view geometry estimation is not feasible. This, in turn, leaves BA under-constrained, often resulting in reconstruction failures. When images have low overlap , it becomes difficult to constrain the scale of the reconstruction. Since relative camera poses estimated

Read the complete text