Rethinking Pose Refinement in 3D Gaussian Splatting under Pose Prior and Geometric Uncertainty
Mangyu Kong, Jaewon Lee, Seongwon Lee, Euntai Kim, [1.5mm] Yonsei University · 2026
Read Rethinking Pose Refinement in 3D Gaussian Splatting under Pose Prior and Geometric Uncertainty by Mangyu Kong, Jaewon Lee, Seongwon Lee, Euntai Kim, [1.5mm] Yonsei University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16538.txt.
Introduction
Kong and co-authors steady pose refinement in 3D Gaussian Splatting. The method's render quality is high, yet results hinge on the starting pose. Guessed poses from retrieval models carry no error bars. Flaws in the rebuilt geometry also poison the solver. Their framework samples poses by Monte Carlo. A Fisher-guided PnP step weighs the geometric uncertainty. No retraining or extra labels are needed. Tests indoors and outdoors gained accuracy and stability.
Catalog record
| CREATOR | Mangyu Kong, Jaewon Lee, Seongwon Lee, Euntai Kim, [1.5mm] Yonsei University |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55519 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.16538
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Rethinking Pose Refinement in 3D Gaussian Splatting under Pose Prior and Geometric Uncertainty Mangyu Kong, Jaewon Lee, Seongwon Lee, Euntai Kim, [1.5mm] Yonsei University Abstract 3D Gaussian Splatting (3DGS) has recently emerged as a powerful scene representation and is increasingly used for visual localization and pose refinement. However, despite its high-quality differentiable rendering, the robustness of 3DGS-based pose refinement remains highly sensitive to both the initial camera pose and the reconstructed geometry. In this work, we take a closer look at these limitations and identify two major sources of uncertainty: (i) pose prior uncertainty, which often arises from regression or retrieval models that output a single deterministic estimate, and (ii) geometric uncertainty, caused by imperfections in the 3DGS reconstruction that propagate errors into PnP solvers. Such uncertainties can distort reprojection geometry and destabilize optimization, even when the rendered appearance still looks plausible. To address these uncertainties, we introduce a relocalization framework that combines Monte Carlo pose sampling with Fisher Information–based PnP optimization. Our method explicitly accounts for both pose and geometric uncertainty and requires no retraining or additional supervision. Across diverse indoor and outdoor benchmarks, our approach consistently improves localization accuracy and significantly increases stability under pose and depth noise. Our code is available on the project website . 1 Introduction Figure 1 : UGS-Loc. Our proposed framework refines camera pose by incorporating pose prior uncertainty via Monte Carlo sampling and geometric uncertainty via Uncertainty fields, achieving robust localization without retraining. Figure 2 : Comparison between Previous 3DGS-based Pose Refinement and Our Proposed UGS-Loc. (a) Existing pipelines refine a single deterministic pose prior using rendered depth, which makes them vulnerable to inaccurate priors and geometric errors—leading to bad correspondences and unstable refinement. (b) Our method instead applies Monte Carlo pose sampling and geometric uncertainty from Fisher Information to guide correspondences and focus refinement on reliable regions, achieving robust pose estimation even under poor pose priors and uncertain geometry. Visual localization aims to estimate the 6-DoF camera pose of a query image within a known scene and plays a key role in AR/VR, autonomous driving, and robotics. A main factor governing the performance of localization methods is how the scene is represented. Traditional structure-based pipelines rely on sparse 3D point clouds reconstructed via Structure-from-Motion 44 ; 46 . Local features extracted from the query image are matched to reference images or 3D points 45 ; 11 ; 41 , and the resulting correspondences are used to estimate the camera pose through PnP algorithms 13 ; 25 ; 12 . Scene Coordinate Regression (SCR) approaches 4 ; 5 ; 3 extend this paradigm by predicting dense scene coordinates, and Absolute Pose Regression (APR) methods 19 ; 8 ; 27 ; 7 ; 48 learn to map images directly to poses. These approaches illustrate how advances in map representation, from sparse geometric maps to implicitly learned coordinate or pose models, continue to shape the design of localization systems. Recently, neural scene representations such as NeRF 38 and 3DGS 20 emerged as a new type of scene representation, and they triggered the development of new pose-refinement methods. These methods can be summarized as render-and-compare. They first render a synthetic view from an initial pose obtained by retrieval 1 ; 23 , APR, or SCR, and then align this view to the query image using photometric loss 64 ; 53 ; 30 or feature distances 56 ; 6 ; 33 . More recent methods 32 ; 17 ; 27 extend this idea: they find dense 2D–2D correspondences 26 ; 52 between the query and the rendered view, lift these matches into 3D using rendered depth, and solve the pose with PnP-RANSAC. This correspondence-driven version of render-and-compare now leads pose refinement and achieves state-of-the-art relocalization accuracy in rendering-based pose refinement. However, when the pose refinement methods are applied to 3DGS, we believe that the existing pose refinement methods overlook some points and may degrade some performance. 3D Gaussian Splatting models geometry with explicit elliptical primitives, but the primitives are just approximate and the depth they render is not uniformly reliable. Regions reconstructed from sparse training views 61 or contaminated by dynamic objects 21 often exhibit noticeable artifacts, leading to spatial variations in geometric accuracy. Despite these spatial variations, current refinement methods treat all rendered depths as equally trustworthy when lifting 2D–2D correspondences into 3D. As a result, they ignore the fact that reconstruction fidelity can vary significantly across different parts of the scene. This challenge becomes even more pronounced when considering the dependence on an accurate pose prior. Since 3DGS-based refinement 32 ; 17 ; 27 operates by rendering synthetic views from a single initial estimate, the quality of correspondence hinges on how well this pose is aligned with the true scene content. When the pose prior is biased or occluded by artifacts, the rendered viewpoints may exhibit limited visibility, causing feature matchers 26 ; 11 ; 52 to struggle to establish reliable correspondences. To address these challenges, we propose UGS-Loc, an uncertainty-aware pose refinement framework tailored to 3D Gaussian Splatting. UGS-Loc represents the pose prior as a distribution and explores it through Monte Carlo sampling, enabling the refinement pipeline to recover even when the initial pose lies far from reliable regions of the scene. In parallel, we quantify geometric uncertainty directly from the Gaussian parameters via Fisher Information 18 and integrate this signal into the PnP-RANSAC process, guiding correspondence sampling toward geometrically trustworthy regions. By jointly modeling pose and geometric uncertainties inherent to 3DGS, UGS-Loc achieves robust, training-free localization that remains stable and consistent across challenging scene conditions. 2 Related Works Visual Localization. The task of visual localization is to determine the pose of a query image in relation to a reconstructed 3D scene. A commonly used pre-built representation of the scene is a 3D point cloud reconstructed via Structure-from-Motion (SfM) 67 . Each 3D point in this cloud is associated with a set of visual descriptors from database images, allowing the system to establish reliable 2D–3D correspondences during localization. Query image keypoints are matched between the query image and the 3D model to establish 2D–3D correspondences 44 ; 54 ; 11 ; 45 . Scene coordinate regression (SCR) methods 58 ; 3 ; 4 train the network to acquire 2D-3D correspondences directly. The correspondences obtained from both methods are then fed into a PnP solver to estimate the camera pose. Alternatively, absolute pose regression (APR) 19 ; 48 predicts the 6-DoF camera pose directly from a query image through a neural network. However, these approaches often suffer from limited pose accuracy. Our method refines the poses estimated by SCR and APR methods using 3D Gaussian Splatting (3DGS) without requiring any additional training. Radiance Field-based Visual Localization. To overcome the sparsity and limited expressiveness of traditional 3D point cloud representations, recent works have explored dense scene representations such as radiance fields for visual localization. Leveraging the powerful novel-view synthesis capability of Neural Radiance Fields (NeRF) 38 and Gaussian Splatting (3DGS) 20 , these methods utilize photo-realistic rendering for both pose refinement and data augmentation. For instance, radiance field–based augmentation enriches APR frameworks 8 ; 27 ; 40 by generating synthetic training views across diverse viewpoints and lighting conditions, improving robustness and generalization to unseen domains.