NeuraLoc: Visual Localization in Neural Implicit Map with Dual Complementary Features

Hongjia Zhai, Boming Zhao, Hai Li, Xiaokun Pan, Yijia He, Zhaopeng Cui, Hujun Bao, Guofeng Zhang · 2025

Read NeuraLoc: Visual Localization in Neural Implicit Map with Dual Complementary Features by Hongjia Zhai, Boming Zhao, Hai Li, Xiaokun Pan, Yijia He, Zhaopeng Cui, Hujun Bao, Guofeng Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-06117.txt.

Introduction

Hongjia Zhai and co-authors teach a neural scene map to tell a camera where it stands in a 2025 paper. Maps built from glow fields are sharp, but past tools lack shape checks or eat storage on stored traits. The new tool learns a field of point marks inside the map itself, so no trait bank is kept.

A second field adds scene sense, which clears up marks that look alike. A match step lines up photo marks and map marks to close the gap between the two. On two test sets the tool trains three times faster and stores far less than past tools, while matching or beating them.

Catalog record

CREATORHongjia Zhai, Boming Zhao, Hai Li, Xiaokun Pan, Yijia He, Zhaopeng Cui, Hujun Bao, Guofeng Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-06117

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

NeuraLoc: Visual Localization in Neural Implicit Map with Dual Complementary Features Thanks: * Corresponding author: Guofeng Zhang ([email protected]) Thanks: This work was partially supported by the NSF of China (No.6242500063). Hongjia Zhai, Boming Zhao, Hai Li, Xiaokun Pan, Yijia He, Zhaopeng Cui, Hujun Bao, Guofeng Zhang Abstract Recently, neural radiance fields (NeRF) have gained significant attention in the field of visual localization. However, existing NeRF-based approaches either lack geometric constraints or require extensive storage for feature matching, limiting their practical applications. To address these challenges, we propose an efficient and novel visual localization approach based on the neural implicit map with complementary features. Specifically, to enforce geometric constraints and reduce storage requirements, we implicitly learn a 3D keypoint descriptor field, avoiding the need to explicitly store point-wise features. To further address the semantic ambiguity of descriptors, we introduce additional semantic contextual feature fields, which enhance the quality and reliability of 2D-3D correspondences. Besides, we propose descriptor similarity distribution alignment to minimize the domain gap between 2D and 3D feature spaces during matching. Finally, we construct the matching graph using both complementary descriptors and contextual features to establish accurate 2D-3D correspondences for 6-DoF pose estimation. Compared with the recent NeRF-based approaches, our method achieves a 3 × \times faster training speed and a 45 × \times reduction in model storage. Extensive experiments on two widely used datasets demonstrate that our approach outperforms or is highly competitive with other state-of-the-art NeRF-based visual localization methods. Project page: https://zju3dv.github.io/neuraloc I INTRODUCTION Visual localization plays a crucial role in many robotics applications [ 1 , 2 , 3 ] , aiming to estimate the 6-Degree-of-Freedom (6-DoF) camera pose of a query image with respect to a pre-built 3D map. With the advancement of deep learning, convolutional neural networks (CNNs) have shown great potential in extracting high-level contextual features, significantly promoting the development of visual localization. Current visual localization approaches can be broadly categorized into feature-based and regression-based methods. Feature-based methods [ 4 , 5 ] typically represent the scene using point clouds. These methods rely on keypoint detection and matching techniques [ 6 , 7 , 8 , 9 ] to reconstruct a 3D point map. Once 2D-3D correspondences between the query image and the point-cloud map are established, the 6-DoF pose can be estimated via the Perspective-n-Point (PnP) algorithm [ 10 ] . However, their performance is limited by the repeatability and discriminative power of the extracted or learned keypoint descriptors. Furthermore, feature-based methods require substantial storage to retain map information, such as 3D points, descriptors, and covisibility relationships, which can be storage-intensive. On the other hand, regression-based approaches [ 11 , 12 , 13 ] employ CNNs to directly regress the pose from features extracted from a single image or to estimate the 3D coordinates of pixels in the camera’s view. While these methods avoid the need for storing point-wise descriptors and demand less storage space, their generalization performance in large, complex scenarios is relatively poor. Additionally, regression-based methods typically require large amounts of 3D data to optimize the regression network. As a result, while these methods are more storage-efficient, they are generally inferior to feature-based methods in terms of localization accuracy and generalization performance. Recently, NeRF [ 14 ] has emerged as a new paradigm for implicit scene representation. The neural implicit representation uses multi-layer-perceptions (MLPs) and additional parametric encoding [ 15 , 16 , 17 , 18 , 19 ] to model the scene property and achieve impressive results in novel view synthesis and surface reconstruction. Benefiting from differentiable volume rendering [ 20 ] , NeRF-based approaches enable end-to-end parameter optimization without the need of 3D supervision. Several recent works [ 3 , 21 , 22 , 23 , 24 ] have explored using neural implicit representations to support 6-DoF pose estimation and refinement. For example, iNeRF [ 21 ] was the first method to use a pre-trained NeRF model for estimating the pose of an unknown query image. However, it is limited to object-level localization and requires an initial pose estimate. In visual localization, LENS [ 25 ] and NeRF-SCR [ 23 ] are regression-based approaches that use NeRF to synthesize virtual camera views to aid scene coordinate regression networks. However, similar to other regression-based methods, these approaches suffer from unsatisfactory localization results due to the lack of geometric constraints. To introduce geometric constraints, PNeRFLoc [ 22 ] uses point-based neural representation [ 26 ] to represent the scene map and performs 2D-3D feature matching for pose estimation. This was the first work to combine feature-based methods with NeRF-based representations for visual localization. While PNeRFLoc achieves satisfactory results across various scenes, it requires explicit storage of features for each point, leading to significant storage requirements for the reconstructed scene model. To address the issues mentioned above, we propose an efficient and novel visual localization approach, NeuraLoc, based on a learned neural implicit map. Specifically, to reduce the size of the scene model, we avoid explicitly storing point cloud descriptors. Instead, we implicitly learn a descriptor field from an MLP decoder, distilled from a 2D keypoint detection model [ 6 ] . Additionally, to address the semantic ambiguity of the learned descriptors and filter outliers, we distill a semantic contextual feature field, which helps construct the accurate matching graph. Furthermore, to reduce the domain gap between 2D and 3D descriptor feature spaces, we introduce a similarity alignment loss to align 2D-3D and 2D-2D similarity distributions. Finally, we construct a matching graph using the dual complementary features to establish 2D-3D correspondences for 6-DoF pose estimation. In summary, the major contributions of our proposed approach are summarized as follows: • We propose an efficient visual localization approach based on a reconstructed neural implicit map, achieving accurate localization with fewer scene model parameters. • To maintain localization performance with a compact scene model, we introduce dual complementary descriptors and semantic contextual features distilled from 2D foundation models for accurate 6-DoF pose estimation. • We reduce the domain gap between 2D and 3D feature spaces during matching by applying alignment loss on descriptor similarity distributions. II RELATED WORKS II-A Visual Localization Visual localization approaches can be broadly divided into regression-based [ 11 , 12 , 27 ] , and feature-based methods [ 4 , 22 , 28 ] . Regression-based approaches often rely on CNNs to directly regress either the camera pose [ 11 ] or the scene coordinates [ 29 ] . One of the key limitations of these methods is their heavy dependence on large amounts of 3D ground truth data for supervision. Additionally, they generally lack interpretability, which limits their performance in terms of both localization accuracy and generalization. On the other hand, feature-based methods [ 4 , 5 ] , which are the most commonly used for visual localization, utilize sparse or dense feature matching to estimate camera poses by establishing 2D-3D correspondences. Thanks to advanced keypoint detection [ 6 , 8 ] and matching techniques [ 7 , 30 ] , feature-based approaches can reliably compute 2D-3D correspondences between query images and a prebuilt 3D map. For pose estimation, these methods typically apply Perspective-n-Point [ 10 ] and RANSAC [ 31 ] algorithms to register query images. Regression-based methods tend

Read the complete text