FA-BARF: Frequency Adapted Bundle-Adjusting Neural Radiance Fields

Rui Qian, Chenyangguang Zhang, Yan Di, Guangyao Zhai, Ruida Zhang, Jiayu Guo, Benjamin Busam, Jian Pu · 2025

Read FA-BARF: Frequency Adapted Bundle-Adjusting Neural Radiance Fields by Rui Qian, Chenyangguang Zhang, Yan Di, Guangyao Zhai, Ruida Zhang, Jiayu Guo, Benjamin Busam, Jian Pu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-12086.txt.

Introduction

Rui Qian and co-authors present, in a 2025 paper, a faster way to rebuild 3D scenes from photos when the camera positions are not known exactly. Neural radiance fields render strikingly real new views of a scene. They also need the camera poses, and a hand tuned schedule that slowly raises detail is used to keep the joint fitting of scene and poses from failing.

That schedule, a low pass filter over time, also slows the fitting down. The new method swaps it for a frequency adapted filter that acts over space instead. The authors build theory linking the scene encoding to pose fitting, and show their filter damps the wobble the old one caused. A spatial filter can also use the overlap of pose uncertainty across views to correct poses productively. In broad tests, the method sped fitting while holding quality, they report.

Catalog record

CREATORRui Qian, Chenyangguang Zhang, Yan Di, Guangyao Zhai, Ruida Zhang, Jiayu Guo, Benjamin Busam, Jian Pu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-12086

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FA-BARF: Frequency Adapted Bundle-Adjusting Neural Radiance Fields Rui Qian, Chenyangguang Zhang, Yan Di, Guangyao Zhai, Ruida Zhang, Jiayu Guo, Benjamin Busam, Jian Pu Abstract Neural Radiance Fields (NeRF) have exhibited highly effective performance for photorealistic novel view synthesis recently. However, the key limitation it meets is the reliance on a hand-crafted frequency annealing strategy to recover 3D scenes with imperfect camera poses. The strategy exploits a temporal low-pass filter to guarantee convergence while decelerating the joint optimization of implicit scene reconstruction and camera registration. In this work, we introduce the Frequency Adapted Bundle Adjusting Radiance Field (FA-BARF), substituting the temporal low-pass filter for a frequency-adapted spatial low-pass filter to address the decelerating problem. We establish a theoretical framework to interpret the relationship between position encoding of NeRF and camera registration and show that our frequency-adapted filter can mitigate frequency fluctuation caused by the temporal filter. Furthermore, we show that applying a spatial low-pass filter in NeRF can optimize camera poses productively through radial uncertainty overlaps among various views. Extensive experiments show that FA-BARF can accelerate the joint optimization process under little perturbations in object-centric scenes and recover real-world scenes with unknown camera poses. This implies wider possibilities for NeRF applied in dense 3D mapping and reconstruction under real-time requirements. The code will be released upon paper acceptance. I Introduction In the last few decades, Structure from Motion (SfM) [ 1 ] and visual Simultaneous Localization and Mapping (visual SLAM) [ 2 , 3 ] techniques have gained significant interest from both the computer vision and robotic communities, including a wide range of applications, such as robot navigation [ 4 ] and augmented reality [ 5 ] . As a crucial part of refining a visual reconstruction to produce jointly optimal 3D structure and viewing parameter estimates in SfM and SLAM, classical bundle adjustment is a large sparse geometric parameter estimation problem, the parameters being the combined 3D feature coordinates and camera poses. While NeRF [ 6 ] provides a space-efficient implicit neural representation of dense geometric reasoning, bundle adjustment combined with the implicit 3D structure integrates abundant geometric information with a compact memory footprint for downstream vision tasks, which used to be limited by the sparse nature of output 3D point clouds in the classical context. Given a collection of images captured by camera sensors, implicit bundle adjustment targets to recover the 3D scene as a neural network mapping 3D features to complex signals (e.g. density or color), which can synthesize images from arbitrary views through volumetric rendering [ 7 ] , and register the corresponding camera poses to locate the ego-motion of sensors. Considering camera poses as independent variables in SE(3), the BARF series methods [ 8 , 9 , 10 ] render the implicit model of the 3D scene to the observed views through initialized poses, construct photometric error between rendered and ground truth pixels as the loss function, and optimize poses and learnable scene representation jointly. Fig. 1: Comparision of pose optimization process between FA-BARF and BARF. (a) FA-BARF utilizes a frequency adapted spatial low-pass filter to adjust the ability of optimizing poses among different frequencies. (b) BARF adopts a temporal low-pass filter to guide pose optimization from low frequency to high frequency. (c) The temporal low-pass filter causes frequency fluctuation, impeding the process of pose optimization with frequency switch during the training process. Despite BARF’s notable ability of reconstruction and registration, the adopted hand-crafted frequency annealing strategy [ 11 ] sacrifices the efficiency of the implicit model’s learning process to guarantee the convergence of the algorithm. As illustrated in Fig. 1 , BARF applies a smooth mask on the different frequency bands (from low to high) of the implicit model over the course of optimization, acting like a temporal low-pass filter. The temporal filter guides poses from a coarser direction associated with lower frequencies to a finer direction associated with higher frequencies while introducing the frequency fluctuation. The frequency fluctuation means the optimization process of poses is impeded when the learned frequencies are disturbed by higher frequencies joined later. To address the decelerated training process and negative optimization impact caused by the temporal low-pass filter, we propose Frequency Adapted Bundle-Adjusting NeRF (FA-BARF), an innovative implicit bundle adjustment transforming the temporal low-pass filter to a frequency-adapted spatial low-pass filter. In this paper, we aim to preclude the negative impact of frequency fluctuation while recovering NeRF from imperfect camera poses. Starting with a theoretical framework, we analyze the influence on pose optimization caused by different frequency components in NeRF representation through position encoding [ 6 ] . Furthermore, we show that our frequency-adapted strategy can mitigate frequency fluctuation through substituting the temporal low-pass filter for a frequency-adapted spatial low-pass filter. The proposed spatial low-pass filter also enhances the sensitivity of rendered views related to varying poses and optimizes corresponding poses effectively by introducing radial uncertainty overlap among different views. To this end, we validate that FA-BARF can accelerate pose convergence and NeRF training process under little perturbations in object-centric scenes, and also recover real-world scenes in the form of implicit representation with unknown camera poses. In summary, we present the following contributions: • We provide a theoretical framework to analyze the relationship between the position encoding and pose optimization, setting a foundation for interpreting the impact of different frequencies in the joint optimization of reconstruction and registration. • We present that the proposed frequency-adapted low-pass filter can guarantee the optimal results of reconstruction and registration by eliminating the frequency fluctuation phenomenon caused by the classical temporal filter and exploiting the radial uncertainty overlap of different views. • Our proposed FA-BARF can curtail more than 50% of training time, and improve registration accuracy and view synthesis quality, compared to the original BARF in object-centric scenes. In real-world scenes, FA-BARF can also outperform with fewer registration errors and higher perceptual similarity in view synthesis. II Related Work Implicit Bundle-Adjusting Algorithms. Given a set of input image tracks, bundle adjustment is performed to refine a visual reconstruction to produce jointly optimal structure and viewing parameter estimates [ 12 ] in modern SfM [ 13 ] and visual SLAM systems [ 2 , 14 ] , which aim to recover the 3D feature from initial noisy or unknown camera poses. As the dawn of the Neural Radiance Field (NeRF) [ 15 ] is breaking, the 3D implicit map has been integrated into the framework of bundle adjustment, as an extension of classical direct methods [ 16 ] , exploiting photometric consistency to define the loss function. According to different optimization patterns related to camera poses, these implicit bundle adjustment algorithms tilt into two main orientations, (a) global registration [ 17 , 8 , 15 ] that optimizes absolute poses consistently and (b) local-to-global registration [ 18 , 10 ] that optimizes absolute poses and relative poses progressively. Note that global registration methods are fundamental strategies adopted by local-to-global registration methods in the local optimization phase, our work targets to enhance the accuracy and convergence rate of poses in the SE(3) manifold, providing a better baseline with less time-consuming in different implicit bundle adjustment algorithms. Explicit and Implicit Pose Optimization. According to different parameterization of camera poses, the pose optimization algorithms can be

Read the complete text