A View-consistent Sampling Method for Regularized Training of Neural Radiance Fields
Aoxiang Fan, Corentin Dumery, Nicolas Talabot, Pascal Fua · 2025
Read A View-consistent Sampling Method for Regularized Training of Neural Radiance Fields by Aoxiang Fan, Corentin Dumery, Nicolas Talabot, Pascal Fua (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04408.txt.
Introduction
Aoxiang Fan and three co-authors improve a 3D scene method that learns a space from flat photos, in a 2025 paper. The method, neural radiance fields, builds a volume it can render from new angles. Depth hints help a lot on real photos, but models that guess depth need costly 3D training data and often err, above all outdoors.
Instead of one fixed depth guess per ray, the team builds a spread of likely depths from image clues. Low-level color cues join high-level features drawn from large foundation models at the matching picture points. Sampling depths from that spread during training quietly steers the field toward shapes that agree across views. The trick adds guidance without trusting any single depth guess.
Catalog record
| CREATOR | Aoxiang Fan, Corentin Dumery, Nicolas Talabot, Pascal Fua |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-04408
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A View-consistent Sampling Method for Regularized Training of Neural Radiance Fields Aoxiang Fan, Corentin Dumery, Nicolas Talabot, Pascal Fua Abstract Neural Radiance Fields (NeRF) has emerged as a compelling framework for scene representation and 3D recovery. To improve its performance on real-world data, depth regularizations have proven to be the most effective ones. However, depth estimation models not only require expensive 3D supervision in training, but also suffer from generalization issues. As a result, the depth estimations can be erroneous in practice, especially for outdoor unbounded scenes. In this paper, we propose to employ view-consistent distributions instead of fixed depth value estimations to regularize NeRF training. Specifically, the distribution is computed by utilizing both low-level color features and high-level distilled features from foundation models at the projected 2D pixel-locations from per-ray sampled 3D points. By sampling from the view-consistency distributions, an implicit regularization is imposed on the training of NeRF. We also utilize a depth-pushing loss that works in conjunction with the sampling technique to jointly provide effective regularizations for eliminating the failure modes. Extensive experiments conducted on various scenes from public datasets demonstrate that our proposed method can generate significantly better novel view synthesis results than state-of-the-art NeRF variants as well as different depth regularization methods. 1 Introduction 3D scene reconstruction from multiple images is a long-standing vision problem [ 11 ] but the recent advent of Neural Radiance Fields (NeRFs) [ 28 ] has delivered a significant performance boost, especially given a dense set of input images. However, in much the same way the old shape-from-shading was ill-posed [ 32 ] , so is the NeRF reconstruction problem: as shown in [ 62 ] , in the absence of explicit or implicit regularization, a set of training images can be fitted independently of the recovered geometry. This phenomenon, known as shape-radiance ambiguity, is particularly evident when the input views are not dense enough, even though using Multi-Layer Perceptrons (MLPs) for scene representation weakly regularizes the scene reconstructions [ 62 , 60 ] . Many kinds of regularizers have been proposed to improve on this, such as imposing geometric constraints [ 19 , 30 ] , training to directly predict radiance fields by using networks conditioned on image features [ 6 , 59 , 50 ] , or constraining depth [ 8 , 49 , 60 ] . The first is difficult to do for complicated scenes while the second is often limited to very specific 3-view setting and not easily generalizable to unbounded scenes or scalable to more input views. The last, depth regularization, has proved to be the more widely applicable. However, it typically requires excessive expensive 3D supervision, and can produce unreliable predictions on challenging open-space scenes that produce artifacts in the final NeRF reconstruction. Figure 1 : View-consistent sampling. Our central idea is to pre-compute a view-consistency distribution along rays and to perform importance sampling according to this distribution. As a result, the sampling will concentrate around surface points instead of random points in the capture volume. To remedy this, we propose to use view-consistency distributions per-ray instead of fixed depth predictions to implement a sampling technique along the rays that implicitly regularizes the training of NeRF, as depicted by Fig. 1 . Specifically, we first distill geometric information from the redundant feature representations of foundation models to reduce their dimensionality and to alleviate memory requirements, while preserving the information most likely to be consistent across views and discarding the rest. Given these high-level distilled features along with low-level color features, we compute a view-consistency metric for every point along the rays and introduce an adaptive sampling scheme that favors view-consistent points, on the assumption that they are more likely to lie on a real-world surface. Furthermore, we also introduce a depth-pushing loss to force the model to favor samples that are farther away from the camera origin, which prevents the kind of background collapse artifact [ 2 ] that frequently happens in NeRF reconstruction of real-world unbounded scenes. In effect, the proposed view-consistent sampling and the depth-pushing loss focus the NeRF reconstruction process on the part of the capture volume close to the true surface, thus providing implicit regularization and preventing the overfitting problem [ 62 ] . Our contribution is therefore a novel view-consistent sampling technique to implicitly regularize the training of NeRF. We show that our method is able to achieve significantly better novel view synthesis results compared to existing NeRF competitors with regularizations. Our implementation is based on open-source software and will be made publicly available. 2 Related Works NeRF Variants. The emergence of Neural Radiance Fields (NeRF) [ 28 ] is an immediate consequence of the study on Implicit Neural Representations (INR) [ 42 , 37 , 14 , 27 ] , which laid the foundation of NeRF by introducing powerful network-based scene representation models. NeRFs use them to effectively encode 3D scene properties and trains on posed images via a volume rendering equation that differentially relates 3D scenes to 2D images. NeRFs deliver outstanding image synthesis results but tend to be slow, sometimes requiring several days for a single scene. A number of accelerated approaches have therefore been proposed [ 40 , 10 , 7 , 29 ] , using various kinds of efficient scene representation techniques. Interestingly, not only are these approaches faster, they also tend to yield better image synthesis results. Other works have focused more directly on improving the quality of the synthesis results [ 62 , 1 , 2 , 3 , 45 ] , by introducing unbounded scene representations or reducing aliasing artifacts. Nerfacto [ 43 ] , introduced in the popular Nerfstudio project, combines many components of these approaches into an integrated one. Hence, we use it as the basis for implementing our own approach. Gaussian Splatting. 3D Gaussian Splatting [ 17 ] has recently emerged as a powerful alternative to NeRFs for novel view synthesis results using an explicit point-based representation. It is arguably better at novel-view synthesis but NeRF-based models continue to shine in many scenarios, offering unique advantages through their continuous representations. This is the case in applications such as surface reconstruction [ 24 ] , 3D scene understanding [ 18 ] and inverse rendering [ 16 ] . In addition, the general regularization method we propose could be potentially used in Gaussian Splatting as well, by simply changing the point sampling method to a Gaussian placement one. Regularizers. A straightforward approach to improving the performance of NeRFs is to incorporate geometric priors to regularize and guide the training process. To this end, many methods have been proposed. They rely on depth guidance [ 8 , 49 , 48 , 34 , 60 ] , geometric constraints [ 30 , 39 , 44 , 19 ] , or pre-training on similar scenes [ 6 , 59 , 50 , 53 , 56 ] . However, these methods are all plagued by generalization issues. For depth priors, it is difficult to obtain accurate depth predictions, especially in real-world unbounded scenes. The geometry-based constraints often fail to properly refine the results in complex unbounded scenes. Prediction-based methods are mostly restricted to a 3-view setting in bounded scenes, due to the limitations of