Category-level Meta-learned NeRF Priors for Efficient Object Mapping
Saad Ejaz, Hriday Bavle, Laura Ribeiro, Holger Voos, Jose Luis Sanchez-Lopez · 2025
Read Category-level Meta-learned NeRF Priors for Efficient Object Mapping by Saad Ejaz, Hriday Bavle, Laura Ribeiro, Holger Voos, Jose Luis Sanchez-Lopez (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01582.txt.
Introduction
Saad Ejaz, Hriday Bavle, Laura Ribeiro, Holger Voos, and Jose Luis Sanchez-Lopez present PRENOM, a system that maps objects in three dimensions quickly. It leans on what it already knows about each object's category. A robot that knows chairs in general can reconstruct a new chair from fewer views. Older category priors blur sharp edges and cost heavy computation, while detailed neural rendering had not been tied to category knowledge.
PRENOM meta-learns from synthetic practice tasks built from open shape collections. It then fits a small neural rendering model to each real object. That yields fine detail plus an estimate of the object's canonical pose. The system runs in a multi-object, real-time setting. Efficient object maps serve robots that must grasp and navigate.
Catalog record
| CREATOR | Saad Ejaz, Hriday Bavle, Laura Ribeiro, Holger Voos, Jose Luis Sanchez-Lopez |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01582
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Category-level Meta-learned NeRF Priors for Efficient Object Mapping Saad Ejaz, Hriday Bavle, Laura Ribeiro, Holger Voos, Jose Luis Sanchez-Lopez Abstract In 3D object mapping, category-level priors enable efficient object reconstruction and canonical pose estimation, requiring only a single prior per semantic category (e.g., chair, book, laptop, etc.). DeepSDF has been used predominantly as a category-level shape prior, but it struggles to reconstruct sharp geometry and is computationally expensive. In contrast, NeRFs capture fine details but have yet to be effectively integrated with category-level priors in a real-time multi-object mapping framework. To bridge this gap, we introduce PRENOM, a Prior-based Efficient Neural Object Mapper that integrates category-level priors with object-level NeRFs to enhance reconstruction efficiency and enable canonical object pose estimation. PRENOM gets to know objects on a first-name basis by meta-learning on synthetic reconstruction tasks generated from open-source shape datasets. To account for object category variations, it employs a multi-objective genetic algorithm to optimize the NeRF architecture for each category, balancing reconstruction quality and training time. Additionally, prior-based probabilistic ray sampling directs sampling toward expected object regions, accelerating convergence and improving reconstruction quality under constrained resources. Experimental results highlight the ability of PRENOM to achieve high-quality reconstructions while maintaining computational feasibility. Specifically, comparisons with prior-free NeRF-based approaches on a synthetic dataset show a 21% lower Chamfer distance. Furthermore, evaluations against other approaches using shape priors on a noisy real-world dataset indicate a 13% improvement averaged across all reconstruction metrics, and comparable pose and size estimation accuracy, while being trained for 5 × \times less time. Code available at: https://github.com/snt-arg/PRENOM I Introduction Object-level mapping has gained significant attention for its ability to separate objects from cluttered environments for high-level robotic tasks. Recent works use compact primitives such as cuboids [ 1 ] and quadrics [ 2 ] to approximate object bounds, while reconstructing dense geometry using Signed Distance Fields (SDFs) [ 3 , 4 ] , Neural Radiance Fields (NeRFs) [ 5 , 6 ] , and Gaussian splats [ 7 ] . Typical methods operate without explicit priors and treat all objects uniformly, overlooking object semantics. In contrast, 3D priors capture common geometric properties, providing valuable cues for reconstruction while also enabling canonical pose estimation of objects. Fig. 1 : PRENOM uses category-level priors to accelerate object-level NeRF training and localize objects in a canonical frame. Top left shows priors loaded during mapping (displayed all at once for illustration). PRENOM operates online, training objects in parallel while initializing them sequentially as they are observed. The bottom rows demonstrates how priors provide a reasonable initial estimate, enabling faster and more accurate reconstruction. Time indicates dedicated GPU training time. These priors can be broadly classified into three categories. Instance-level priors [ 8 , 9 ] offer high accuracy, but require knowledge of the environment, through precise CAD models of each object instance, limiting their applicability to specialized use cases. On the other end, generalized priors [ 10 ] use a single prior for all objects but rely on bulky diffusion models, making them computationally intensive even for a single object. Category-level priors are a suitable trade-off as they rely solely on semantic labels, and thus can work with unseen objects within predefined categories of interest. Recent works represent such priors using variational autoencoders [ 11 ] , shape embeddings [ 12 ] , and DeepSDFs [ 13 , 14 , 15 ] . While these methods offer reasonable global shape and pose estimation, they are computationally demanding, requiring high-end GPUs; use identical prior designs for all categories, ignoring differences in shape complexity; and in most scenarios struggle with reconstructing sharp geometry. To address these limitations, we present PRENOM, a Prior-based Efficient Neural Object Mapper that represents objects by lightweight NeRF models, with meta-learned [ 16 ] parameter initializations as category-level priors, within a real-time RGB-D multi-object mapping framework. Having been trained on large shape datasets, these priors can localize objects within a canonical frame while also benefiting from the high-fidelity reconstructions typically associated with NeRFs. Further integrating them with neural architectures optimized per-category and prior-based probabilistic ray sampling, we achieve faster training convergence (see Figure 1 ), enabling accurate reconstructions with limited GPU resources. The primary contributions of this paper are as follows: • An online object mapping framework that uses lightweight object-level NeRFs with category-level meta-learned priors to enable accurate object reconstruction on resource-constrained systems. • A pipeline for optimizing NeRF architectures per object category to achieve a balance between reconstruction performance and training speed. • Integration of prior-based probabilistic ray sampling in object-level NeRFs for improved convergence leading to better reconstruction quality. II Related Works II-A 3D Object Mapping Object-level mapping has been extensively explored in the context of SLAM, where camera poses and object states are jointly estimated by enforcing multi-view consistency [ 1 , 2 , 13 , 11 ] . Alternatively, it has also been studied independently, taking advantage of known camera poses to focus purely on object representation and reconstruction [ 12 , 3 , 17 , 5 , 6 , 15 , 10 ] . While the former approach prioritizes simple, optimizable object representations to support joint estimation, the latter decouples localization and mapping, shifting the focus on improving reconstruction fidelity. To achieve this, implicit neural object representations, such as object-level SDFs [ 3 , 4 ] and object-level NeRFs [ 18 , 17 , 5 , 6 ] , have been widely adopted to generate high-quality meshes. Our approach falls in this latter category. Considering only object representation, RO-MAP [ 6 ] and [ 17 ] are the closest to our approach; however, their reliance on random weight initialization and uniform ray sampling results in slow training, requiring a high-end GPU for real-time multi-object mapping. Moreover, scale ambiguity in the monocular RO-MAP results in noisy object pose estimation, which is why we opt for an RGB-D pipeline. II-B Category-level Priors for Object Mapping NodeSLAM [ 11 ] employs variational autoencoders as category-level shape priors to predict occupancy grids, jointly estimate the shape and pose of objects. However, it struggles with reconstructing complex shapes. Frodo [ 12 ] represents shape priors using encoder-decoder networks to model objects as point clouds and SDFs. Unlike our sequential method, it operates in batch mode. Several works [ 13 , 14 , 15 ] adopt the DeepSDF architecture [ 19 ] as a shape prior, learning latent embeddings as shape priors, and training decoder networks to predict object SDFs. These approaches require long training times of several seconds per object on a low-end GPU while having limited capability to reconstruct fine geometry. Moreover, these methods also jointly estimating the object pose with shape reconstruction. While this can refine object localization, it increases training time and makes reconstruction accuracy dependent on state estimation quality. In contrast, our approach follows [ 11 ] by estimating the object pose, before initializing reconstruction. However, unlike the CNNs used by [ 11 ] , we rely solely on our priors to estimate and refine the pose. III Methodology Fig. 2 : Overview of the two key components of PRENOM. Our proposed methodology is composed of two key modules as can be seen in Figure 2 . The offline prior trainer ( Section III-A )