MLP Splatting: Object-Centric Neural Fields

Shinjeong Kim *, Yuzhou Cheng *, Xin Kong *, Paul H. J. Kelly, Andrew J. Davison · 2026

Read MLP Splatting: Object-Centric Neural Fields by Shinjeong Kim *, Yuzhou Cheng *, Xin Kong *, Paul H. J. Kelly, Andrew J. Davison (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03877.txt.

Introduction

Kim and co-authors split scenes into object-level neural primitives with MLP Splatting. Photorealistic view synthesis is strong in NeRF and Gaussian Splatting. Splitting a scene into objects still needs extra segmentation or grouping there. MLP Splatting uses a few expressive light-field primitives instead. Each primitive is a compact MLP with local spatial support. It predicts radiance and opacity on its own.

Rendering composites ray-primitive overlaps sparsely and fast. RGB supervision alone trains the primitives. They settle onto local regions that often match objects or parts. Picking a handful of primitives enables object-level editing with no masks. Optional semantic distillation adds open-vocabulary interaction and instant segmentation. Memory use falls to about one seventh of rival semantic 3DGS methods. Rendering runs five times faster. Scene editors get object control without giving up photo quality.

Catalog record

CREATORShinjeong Kim *, Yuzhou Cheng *, Xin Kong *, Paul H. J. Kelly, Andrew J. Davison
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2606.03877

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MLP Splatting: Object-Centric Neural Fields Shinjeong Kim *, Yuzhou Cheng *, Xin Kong *, Paul H. J. Kelly, Andrew J. Davison Abstract 3D representations are fundamental to scene rendering, understanding, and interaction. Recent approaches, such as 3D Gaussian Splatting and Neural Radiance Fields, achieve impressive photorealistic novel-view synthesis, but lack the ability to easily decompose scene elements into a few primitives, requiring additional segmentation or grouping for object-level manipulation. We present MLP-Splatting, a method that enables scene decomposition via a few expressive light-field primitives while providing photorealistic novel-view synthesis. MLP-Splatting models each primitive as an independent compact MLP with localized spatial support that predicts radiance and opacity. In contrast to low-level Gaussian primitives or a single global radiance field, our neural primitives provide greater expressive capacity while remaining spatially localized. Rendering is performed through efficient sparse volumetric compositing over ray–primitive interactions. Our primitives are supervised using RGB supervision alone, which yields primitives that represent local scene regions often corresponding to objects or object parts, enabling interactive object-level editing without segmentation masks by selecting a handful of primitives. Our method, augmented with optional semantic feature distillation, enables open-vocabulary scene interaction and open-set instant segmentation. Compared to state-of-the-art methods, we achieve substantially lower memory usage (1/15 × \times ) and faster rendering (3 × \times ), as we show in our experiments compared to semantic 3DGS methods. Project Page: https://shinjeongkim.com/mlp-splatting * * footnotetext: Equal contribution. 1 Introduction Reconstructing a 3D scene remains a fundamental and enduring problem in computer vision, closely tied to the scene representation. Different application goals have led to diverse primitives, ranging from discrete point clouds featuring ease of obtaining and processing, point-based primitives such as 3D Gaussian Splatting (3DGS) [ 13 ] for editability, mesh representations [ 22 ] tailored for watertight modeling and physical simulation, and neural radiance fields (NeRF) [ 26 ] for photorealistic novel-view synthesis. (a) (b) Figure 1 : MLP-Splatting Overview. (a) Conceptual comparison of 3D scene representations along neurality, atomicity, and semantic awareness. Our method combines the expressiveness of neural fields with the atomicity of primitives, enabling representations that better align with semantic structure. (b) Example scene and learned primitives corresponding to objects or object parts. From another perspective, scene representations differ fundamentally in the level of abstraction of their primitives. Although many existing representations operate at the level of low-level geometric or radiometric elements, a natural question arises: should higher-level entities, such as objects or object parts, serve as the basic functional units of the representation? In this work, we revisit this question from an object-level perspective. Most existing scene representations are not inherently object-centric: their basic primitives correspond to low-level geometric [ 22 , 9 , 24 , 5 , 13 ] or radiometric elements [ 26 , 27 ] , while objects emerge only as derived groupings or semantic overlays. By object-centric, we mean that the representation’s fundamental functional units correspond to objects or meaningful parts of objects. Although recent advances have begun to incorporate object awareness into neural scene representations, this awareness is typically introduced by distilling semantic knowledge onto pre-existing representations, such as implicit radiance fields in NeRF-based methods [ 17 , 44 , 20 , 15 ] or 3DGS [ 42 , 46 , 4 ] . In both cases, semantics are imposed on representations originally designed for photometric realism rather than arising from representations whose primitive units are themselves object- or part-level entities. Yet we argue that the structure of real scenes itself naturally gives rise to object- and part-level organization, and that a scene representation can be designed to capture this structure directly. Many object-level properties in 3D space—such as spatial occupancy, color, and density—exhibit strong coherence within an object, while sharp transitions are concentrated primarily near object boundaries, despite potentially high intrinsic complexity. We therefore introduce MLP-Splatting, which assigns independent MLP-based primitives to spatially significant parts of objects. These primitives collectively form a scene-level radiance field for rendering. Each primitive is continuous and differentiable, shares no parameters with the others, and in principle has unbounded expressive capacity, making it well-suited to modeling coherent object-level structure. To showcase this object-centric property more clearly, we optionally augment the representation with semantic guidance through feature distillation [ 17 , 46 ] . This allows the learned primitives and their associated embeddings to support tasks such as segmentation and language-guided editing. With this simple supervision, MLP-Splatting achieves semantic metrics comparable to Feature-3DGS [ 46 ] , despite exhibiting orders-of-magnitude differences in model memory due to the nature of the representation. We further analyze the theoretical memory bound required to reach the same photometric error. In summary, our contributions are fourfold: 1. Object-centric neural scene representation. We introduce MLP-Splatting , a neural scene representation in which each primitive is modeled as an independent perceptron. This formulation treats objects or parts of objects as the fundamental functional units of the representation. 2. Emergent object-level structure. Under pure RGB supervision, primitives naturally specialize to coherent scene regions, enabling object-level editing without segmentation . With feature distillation, we achieve state-of-the-art rendering quality and competitive semantics in the Replica [ 36 ] and ScanNet [ 6 ] datasets. 3. Efficient neural-primitive rendering. Our scene paradigm and efficient tile-based compositing pipeline enable real-time rendering at \sim 8 FPS (vs. \sim 3 FPS in [ 46 ] ) with only 32 MB of memory (vs. 630 MB in [ 46 ] ). 4. Theoretical capacity analysis. We derive the Equal-Quality Memory Law , showing that achieving the same pixel error requires \Omega(\varepsilon^{-2}) parameters for Gaussian splatting but only \Theta(\varepsilon^{-3/s}) for MLP-Splatting, where s is the local smoothness order of the scenes. 2 Related Work 2.1 3D Scene Representations NeRFs [ 26 ] represent a scene as a continuous function that maps 3D positions and viewing directions to densities and radiance, optimized by differentiating the volumetric rendering of posed images. A large body of work has since improved the fidelity–efficiency trade-off [ 3 , 33 , 8 , 27 , 2 ] and broadened the operating regime [ 25 , 1 ] . Instant-NGP [ 27 , 37 ] introduces multiresolution hash encodings and fused CUDA kernels to accelerate training and inference drastically. For large, unbounded outdoor captures, mip-NeRF 360 [ 1 ] addresses aliasing and scale imbalance via cone-based integration and scene parameterization, while Zip-NeRF [ 2 ] further improves anti-aliasing by combining rendering/signal-processing ideas with grid-based representations. In parallel, methods based on optimizable explicit primitives, such as 3D/2D Gaussian Splatting [ 13 , 12 ] , ellipsoid rendering [ 24 , 47 ] , polyhedral element rendering [ 9 , 38 ] , etc., replace dense ray marching on a volumetric neural field with a fast visibility-aware rasterization or efficient closed-form volumetric rendering on a set of independently-optimizable unit geometric primitives, achieving high-quality real-time rendering. Our work seeks to combine the representational flexibility of neural fields with the computational efficiency of splatting-style pipelines, while pursuing a distinct representational goal: making object- and part-level structure emerge naturally from the scene representation itself. 2.2 Object-level Scene Representations A growing line of research seeks representations that explicitly expose objects for editing, tracking, and compositional reasoning. iLabel [ 45 ] revealed that a single MLP trained for scene

Read the complete text