From Volume Rendering to 3D Gaussian Splatting: Theory and Applications
Vitor Pereira Matias * 4 , Daniel Perazzo * 2 , Vinicius Silva 3 , Alberto Raposo 3, Luiz Velho 2 , Afonso Paiva 4 , and Tiago Novello 2 · 2025
Read From Volume Rendering to 3D Gaussian Splatting: Theory and Applications by Vitor Pereira Matias * 4 , Daniel Perazzo * 2 , Vinicius Silva 3 , Alberto Raposo 3, Luiz Velho 2 , Afonso Paiva 4 , and Tiago Novello 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18101.txt.
Introduction
Vitor Pereira Matias, with six co-authors, surveys a fast-moving shift in three-dimensional graphics toward Gaussian splatting. Scenes are stored as clouds of soft three-dimensional blobs that render quickly and plug into normal graphics pipelines. The price is heavy memory use, lighting baked into the scene, and weak support for effects that need extra rays.
The tutorial walks the full pipeline from its math roots, then reviews fixes for those limits. Uses covered include surface rebuilding, avatars, animation, and content creation. Graphics readers get one concise tour of where this real-time method stands and what still holds it back.
Catalog record
| CREATOR | Vitor Pereira Matias * 4 , Daniel Perazzo * 2 , Vinicius Silva 3 , Alberto Raposo 3, Luiz Velho 2 , Afonso Paiva 4 , and Tiago Novello 2 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-18101
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Volume Rendering to 3D Gaussian Splatting: Theory and Applications Vitor Pereira Matias * 4 , Daniel Perazzo * 2 , Vinicius Silva 3 , Alberto Raposo 3, Luiz Velho 2 , Afonso Paiva 4 , and Tiago Novello 2 Abstract The problem of 3D reconstruction from posed images is undergoing a fundamental transformation, driven by continuous advances in 3D Gaussian Splatting (3DGS). By modeling scenes explicitly as collections of 3D Gaussians, 3DGS enables efficient rasterization through volumetric splatting, offering thus a seamless integration with common graphics pipelines. Despite its real-time rendering capabilities for novel view synthesis, 3DGS suffers from a high memory footprint, the tendency to bake lighting effects directly into its representation, and limited support for secondary-ray effects. This tutorial provides a concise yet comprehensive overview of the 3DGS pipeline, starting from its splatting formulation and then exploring the main efforts in addressing its limitations. Finally, we survey a range of applications that leverage 3DGS for surface reconstruction, avatar modeling, animation, and content generation—highlighting its efficient rendering and suitability for feed-forward pipelines. * * footnotetext: denotes equal contribution. I Introduction 3D reconstruction from posed images is a long-standing problem in visual computing that is undergoing a fundamental disruption, driven by advances in Neural Radiance Fields (NeRFs) [ 1 ] and 3D Gaussian Splatting (3DGS) [ 2 ] . Given a set of input views with known poses from an unknown scene, the goal is to optimize the parameters of a 3D representation that accurately captures the scene’s geometry and appearance. NeRFs have had a significant impact on this task by representing scene geometry (as volume density) and radiance using neural networks, optimized through differentiable volume rendering [ 3 ] . This framework enables highly detailed Novel View Synthesis (NVS) of real-world scenes, with straightforward integration of 3D reconstruction into deep learning pipelines. However, representing a scene volumetrically through a neural network can be inefficient, as the network must learn to represent both occupied and empty regions of the domain. During training, its global representation requires supervision across the entire domain, including empty space, resulting in high computational costs and making real-time rendering impractical–crucial for applications. To address these challenges, Kerbl et al. [ 2 ] introduced 3DGS, which models the scene as a collection of 3D Gaussians with colors and leverages volume splatting [ 4 ] for differentiable rendering. By avoiding costly queries in empty space and employing rasterization, 3DGS achieves both high detail and real-time performance. Figure 1 gives an overview of 3DGS. Fig. 1: Illustration of the 3DGS pipeline. Given a set of posed input images (left), a sparse point cloud from Structure-from-Motion (SfM) is used to initialize a set of colored 3D Gaussians (middle). These Gaussians are then optimized via volume splatting and can support downstream tasks such as novel view synthesis and surface reconstruction (right). The objective of this tutorial is to present the 3DGS method by deriving its splatting formulation from the volume rendering equation, along with techniques for Gaussian initialization and adaptation during training. We then review recent approaches that address key limitations of the original 3DGS method, including its high memory consumption and limited support for secondary-ray effects. Finally, we discuss a range of applications of 3DGS, including surface reconstruction, animation, avatar modeling, and feed-forward 3D reconstruction from sparse views. In summary, our contributions are: • An intuitive mathematical derivation of 3DGS from the volume rendering equation. • A survey of 3DGS extensions and applications across a variety of 3D reconstruction tasks. II Volume Rendering Preliminaries Our objective is to use differentiable rendering to reconstruct a 3D scene—parameterized by \theta —from a set of N posed multi-view images and their corresponding camera intrinsics and extrinsics \{\mathbf{I}_{i},\mathbf{K}_{i},\mathbf{W}_{i}\}_{i=1}^{N} . For each view i , we render the scene using the current \theta , obtaining an image \mathbf{I}_{\theta,i} , which is then compared to its corresponding input view \mathbf{I}_{i} , enforcing \mathbf{I}_{i}\approx\mathbf{I}_{\theta,i} . This is achieved by minimizing the photometric loss: \displaystyle\mathcal{L}(\theta)=\frac{1}{N}\sum_{i=1}^{N}\left\|\mathbf{I}_{\theta,i}-\mathbf{I}_{i}\right\|^{2}, (1) using standard gradient descent algorithms such as Adam [ 5 ] . Traditional mesh-based representations are typically non-differentiable, making them unsuitable for optimizing loss ( 1 ) via gradient descent methods. To overcome this, NeRFs [ 1 ] and 3DGS [ 2 ] employed volumetric representations to enable differentiable volume rendering. Specifically, the scene is parameterized by a density field \sigma_{\theta}:\mathbb{R}^{3}\to\mathbb{R} , quantifying particle light absorption, and a color field c_{\theta}:\mathbb{R}^{3}\to\mathbb{R} , quantifying emitted radiance. Thus, treating the scene as a “cloud” that both absorbs and emits light. Although seemingly unsuitable for representing solid objects initially, this approach has yielded strong results in novel view synthesis [ 6 , 7 ] and detailed surface reconstruction [ 8 , 9 ] . Let \gamma(t)=\mathbf{o}_{i}+t\mathbf{v} be the view ray associated with a pixel \mathbf{p} in the input image \mathbf{I}_{i} , where \mathbf{o}_{i} is the camera position. Our goal is to compute the color observed along this ray using volume rendering techniques [ 10 ] . We restrict the ray domain to the interval [t_{n},t_{f}] , where t_{n} and t_{f} correspond to the near and far bounds. Ignoring scattering, the accumulated radiance I(\gamma(t)) along the ray satisfies the volumetric light transport ordinary differential equation (ODE) [ 3 ] : \displaystyle I^{\prime}(t)=\underbrace{\sigma_{\theta}(t)c_{\theta}(t)}_{\text{Emission}}-\underbrace{\sigma_{\theta}(t)I(t)}_{\text{Absorption}}. (2) For simplicity, we omit \gamma in the notation. The ODE ( 2 ) can be solved by separation of variables [ 3 , Sec. 4] . Assuming zero background emission as the initial condition, i.e., I(t_{n})=0 , the accumulated radiance at the end of the ray is given by: \displaystyle I_{f}:=I(t_{f})=\int_{t_{n}}^{t_{f}}\sigma_{\theta}(t)c_{\theta}(t)\exp\left(-\int_{t_{n}}^{t}\sigma_{\theta}(s)\,ds\right)dt. (3) The transmittance \exp(\cdots) captures the attenuation of light due to absorption along the ray. In practice, to render the predicted image \mathbf{I}_{\theta,i} for view i , we approximate the integral ( 3 ) using numerical quadrature methods resulting in the volume rendering equation [ 1 ] : \displaystyle\!\!\!I_{f}\!\approx\!\!\sum_{i=1}^{N}\!c_{\theta}(t_{i})\!\Big(1\!-\!\exp\!\big(\!-\sigma_{\theta}(t_{i})\delta_{i}\big)\Big)\!\prod_{j=1}^{i-1}\!\exp\!\big(\!-\sigma_{\theta}(t_{j})\delta_{j}\big), (4) where \delta_{i}=t_{i}-t_{i-1} is the step size between consecutive samples. This formulation allows gradient-based optimization, as it is fully differentiable. In NeRF, the functions \sigma_{\theta} and c_{\theta} are modeled by neural networks, and ( 4 ) is used to optimize \theta via the photometric loss ( 1 ). However, the global nature of neural representations requires supervision across the entire domain—including empty space—to compute ( 4 ). This results in high computational overhead and severely limits real-time rendering capabilities, essential for applications. III 3D Gaussian Splatting Gaussian splatting overview . To overcome the computational cost of evaluating the volume rendering equation ( 3 ), Kerbl et al. [ 2 ] introduced 3D Gaussian Splatting (3DGS), which represents the density and radiance fields \sigma_{\theta} and c_{\theta} using a collection of colored 3D Gaussians, rendered efficiently via volume splatting [ 4 ] . We follow the pipeline illustrated in Fig. 2 to describe the main stages of 3DGS. The input consists of a set of posed images \{\mathbf{I}_{i}\} , from which a colored point cloud is obtained via Structure-from-Motion (SfM) [ 11 ] ; see Fig. 2 (top-left). This point cloud serves as the basis for initializing a set of M Gaussians, g_{i}:=\{\boldsymbol{\mu}_{i},\boldsymbol{\Sigma}_{i},\sigma_{i},c_{i}\}, where \boldsymbol{\mu}_{i}\in\mathbb{R}^{3} is the Gaussian center, \boldsymbol{\Sigma}_{i}\in\mathbb{R}^{3\times 3} is the covariance matrix, \sigma_{i}\in\mathbb{R} is the opacity, and c_{i}\in\mathbb{R}^{3} is the RGB color. Each Gaussian defines a density function \mathcal{G}_{i}\!:\!\mathbb{R}^{3}\!\rightarrow\!\mathbb{R} given by: