Splatography: Sparse multi-view dynamic Gaussian Splatting for filmmaking challenges
Adrian Azzarelli, Nantheera Anantrasirichai, David R Bull · 2025
Read Splatography: Sparse multi-view dynamic Gaussian Splatting for filmmaking challenges by Adrian Azzarelli, Nantheera Anantrasirichai, David R Bull (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-05152.txt.
Introduction
Adrian Azzarelli and co-authors present Splatography, a way to rebuild moving 3D scenes for film from few cameras. Dense camera rigs let deformable Gaussian methods make sharp dynamic scenes, but film budgets often mean sparse rigs that miss complex motion.
Their method splits the scene into a foreground and a background part, each with its own Gaussians and warp field, guided by masks from the first frame. Each part trains on its own loss in a first stage, then both move in dynamic training. The split helps sparse views keep fine action clear.
Catalog record
| CREATOR | Adrian Azzarelli, Nantheera Anantrasirichai, David R Bull |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-05152
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Splatography: Sparse multi-view dynamic Gaussian Splatting for filmmaking challenges Adrian Azzarelli, Nantheera Anantrasirichai, David R Bull Abstract Deformable Gaussian Splatting (GS) accomplishes photorealistic dynamic 3-D reconstruction from dense multi-view video (MVV) by learning to deform a canonical GS representation. However, in filmmaking, tight budgets can result in sparse camera configurations, which limits state-of-the-art (SotA) methods when capturing complex dynamic features. To address this issue, we introduce an approach that splits the canonical Gaussians and deformation field into foreground and background components using a sparse set of masks for frames at t = 0 t=0 . Each representation is separately trained on different loss functions during canonical pre-training. Then, during dynamic training, different parameters are modeled for each deformation field following common filmmaking practices. The foreground stage contains diverse dynamic features so changes in color, position and rotation are learned. While, the background containing film-crew and equipment, is typically dimmer and less dynamic so only changes in point position are learned. Experiments on 3-D and 2.5-D entertainment datasets show that our method produces SotA qualitative and quantitative results; up to 3 3 PSNR higher with half the model size on 3-D scenes. Unlike the SotA and without the need for dense mask supervision, our method also produces segmented dynamic reconstructions including transparent and dynamic textures. Code and video comparisons are available online: https://interims-git.github.io/ Figure 1 : Sparse view 3-D reconstruction: Our dynamic representation offers foreground-background separability and high quality 3-D reconstruction without the need for dense mask priors. This paper focuses on filmmaking challenges, including but not limited to sparse view and reflective, transparent and dynamic textures 1 Introduction Gaussian Splatting (GS) is increasingly used for photorealistic dynamic 3-D reconstruction from multi-view video (MVV) data. While GS methods can unlock creative potential in filmmaking applications 7 ; 37 ; 14 ; 12 ; 18 ; 3 , current approaches 27 ; 36 ; 18 ; 10 ; 25 rely on a large number of cameras for robust dynamic reconstruction. This can be impractical for some filmmaking productions, where the number of cameras is limited by filming budgets, or issues with acting challenges arise due to isolating filming spaces designed to minimize background noise 9 ; 19 ; 32 ; 28 . Hence, improving sparse view 3-D reconstruction (SV3D) for dynamic scenes is a key challenge. In sparse MVV configurations, fewer cameras typically result in densely viewed foregrounds but sparsely viewed backgrounds. This conflicts with the workings of state-of-the-art (SotA) approaches that attribute Gaussian point importance based on full-image reconstruction quality 36 ; 27 ; 16 . Thus, backgrounds receive more attention during training due to their larger area, leading to over-reconstruction in backgrounds and under-reconstruction in foregrounds; see Fig. 2 . While this could be resolved by masking out backgrounds for each frame using priors generated with SAM2/MiVOS/etc. 33 ; 8 , the issue of densely masking reflective, transparent and dynamic (RTD) textures (like fire and smoke) has not been resolved. Transparency is especially problematic as disentangling dimming and lighting material effects from a background cannot be achieved with simple binary masks. RTD textures are common in filmmaking, so dense mask priors would limit the use of costumes and props. For SV3D we also find that, even before dynamic training, SotA methods cannot reconstruct robust canonical representations. These methods work by learning to deform a canonical (time-independent) 3-D GS. Despite its importance, robust canonical pre-training has yet to be investigated for enhancing dynamic reconstruction. Consequently, this paper’s primary focus is to tackle SV3D reconstruction challenges by: (1) proposing a new dynamic scene representation that disentangles the foreground and background to deal with point-importance, and (2) developing a new strategy for training canonical representations to deal with poor initialization. Linking this work to filmmaking, we leverage foreground-background separation to apply background-specific constraints based on filmmaking practices; discussed in the following paragraphs. This aims to suppress the reconstruction artifacts associated with background and foreground reconstruction, which we identify in Fig. 2 . We also place emphasis on reconstructing RTD textures as they are more common in filmmaking than most other practices. Figure 2 : Novel views (right) reveal over-reconstructed backgrounds and under-reconstructed foregrounds in SotA The proposed scene representation in Fig. 3 uses only a single mask per view at t=0 to segment the initial point cloud into canonical foreground and background representations. During canonical training, the representations are separately trained using specialized loss functions that rely on the aforementioned masks. The foreground loss discourages floaters and smooth-edge artifacts, and the background loss suppresses view-dependent occlusions and foreground artifacts present along the mask-edge. To model dynamics, we propose representing the foreground and background as separate hex-plane deformation fields, jointly trained without mask supervision. The assumption made is that, during filming, backgrounds are weakly dynamic as the only temporal changes are due to film crew moving around. Color changes are not expected as background lighting is usually static and dim. So, dynamic background features are captured using only point displacement. By doing this, the background is less likely to over-compensate for under-reconstructed foregrounds. Regarding foreground dynamics, we model changes in position, rotation and color via a hex-plane model as we expect the presence of RTD textures. We also modify the temporal opacity function in 27 to better represent deformable textures and introduce regularizers for each relative parameter. Figure 3 : Left: The canonical representation is constructed by masking the initial point cloud and training the foreground and background representations G_{f} and G_{b} on specialized loss functions that minimize over-reconstruction. Right: Dynamic features for G_{f} and G_{b} are jointly trained using the proposed plane-based design. For G_{b} , we only learn motion. For G_{f} , we learn motion, rotation and color change using a novel combination of plane features, and also temporal opacity using an exponential peaking function. Dealing with point densification, prior works rely on cloning and splitting strategies that assess volumetric importance (i.e. where to instance points) based on full image reconstruction quality 22 ; 36 ; 27 . For SV3D, sparsely viewed backgrounds dominate these metrics leading to over-reconstruction in unimportant regions of space. This also leads to temporal bias, as instances in time with greater reconstruction error may produce clones that negatively affect temporal consistency in other views. We consequently leverage the separation of foreground and background representations to limit cloning only for foreground points. To avoid view dependent and temporal bias during densification, we propose a reference-free strategy that samples point motion over time and uses quantile selection to clone the most dynamic points. As this approach is reference-free, it does not require tracking additional point parameters for densification, as in prior works. In summary, this paper aims to resolve current issues when reconstructing dynamic multi-view SV3D content, including RTD textures, via the following contributions: 1. A novel approach to disentangling and separately training canonical foreground and background representations using a sparse set of mask; including the proposal of representation-specific loss functions 2. A new formulation for dynamic representations that treat foregrounds and backgrounds differently by limiting the expressiveness of background features. By also limiting the number of background Gaussians being modeled, this encourages the model to focus more on the foreground representation 3. A reference-free and minimally invasive approach