LoD-Structured 3D Gaussian Splatting for Streaming Video Reconstruction
Xinhui Liu, Can Wang, Lei Liu, Zhenghao Chen, Wei Jiang, Wei Wang, Dong Xu · 2026
Read LoD-Structured 3D Gaussian Splatting for Streaming Video Reconstruction by Xinhui Liu, Can Wang, Lei Liu, Zhenghao Chen, Wei Jiang, Wei Wang, Dong Xu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18475.txt.
Introduction
Xinhui Liu and six colleagues work on free-viewpoint video, where a viewer can move around a scene as it plays. To make that work, a system must rebuild the scene in three dims from a few cams. Live streams add hard caps: the build must be fast, the files must be small, and the view must still look real.
Their plan builds the scene from soft blobs of light, a trick famed for its speed. They stack those blobs in grades, from coarse to fine, so the stream can send just what each viewer needs. Fine grades can ride on top when the link is strong. The aim is smooth, sharp views on live streams with few cams. Such tech could serve sports, shows, and calls where users roam at will.
Catalog record
| CREATOR | Xinhui Liu, Can Wang, Lei Liu, Zhenghao Chen, Wei Jiang, Wei Wang, Dong Xu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-18475
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LoD-Structured 3D Gaussian Splatting for Streaming Video Reconstruction Xinhui Liu, Can Wang, Lei Liu, Zhenghao Chen, Wei Jiang, Wei Wang, Dong Xu Abstract Free-Viewpoint Video (FVV) reconstruction enables photorealistic and interactive 3D scene visualization; however, real-time streaming is often bottlenecked by sparse-view inputs, prohibitive training costs, and bandwidth constraints. While recent 3D Gaussian Splatting (3DGS) has advanced FVV due to its superior rendering speed, Streaming Free-Viewpoint Video (SFVV) introduces additional demands for rapid optimization, high-fidelity reconstruction under sparse constraints, and minimal storage footprints. To bridge this gap, we propose StreamLoD-GS, an LoD-based Gaussian Splatting framework designed specifically for SFVV. Our approach integrates three core innovations: 1) an Anchor- and Octree-based LoD-structured 3DGS with a hierarchical Gaussian dropout technique to ensure efficient and stable optimization while maintaining high-quality rendering; 2) a GMM-based motion partitioning mechanism that separates dynamic and static content, refining dynamic regions while preserving background stability; and 3) a quantized residual refinement framework that significantly reduces storage requirements without compromising visual fidelity. Extensive experiments demonstrate that StreamLoD-GS achieves competitive or state-of-the-art performance in terms of quality, efficiency, and storage. Fig. 1: StreamLoD-GS enables fast, on-the-fly, high-fidelity reconstruction for Streaming Free-Viewpoint Video. Right : Comparison with existing methods with 5 Training Views on Meet Room [ 22 ] dataset. I Introduction Constructing Free-Viewpoint Videos (FVVs) from multi-view images promise to revolutionize visual media by bridging the gap between our perception of the dynamic 3D world and the constraints of conventional 2D video [ 6 ] . Recently, 3D Gaussian Splatting (3DGS) [ 17 ] has emerged as a cornerstone for FVV reconstruction [ 9 , 7 , 6 , 58 , 45 , 19 , 50 ] , primarily due to its capacity for photorealistic and real-time synthesis. Due to the increasing demand for real-time interactivity and immersive experiences, streaming Free-Viewpoint Video (SFVV) [ 44 , 12 , 13 , 52 , 14 , 15 ] has emerged as a focal point of research, offering dynamic, real-time viewing experiences that transcend traditional media boundaries. However, SFVV introduces new challenges and demands for 3DGS, including: 1) Fast optimization and rendering: the ability to rapidly optimization and render novel views in real-time; 2) High fidelity even with sparse views: ensuring the visual quality of rendered scenes, even when only limited camera setups are available, as capturing and transmitting dense views in real-world environments is often logistically and financially prohibitive; 3) low storage requirements for transport (bandwidth): minimizing the storage of 3DGS points, to optimize bandwidth usage. To address these challenges, 3DGStream [ 44 ] pioneered the extension of 3DGS to SFVV by leveraging Instant-NGP [ 30 ] , which combines neural network representations and multi-resolution hashing to accelerate data querying and rendering. Beyond representation, other contemporary approaches [ 52 , 13 , 12 , 15 ] have shifted focus toward motion field modeling, utilizing residual learning [ 48 ] to isolate dynamic regions and reduce optimization costs. Despite these advancements, most 3DGS-based frameworks [ 44 , 12 , 13 , 52 ] still depend on dense, synchronized camera arrays to provide sufficient views for high-quality reconstruction. In practice, however, factors such as bandwidth constraints and spatial limitations often restrict systems to only a few cameras, hindering their applicability in real-world scenarios. When attempting SFVV reconstruction under sparse training views, two primary challenges emerge. First, as the number of input views decreases ( e.g. , 7, 5, 4, or fewer), weakened multi-view consistency causes the model to overfit to observed views, leading to artifacts in novel-view synthesis. Second, sparse inputs often trigger the production of redundant or misplaced Gaussians during the densification process [ 34 , 51 ] , which inflates storage and transmission overhead that are particularly critical for mobile and low-end devices with limited memory and bandwidth. Drawing inspiration from Level-of-Detail (LoD) hierarchies [ 28 , 38 , 41 , 40 , 20 ] , which effectively reduce geometric redundancy while enhancing rendering efficiency, we propose StreamLoD-GS, an LoD-based Gaussian Splatting framework tailored for SFVV. Our approach comprises three core innovations: First, we introduce an LoD-structured 3DGS with Anchor and Octree representations. To prevent overfitting and facilitate stable optimization, we introduce a hierarchical Gaussian dropout technique. This strategy dynamically selects detail levels based on viewing distance and employs level-aware stochastic dropout during training. Ultimately, this design mitigates Sparse-View overfitting while preserving the gradient flow essential for robust convergence. Second, leveraging the inherent temporal coherence in video sequences, we develop a Gaussian Mixture Models [ 35 ] (GMM)-based motion partitioning mechanism. This module intelligently identifies dynamic anchors for selective refinement while freezing static regions, thereby exploiting background stability and motion continuity to significantly reduce redundant computations. Finally, to enable bandwidth-efficient streaming, we incorporate a quantized residual refinement framework. This design bridges static frozen anchors and dynamically updated regions through quantized residuals, drastically minimizing storage and transmission footprints without sacrificing visual fidelity. Our major contributions are summarized as follows. • We propose an Anchor- and Octree-based LoD-structured 3DGS representation, integrated with a hierarchical Gaussian dropout technique to stabilize training and ensure high-fidelity synthesis in Sparse-View SFVV. • We introduce a GMM-driven mechanism to decouple dynamic and static content, enabling targeted refinement of moving regions while maintaining the structural stability of the background. • We develop a quantized residual refinement framework that compresses dynamic updates, facilitating efficient data transmission and storage for low-bandwidth environments. • Extensive experiments demonstrate that StreamLoD-GS achieves competitive or state-of-the-art performance in terms of quality, efficiency, and storage footprints compared to contemporary SFVV approaches. II Related Works II-A Streaming 3DGS Constructing streamable dynamic scenes via on-the-fly, frame-by-frame training presents significant challenges compared to offline learning from complete multi-view videos. Initial efforts adapted NeRFs [ 29 , 1 , 2 , 5 , 11 , 30 ] to this task. For instance, StreamRF [ 22 ] utilized an incremental learning paradigm by modeling per-frame differences, while ReRF [ 48 ] modeled residual information between adjacent frames for long-duration scenes. NeRFPlayer [ 43 ] , on the other hand, decomposed the 4D spatio-temporal space to optimize for model compactness and reconstruction speed. The advent of 3DGS marked a major leap forward, offering superior training efficiency and rendering speeds. Building on this, streaming 3DGS is designed to reconstruct dynamic scenes through on-the-fly, frame-by-frame training, enabling real-time updates and adjustments as the scene evolves [ 44 , 12 , 52 , 24 , 13 , 15 , 45 ] . 3DGStream [ 44 ] proposed a Neural Transformation Cache with multi-resolution hashing to advance 3DGS modeling. Other methods have focused on advanced motion modeling, such as HiCoM [ 12 ] with its hierarchical coherent motion mechanism and IGS [ 52 ] , which uses an anchor-driven network to learn motion residuals in a single step for faster training. To enhance model expressiveness and tackle data size, QUEEN [ 13 ] proposed a framework based on Gaussian residuals combined with learned quantization and sparsity for compression. To optimize the rate-distortion trade-off for SFVV, 4DGC [ 15 ] introduces a rate-aware compression framework with motion-grid-based representation and differentiable quantization. Despite advancements in fidelity and efficiency, a common bottleneck remains: these methods require dense, multi-view images at every frame for 3D reconstruction. This dependence on synchronized multi-camera