GSStream: 3D Gaussian Splatting based Volumetric Scene Streaming System
Zhiye Tang, Qiudan Zhang, Lei Zhang, Junhui Hou, You Yang, Xu Wang · 2026
Read GSStream: 3D Gaussian Splatting based Volumetric Scene Streaming System by Zhiye Tang, Qiudan Zhang, Lei Zhang, Junhui Hou, You Yang, Xu Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-09718.txt.
Introduction
Zhiye Tang and coauthors build GSStream, a system for streaming volumetric scenes built from 3D Gaussian splatting over limited networks. One part forecasts where each viewer will look next, learning from the viewing paths of many users. A second part, trained by reinforcement learning, picks stream quality as network states change. Tests show better picture quality and leaner network use than rival systems.
Catalog record
| CREATOR | Zhiye Tang, Qiudan Zhang, Lei Zhang, Junhui Hou, You Yang, Xu Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67902 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.09718
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GSStream: 3D Gaussian Splatting based Volumetric Scene Streaming System Zhiye Tang, Qiudan Zhang, Lei Zhang, Junhui Hou, You Yang, Xu Wang Abstract Recently, the 3D Gaussian splatting (3DGS) technique for real-time radiance field rendering has revolutionized the field of volumetric scene representation, providing users with an immersive experience. But in return, it also poses a large amount of data volume, which is extremely bandwidth-intensive. Cutting-edge researchers have tried to introduce different approaches and construct multiple variants for 3DGS to obtain a more compact scene representation, but it is still challenging for real-time distribution. In this paper, we propose GSStream, a novel volumetric scene streaming system to support 3DGS data format. Specifically, GSStream integrates a collaborative viewport prediction module to better predict users’ future behaviors by learning collaborative priors and historical priors from multiple users and users’ viewport sequences and a deep reinforcement learning (DRL)-based bitrate adaptation module to tackle the state and action space variability challenge of the bitrate adaptation problem, achieving efficient volumetric scene delivery. Besides, we first build a user viewport trajectory dataset for volumetric scenes to support the training and streaming simulation. Extensive experiments prove that our proposed GSStream system outperforms existing representative volumetric scene streaming systems in visual quality and network usage. Demo video: https://youtu.be/3WEe8PN8yvA . I Introduction Lately, 3DGS has significantly transformed the landscape of volumetric scene representation, enabling the achievement of exceptionally high-fidelity real-time rendering [ 10 ] . 3DGS explicitly models a volumetric scene through the aggregation of numerous 3D Gaussians, offering enhanced flexibility and detail compared to traditional representations. However, this substantial advantage in rendering quality is accompanied by a notable increase in data volume, which poses challenges for practical applications, particularly in scenarios requiring efficient storage and transmission [ 1 ] . To mitigate this issue, extensive research efforts have been directed towards optimizing various aspects of the 3DGS framework. In a recent advancement, Lu et al. introduced Scaffold-GS [ 22 ] , which innovatively employs learnable anchors instead of directly training on individual 3D Gaussians. In this approach, each anchor is initially derived from a structure-from-motion (SfM) point cloud and aligned to a voxel grid, establishing a structured foundation for the scene representation. During both the training and rendering phases, these anchors dynamically generate multiple surrounding neural 3D Gaussians on the fly through the use of multi-layer perceptrons (MLPs). This process results in significant memory savings, as it reduces the need to store vast numbers of individual Gaussians. For example, representing an indoor scene measuring 3 meters by 8 meters in high quality using Scaffold-GS requires only a few hundred megabytes of storage, which is a considerable improvement over previous methods. Despite these advancements, the storage requirements still present a substantial obstacle for streaming such high-quality volumetric scenes under the constraints of current network infrastructure, necessitating further optimization and innovation in streaming technologies. (a) Original (b) CaV3 [ 19 ] (c) GS3D [ 9 ] (d) Ours Fig. 1 : Visualization effect of different volumetric scene streaming systems. Our proposed GSStream achieves the best visual performance among the state-of-the-art (SOTA) systems with the help of the collaborative viewport prediction module and the DRL-based bitrate adaptation module. Note that these examples are rendered on the stump scene under a network constraint of 120Mbps, at the 10th second (300th frame) of the viewing process. From the perspective of data formatting, 3DGS-derived representations ( e.g. , Scaffold-GS) can be viewed as an evolution of traditional point clouds. A conventional point cloud typically represents a 3D scene as a collection of discrete points, where each point encompasses attributes such as 3D spatial coordinates (x,y,z) , color information, and normal vectors, among others [ 31 ] . Similarly, Scaffold-GS organizes its data by storing anchors as clusters of vectors, with each vector encapsulating both feature and geometric attributes, including center coordinates and scaling coefficients. This similarity in storage structure implies that established point cloud scene streaming methodologies are inherently compatible with 3DGS scenes, allowing for the adaptation of existing streaming techniques to the 3DGS framework. Nonetheless, the challenge of efficiently streaming point cloud content has been a focal point of extensive research, exploring multiple dimensions and strategies. The prevailing approach in this domain involves the simultaneous application of viewport prediction and tile-based bitrate adaptive streaming to minimize the volume of data transmitted [ 6 ] . In practice, this involves partitioning a volumetric scene into a grid of non-overlapping tiles, each of which is individually encoded or downsampled into multiple representations corresponding to different quality levels. Rather than transmitting all tiles at the highest quality, a bitrate adaptation strategy is employed to selectively determine the quality level of each tile based on its relevance to the user’s current and predicted future viewports. For instance, Zhang et al. [ 45 ] proposed Trans-RL and adopted a sequence prediction paradigm where a single Transformer model is utilized to predict the user’s viewport trajectories. Subsequently, DRL is introduced to address the bitrate adaptation challenge, determining the optimal quality level for each tile to balance visual fidelity and bandwidth usage. However, a significant limitation of this approach is the assumption that all volumetric scenes are divided into an identical number of tiles, which is rarely the case in practical applications. Consequently, existing methods encounter three primary challenges: (1) As a novel form of volumetric scene representation, there is a dearth of comprehensive user viewport trajectory datasets specifically tailored for 3DGS scenes, hindering the development of effective viewport prediction models; (2) Current viewport prediction techniques often treat viewport trajectories across different users in a uniform manner, failing to account for individual differences in viewing behavior and personal preferences, which can lead to suboptimal prediction accuracy; and (3) The full potential of DRL algorithms in addressing the bitrate adaptation problem has yet to be realized, as existing approaches have not fully exploited the capabilities of DRL to dynamically and intelligently manage bitrate allocation based on varying network conditions and user behaviors. In this paper, we introduce GSStream , a comprehensive 3D G aussian S platting-based volumetric scene Stream ing system designed to tackle the aforementioned challenges head-on. The GSStream system harnesses the capabilities of Scaffold-GS [ 22 ] to represent volumetric scenes, thereby enhancing the visual quality experienced by users through head-mounted devices (HMDs). Recognizing that users exhibit diverse behavioral inertia when engaging with splatted Gaussians, GSStream incorporates a collaborative viewport prediction algorithm. This algorithm not only predicts user viewports but also extracts and utilizes user-specific embeddings derived from the collected dataset in a collaborative manner, thereby leveraging individual viewing patterns to improve prediction accuracy and relevance. Additionally, GSStream addresses the variability in the number of tiles across different volumetric scenes by treating the tiles decomposed from each scene as an unordered set. To effectively manage bitrate adaptation under these conditions, GSStream models the problem as a Markov decision process (MDP) [ 36 ] and employs a deep deterministic policy gradient (DDPG)-based [ 32 , 18 ] algorithm to derive optimal bitrate allocation policies. For validation, we have constructed a realistic user viewport trajectory dataset tailored for 3DGS content, encompassing 15 diverse volumetric scenes that include both indoor and outdoor scenarios. We developed a specialized user viewport data collection platform to