Nebula : Enable City-Scale 3D Gaussian Splatting in Virtual Reality via Collaborative Rendering and Accelerated Stereo Rasterization DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-

He Zhu 1 Zheng Liu 1 Xingyang Li 1 Anbang Wu 1 Jieru Zhao 1 Fangxin Liu 1 Yiming Gan 3 Jingwen Leng 1,2,† Yu Feng 1,2,† · 2025

Read Nebula : Enable City-Scale 3D Gaussian Splatting in Virtual Reality via Collaborative Rendering and Accelerated Stereo Rasterization DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503- by He Zhu 1 Zheng Liu 1 Xingyang Li 1 Anbang Wu 1 Jieru Zhao 1 Fangxin Liu 1 Yiming Gan 3 Jingwen Leng 1,2,† Yu Feng 1,2,† (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-20495.txt.

Introduction

He Zhu and co-authors render city-scale three-dimensional scenes inside virtual reality headsets. Gaussian splatting builds photorealistic scenes from millions of soft primitives, but city scale breaks current hardware, and streaming video from the cloud eats bandwidth and adds lag. Their system, Nebula, splits the work between cloud and headset.

Instead of video, the cloud streams intermediate results after a level-of-detail search, slashing the data that must cross the network. A temporal-aware search in the cloud exploits coherence between frames to cut redundant memory access. On the client, a new stereo rasterizer lets the two eyes share most computation while staying bit-accurate. With small hardware additions, motion-to-photon latency improves by nearly a factor of three.

Catalog record

CREATORHe Zhu 1 Zheng Liu 1 Xingyang Li 1 Anbang Wu 1 Jieru Zhao 1 Fangxin Liu 1 Yiming Gan 3 Jingwen Leng 1,2,† Yu Feng 1,2,†
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-20495

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Nebula : Enable City-Scale 3D Gaussian Splatting in Virtual Reality via Collaborative Rendering and Accelerated Stereo Rasterization DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 Thanks: † \dagger Corresponding authors CCS: Computer systems organization Neural networks CCS: Computing methodologies Rasterization CCS: Computer systems organization Real-time system architecture He Zhu 1 Zheng Liu 1 Xingyang Li 1 Anbang Wu 1 Jieru Zhao 1 Fangxin Liu 1 Yiming Gan 3 Jingwen Leng 1,2,† Yu Feng 1,2,† Abstract . 3D Gaussian splatting (3DGS) has drawn significant attention in the architectural community recently. However, current architectural designs often overlook the 3DGS scalability, making them fragile for extremely large-scale 3DGS. Meanwhile, the VR bandwidth requirement makes it impossible to deliver high-fidelity and smooth VR content from the cloud. We present Nebula , a coherent acceleration framework for large-scale 3DGS collaborative rendering. Instead of streaming videos, Nebula streams intermediate results after the LoD search, reducing 1925% data communication between the cloud and the client. To further enhance the motion-to-photon experience, we introduce a temporal-aware LoD search in the cloud that tames the irregular memory access and reduces redundant data access by exploiting temporal coherence across frames. On the client side, we propose a novel stereo rasterization that enables two eyes to share most computations during the stereo rendering with bit-accurate quality. With minimal hardware augmentations, Nebula achieves 2.7 × \times motion-to-photon speedup and reduces 1925% bandwidth over lossy video streaming. 1. Introduction Fig. 1. The rendering pipeline for large-scale 3DGS consists of four stages: LoD search, preprocessing, sorting, and rasterization. First, LoD search traverses the LoD tree to determine a set of Gaussians with a desired LoD granularity. The result Gaussians form a “cut” that separates the top and bottom of the LOD tree. Then, the Gaussians on the cut go through a sequence of operations, i.e., preprocessing, sorting, and rasterization, to render an image, similar to the small-scale 3DGS pipelines ( 46 ) . Neural rendering is ushering in a renaissance in computer graphics by enabling photorealistic and view-dependent rendering, with much higher speeds than conventional ray tracing ( 76 ; 16 ; 74 ) . In recent years, neural rendering has drawn significant attention in the architectural community ( 24 ; 104 ; 25 ; 53 ; 55 ; 52 ; 62 ; 18 ; 35 ; 51 ; 78 ; 57 ; 71 ; 56 ; 31 ; 28 ; 82 ; 65 ) , with 3D Gaussian splatting (3DGS) standing out due to its compact representation and superior rendering performance. While prior 3DGS accelerator designs ( 24 ; 104 ; 25 ; 53 ; 55 ; 52 ; 62 ; 18 ; 35 ) achieve real-time mobile rendering for small-scale scenes ( 2 ; 37 ; 48 ) , they often overlook the scalability challenge of 3DGS, making their designs fragile for large-scale rendering (e.g., city-scale) ( 47 ; 79 ; 66 ; 59 ; 98 ) . As shown in Sec. 3.1 , the memory requirement for such scenes can reach up to 66 GB, far exceeding the typical memory capacity ( < 12 GB) of devices in virtual reality (VR) ( 8 ; 6 ; 99 ) . This memory gap motivates us to design a collaborative rendering framework that leverages the resources available in the cloud to overcome the memory constraints of VR devices. Despite numerous solutions for remote or collaborative rendering ( 69 ; 100 ; 54 ; 107 ; 94 ; 101 ; 108 ) , they primarily target conventional video streaming. Such approaches paired with the HEVC codecs ( 84 ; 96 ; 70 ) often require over 1 Gbps for 4K VR content at 90 FPS ( 68 ) . To alleviate the bandwidth pressure, we make two key insights unique to large-scale 3DGS: 1) in virtual 3D scenes, the number of newly visible Gaussians introduced by continuous pose changes remains roughly constant; and 2) the memory requirement of large-scale 3DGS peaks during the initial level-of-detail (LoD) search, but drops sharply in the subsequent stages. By leveraging these insights, we propose Nebula , a collaborative rendering framework tailored for 3DGS at infinite scale . Rather than streaming fully rendered images, Nebula transmits the intermediate results after the initial LoD search, i.e., Gaussians required for subsequent stages, to the client. We show that Nebula requires a 1925% lower bandwidth compared to conventional video streaming. Additionally, Nebula also exhibits strong scalability: its bandwidth demand is less susceptible to resolution or frame rate increases. Algorithmically, Nebula makes three core contributions. First, we propose a temporal-aware LoD search algorithm in Sec. 4.2 that can be deployed upon existing GPUs out-of-the-box. Specifically, our algorithm regulates the DRAM access by streamingly processing data in LoD search and leverages the temporal similarity across frames to avoid unnecessary data accesses. Second, we design a runtime Gaussian management system in Sec. 4.3 that compresses and transmits only the non-overlapped Gaussians across the adjacent frames to further reduce the data transfer between the cloud and the client. Third, for VR rendering, where two tightly paired stereo displays need to be rendered, we introduce a novel stereo rasterization pipeline in Sec. 4.4 that exploits triagulation ( 34 ; 85 ) , a widely-used technique in computer vision to share most of the computations in the remaining pipeline while still producing bit-accurate images. Architecturally, we show that our stereo rasterization can be easily integrated into any mainstream 3DGS accelerators with minimal hardware augmentations (Sec. 5 ). Overall, Nebula achieves 2.7 \times motion-to-photon speedup and reduces bandwidth by 1925% compared to video streaming. On the cloud side, our temporal-aware LoD search delivers up to 52.7 \times speedup compared to off-the-shelf GPU implementations. On the client side, our stereo rasterization achieves up to 21.7 \times speedup and 5.3 \times speedup, compared to a mobile Ampere GPU and the state-of-the-art accelerators ( 53 ; 104 ) , respectively, all with minimal hardware overhead. The contributions of this paper are as follows: • A collaborative rendering framework that is tailored for large-scale 3DGS with great scalability. • Two techniques, temporal-aware LoD search and stereo rasterization, that accelerate both sides of computation by up to 52.7 \times and 21.7 \times . • Our architecture achieves 2.6 \times speedup and reduces bandwidth by 1925% over lossy video streaming. 2. Background In this section, we first give a brief background on remote rendering in Sec. 2.1 . Then, we introduce the general rendering pipeline for large-scale 3DGS in Sec. 2.2 . 2.1. Remote Rendering Video Streaming. The mainstream approach to remote rendering in AR/VR is video streaming. A client first transmits a pose to a remote server, which then renders an image corresponding to that pose and streams it back to the client. Lastly, the client displays the received image on the screen. To reduce the bandwidth requirements, video streaming typically applies various video compression techniques, which can be classified into two main categories: conventional methods ( 96 ; 70 ) and DNN-based methods ( 15 ; 64 ; 97 ) . While DNN-based compressions offer high compression rates, they are often too compute-intensive for latency-sensitive applications such as

Read the complete text