Lumina : Real-Time Mobile Neural Rendering by Exploiting Computational Redundancy CCS: Computer systems organization Neural networks CCS: Computing methodologies Rasterization CCS: Computer systems organization Real-time system architecture

Yu Feng, Weikai Lin, Yuge Cheng, Zihan Liu, Jingwen Leng, Minyi Guo, Chen Chen, Shixuan Sun, Yuhao Zhu · 2025

Read Lumina : Real-Time Mobile Neural Rendering by Exploiting Computational Redundancy CCS: Computer systems organization Neural networks CCS: Computing methodologies Rasterization CCS: Computer systems organization Real-time system architecture by Yu Feng, Weikai Lin, Yuge Cheng, Zihan Liu, Jingwen Leng, Minyi Guo, Chen Chen, Shixuan Sun, Yuhao Zhu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-05682.txt.

Introduction

Yu Feng and co-authors published a 2025 paper on making lush 3D graphics run in real time on phones. A popular method renders scenes from clouds of soft 3D blobs, but the math is heavy for mobile chips. Their system, Lumina, cuts the work by exploiting repetition. Consecutive frames barely differ, and many color calculations repeat across a scene.

One algorithm reuses results across time; another caches colors so the costly drawing step runs less often. A custom chip design speeds the cache lookups and fixes weak points in the drawing process itself. Against a mobile graphics processor, Lumina ran about four and a half times faster while using a fifth of the energy. Image quality dropped only slightly on both synthetic and real scenes. The authors aim at interactive neural graphics that fit in a pocket without draining the battery.

Catalog record

CREATORYu Feng, Weikai Lin, Yuge Cheng, Zihan Liu, Jingwen Leng, Minyi Guo, Chen Chen, Shixuan Sun, Yuhao Zhu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-05682

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Lumina : Real-Time Mobile Neural Rendering by Exploiting Computational Redundancy CCS: Computer systems organization Neural networks CCS: Computing methodologies Rasterization CCS: Computer systems organization Real-time system architecture Yu Feng, Weikai Lin, Yuge Cheng, Zihan Liu, Jingwen Leng, Minyi Guo, Chen Chen, Shixuan Sun, Yuhao Zhu Abstract . 3D Gaussian Splatting (3DGS) has vastly advanced the pace of neural rendering, but it remains computationally demanding on today’s mobile SoCs. To address this challenge, we propose Lumina , a hardware-algorithm co-designed system, which integrates two principal optimizations: a novel algorithm, 𝒮 2 {\mathcal{S}}^{2} , and a radiance caching mechanism, ℛ ​ 𝒞 {\mathcal{RC}} , to improve the efficiency of neural rendering. 𝒮 2 {\mathcal{S}}^{2} algorithm exploits temporal coherence in rendering to reduce the computational overhead, while ℛ ​ 𝒞 {\mathcal{RC}} leverages the color integration process of 3DGS to decrease the frequency of intensive rasterization computations. Coupled with these techniques, we propose an accelerator architecture, LuminCore , to further accelerate cache lookup and address the fundamental inefficiencies in Rasterization. We show that Lumina achieves 4.5 × 4.5\times speedup and 5.3 × \times energy reduction against a mobile Volta GPU, with a marginal quality loss ( < < 0.2 dB peak signal-to-noise ratio reduction) across synthetic and real-world datasets. 1. Introduction Neural Radiance Fields (NeRF) ( mildenhall2021nerf ) has transformed the landscapes of Virtual/Augmented Reality (VR/AR) ( rojas2023re ; chen2023mobilenerf ; hu2022efficientnerf ; hedman2021baking ) , large-scale landscape modeling ( xiangli2022bungeenerf ; barron2021mip ; lindell2022bacon ; tancik2022block ) , virtual avatar ( chen2022geometry ; jiang2022neuman ; weng2022humannerf ) , novel view synthesis ( ye2023intrinsicnerf ; zhang2022ray ; liang2023envidr ) , and beyond ( gao2022nerf ) . Although NeRF yields impressive results, its intensive computation is always a key bottleneck to achieving real-time and high-resolution rendering ( wu2024recent ; chen2024survey ) . To address this challenge, 3D Gaussian Splatting (3DGS) has been proposed as a fast alternative to the NeRF pipeline ( kerbl20233d ) . Unlike NeRF, which requires dense sampling along each ray, 3DGS projects precomputed Gaussian points onto the rendering screen, simplifying the color integration process and light transport modeling in 3D spaces. Despite numerous efforts ( kerbl20233d ; wu2024recent ; chen2024survey ; fang2024mini ; lee2023compact ; fan2023lightgaussian ) , 3DGS still falls short of the real-time requirement, e.g., 90 frames per second (FPS) ( questprospec ; visionprospec ; wang2023effect ) , in AR/VR applications. For instance, on a mobile Volta GPU ( xaviersoc ) , 3DGS merely achieves 5 - 21 FPS on real-world scenes ( barron2022mipnerf360 ; Knapitsch2017 ; hedman2018deep ) , far from the real-time target. We aim to enhance the efficiency of 3DGS rendering in AR/VR applications. Our characterizations show that Sorting and Rasterization dominate the 3DGS rendering time, accounting for over 90% of the total execution (Sec. 2.2 ). To address the two main bottlenecks in 3DGS, we introduce Lumina , a hardware-algorithm co-designed system in this paper. Fig. 1. The computation flow of today’s 3DGS rendering pipeline is highlighted in green. Gaussian points are first projected onto the rendering screen to determine their intersection with image tiles (e.g., T_{0} , T_{1} ). Next, each tile sorts the intersected points based on their depth values, from the nearest to the farthest. Finally, within each tile, Gaussian points are splatted onto the screen in the sorted order to render the final image governed by the color integration equation (Eqn. 1 ). Note that, only those Gaussians with significant transparency \alpha_{i} ( \alpha_{i}>\frac{1}{255} ) will be integrated in the final pixel. Algorithms. We first propose a novel Sorting-Shared ( {\mathcal{S}}^{2} ) algorithm to address one of the bottlenecks, Sorting. {\mathcal{S}}^{2} algorithm extends 3DGS by leveraging temporal redundancy in rendering (Sec. 3.1 ), sharing sorting results across multiple frames. In addition, {\mathcal{S}}^{2} preemptively executes Sorting, effectively hiding Sorting execution time and mitigating frame stuttering. By doing so, Lumina reduces the Sorting computations without losing quality. While {\mathcal{S}}^{2} algorithm effectively hides Sorting latency in 3DGS, it does leave the other bottleneck, Rasterization, which is compute-intensive due to its color integration process. To accelerate Rasterization, we propose Radiance Caching ( {\mathcal{RC}} ), a lightweight caching mechanism that leverages the previous rendering results to approximate the current pixel values with a negligible compute overhead. {\mathcal{RC}} leverages a key insight: two rays intersecting the same sequence of Gaussian points would likely yield the same pixel values (Sec. 3.2 ). Specifically, we propose to cache a short record of “significant” ray-Gaussian intersections from previous renderings. Instead of executing the whole color integration, this short record allows us to perform only the initial segment of the color integration and determine the pixel values by matching with the cached record. Fine-Tuning. However, directly applying {\mathcal{RC}} to 3DGS occasionally introduces rendering artifacts. To address this, we propose end-to-end fine-tuning for our new pipeline to enhance rendering quality while maintaining the caching efficiency. Overall, our result shows that {\mathcal{RC}} avoids 55% computation, on average, in color integration with minimal impact on quality. Architectural Support. Coupled with two optimizations, we further propose a customized architecture, LuminCore (Sec. 4 ), to address the inherent computation inefficiencies in Rasterization. Our characterization shows that the color integration in Rasterization is inherently sparse, leading to low GPU utilization due to warp divergence. Radiance-cached Rasterization further exacerbates this divergence, transforming a full pixel rendering into a sparser one. Thus, we design dedicated Neural Rendering Units (NRUs) in LuminCore to tackle the sparse color integration in Rasterization. Along with NRUs, we co-design a specialized hardware cache, LuminCache , to speed up the cache lookup in {\mathcal{RC}} . Result. By integrating Lumina into an off-the-shelf mobile SoC with negligible hardware overhead (0.4%), we show that Lumina achieves 4.5\times speedup and 5.3 \times energy efficiency against a mobile Volta GPU, with a marginal quality loss. Our contributions are summarized as follows: • We introduce a plug-and-play {\mathcal{S}}^{2} algorithm that leverages the temporal coherence in 3DGS to completely hide the Sorting overhead during real-time rendering. • We propose a novel {\mathcal{RC}} mechanism that caches ray-Gaussian intersections and reduces the computation of color integration in Rasterization by 55%. • We propose a LuminCore architecture, co-designed to address the GPU warp divergence in Rasterization. • We demonstrate that Lumina can achieve 4.5\times speedup and 5.3 \times energy reduction against a mobile Volta GPU, with a marginal loss on visual quality. 2. Motivation We begin by introducing 3DGS fundamentals in Sec. 2.1 . Then, we identify the main inefficiencies of the 3DGS pipeline in Sec. 2.2 . 2.1. 3DGS Pipeline Why 3DGS? Recent 3DGS ( kerbl20233d ) revolutionizes NeRF rendering by drastically accelerating the conventional NeRF rendering process. Conventional NeRF rendering is known for its computational intensity due to the extensive ray samplings. Each ray sample is required to perform a MLP operation. In contrast, 3DGS proposes a reverse operation called “splatting”, where Gaussian points (or “Gaussians” for short) are directly projected onto the rendering screen. This method sidesteps the compute-heavy task of ray-object intersections by reversing the workflow: rather than rays seeking Gaussians, Gaussians are directly mapped onto the screen. To date, all 3DGS variants adhere to a uniform rendering process ( kerbl20233d ; wu2024recent ; chen2024survey ; fang2024mini ; lee2023compact ; fan2023lightgaussian ) . The primary variation across different 3DGS algorithms

Read the complete text