SLTarch : Towards Scalable Point-Based Neural Rendering by Taming Workload Imbalance and Memory Irregularity

Xingyang Li, Jie Jiang, Yu Feng, Yiming Gan, Jieru Zhao, Zihan Liu, Jingwen Leng, Minyi Guo, {brucelee_sjtu, jiang_jie, y-feng}@sjtu.edu.cn Equal contribution. Corresponding author · 2025

Read SLTarch : Towards Scalable Point-Based Neural Rendering by Taming Workload Imbalance and Memory Irregularity by Xingyang Li, Jie Jiang, Yu Feng, Yiming Gan, Jieru Zhao, Zihan Liu, Jingwen Leng, Minyi Guo, {brucelee_sjtu, jiang_jie, y-feng}@sjtu.edu.cn Equal contribution. Corresponding author (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-21499.txt.

Introduction

Xingyang Li and co-authors present SLTarch, a design for faster point-based neural rendering, in this 2025 paper. Rendering matters for 3D modeling, virtual reality, and self-driving systems. Point-based methods give realistic images at lower cost than classic pipelines.

Real-time speed on mobile hardware, though, is still out of reach. The team pinpoint two bottlenecks: level-of-detail search and splatting. Search suffers from uneven work and scattered memory access on standard graphics chips. Splatting makes threads in a group do wildly different amounts of work. Their architecture tames both problems at once. Smoother workloads should unlock high-quality rendering on portable devices. Games and film tools could also reuse the same core design.

Catalog record

CREATORXingyang Li, Jie Jiang, Yu Feng, Yiming Gan, Jieru Zhao, Zihan Liu, Jingwen Leng, Minyi Guo, {brucelee_sjtu, jiang_jie, y-feng}@sjtu.edu.cn Equal contribution. Corresponding author
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-21499

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SLTarch : Towards Scalable Point-Based Neural Rendering by Taming Workload Imbalance and Memory Irregularity Xingyang Li, Jie Jiang, Yu Feng, Yiming Gan, Jieru Zhao, Zihan Liu, Jingwen Leng, Minyi Guo, {brucelee_sjtu, jiang_jie, y-feng}@sjtu.edu.cn Equal contribution. Corresponding author Abstract Rendering is critical in fields like 3D modeling, AR/VR, and autonomous driving, where high-quality, real-time output is essential. Point-based neural rendering (PBNR) offers a photorealistic and efficient alternative to conventional methods, yet it is still challenging to achieve real-time rendering on mobile platforms. We pinpoint two major bottlenecks in PBNR pipelines: LoD search and splatting . LoD search suffers from workload imbalance and irregular memory access, making it inefficient on off-the-shelf GPUs. Meanwhile, splatting introduces severe warp divergence across GPU threads due to its inherent sparsity. To tackle these challenges, we propose SLTarch , an algorithm-architecture co-designed framework. At its core, SLTarch introduces SLTree , a dedicated subtree-based data structure, and LTcore , a specialized hardware architecture tailored for efficient LoD search. Additionally, we co-design a divergence-free splatting algorithm with our simple yet principled hardware augmentation, SPcore , to existing PBNR accelerators. Compared to a mobile GPU, SLTarch achieves 3.9 × \times speedup and 98% energy savings with negligible architecture overhead. Compared to existing accelerator designs, SLTarch achieves 1.8 × \times speedup with 54% energy savings. I Introduction Rendering is fundamental across various domains, including 3D modeling [ 1 , 2 , 3 ] , augmented and virtual reality (AR/VR) [ 4 , 5 , 6 ] , autonomous driving [ 7 , 8 , 9 ] , and many more [ 10 , 11 ] . High-quality, real-time rendering is essential in these fields to deliver immersive experiences. Among the existing rendering techniques, point-based neural rendering (PBNR) [ 12 , 13 , 14 , 15 ] has emerged as the hottest topic in the recent two years [ 16 , 11 ] . Unlike conventional rasterization techniques, PBNR leverages Gaussian point primitives, also called Gaussians, with learnable parameters to achieve photo-realistic rendering, while it can provide performance advantages over other recent advanced techniques, such as ray tracing [ 17 , 18 , 19 ] and neural radiance field [ 5 , 6 , 20 ] . Motivation. Despite PBNR having its performance advantages over other techniques, PBNR itself is still challenging to completely replace conventional rasterization pipelines on mobile platforms, due to its scalability and runtime performance [ 21 , 22 ] . On large-scale datasets [ 13 , 23 ] , PBNR algorithms [ 13 , 12 ] barely achieve 15 frame-per-second (FPS) on mobile Ampere GPU on Nvidia Orin SoC [ 24 ] , far from real-time VR rendering requirements, 60 FPS [ 25 , 26 ] . To address this, we propose, SLTarch , an algorithm-architecture co-designed system dedicated to real-time rendering in VR. While numerous accelerators have been proposed recently for PBNR [ 22 , 27 , 28 , 29 , 30 , 31 ] , they exclusively focus on accelerating one specific stage of PBNR, splatting , while often overlooking another crucial stage, level-of-detail (LoD) search , in PBNR. Our key observation is that, as the rendering scene scales up, the performance bottleneck gradually shifts from splatting to LoD search. Our experiments show that LoD search can take up to 70% of the overall execution time in Sec. II-B . Thus, it is important to address both stages to achieve high performance across various scales. LoD Search. We first dissect the bottlenecks in the LoD search. The bottlenecks of the LoD search are mainly from two aspects: imbalanced workload and irregular memory access. Algorithmically, PBNR uses a LoD tree to represent scene attributes such as geometry and color. The LoD tree itself is imbalanced because each tree node has an unfixed number of child nodes. Such irregularity makes its tree traversal hard to parallelize on today’s GPUs, often resulting in imbalanced workloads across threads. In addition, the irregular tree traversal often introduces irregular memory accesses, leading to pipeline stalls and costly DRAM accesses [ 32 , 33 ] . To address these two issues, we propose SLTree , a novel data structure that can translate a canonical LoD tree into a subtree-based data structure while preserving the hierarchical relationship in the original LoD tree (Sec. III ). Unlike other tree structures, e.g., kd-tree [ 34 ] or octree [ 35 ] , SLTree ensures balanced workloads across threads by restricting subtrees to similar sizes, allowing one thread to process one subtree at a time. In addition, our SLTree data structure inherently groups closely visited tree nodes together, preserving high spatial locality. With our co-designed traversal algorithm, the tree traversal naturally poses “structures” on memory accesses and turns irregular DRAM accesses into streaming ones. Note that, SLTree does not change the semantics of the algorithm and generates bit-accurate results as the canonical LoD tree. While SLTree traversal addresses static workload imbalance, as the camera pose moves during rendering, the workload could vary dynamically throughout execution. Conventional tree-traversal accelerators [ 32 , 36 , 33 , 37 ] fail to address dynamic workload changes because they often adopt offline scheduling, assigning every thread with an equal workload subtree. To address the dynamic workload, we propose our architectural support, LTcore , which supports dynamic scheduling to adapt to PBNR’s view-dependent workloads with a novel subtree cache for tree node lookup (Sec. IV-B ). Splatting. As the second major bottleneck in PBNRs, splatting suffers from severe warp divergence. Rather than adopting advanced Gaussian-tile intersections [ 22 ] to mitigate this issue, we propose a simple yet principled algorithm-hardware co-design to eliminate warp divergence completely. Our key observation is that adjacent pixels typically integrate similar sets of Gaussians. Based on this insight, we group pixels into small blocks and approximate the Gaussian transparency check by performing this checking at the group-level rather than the per-pixel level. This way, all pixels within a group share the same Gaussian integration list, effectively removing warp divergence from splatting. Results. Our experiments show that, SLTarch achieves 3.9 \times speedup and 98% energy savings against a off-the-shelf mobile GPU with our algorithm-hardware co-design. Compared against existing accelerators, our design can achieve 1.8 \times speedup and 54% energy savings with a comparable area. Our contributions are summarized as follows: • We introduce a novel data structure, SLTree , along with its co-designed algorithm, that tames the imbalance workloads and irregular DRAM accesses in LoD search with bit-accurate results as the canonical LoD tree. • We propose SLTarch , the first-of-its-kind accelerator for large-scale PBNR, to address dynamic imbalance in SLTree traversal and warp divergence in splatting. • SLTarch achieves 3.9 \times speedup and 98% energy savings against a mobile GPU. With a similar chip area, SLTarch achieves 1.8 \times speedup and 54% energy savings against a state-of-the-art PBNR accelerator, GSCore. II Background and Motivation Fig. 1: An example of the scalable PBNR pipeline primarily consists of two steps: LoD search and splatting. In LoD search, Gaussians at the defined LoD are selected; the selected Gaussians are known as a “cut” of the LoD tree. The selected Gaussians first check the intersections with pixels and then “splats” on the screen. In the splatting stage, the green marks highlight

Read the complete text