A 129FPS Full HD Real-Time Accelerator for 3D Gaussian Splatting

Fang-Chi Chang, and Tian-Sheuan Chang, Senior Member, IEEE · 2026

Read A 129FPS Full HD Real-Time Accelerator for 3D Gaussian Splatting by Fang-Chi Chang, and Tian-Sheuan Chang, Senior Member, IEEE (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10223.txt.

Introduction

Chang and Chang build a low-power hardware accelerator for 3D Gaussian splatting that renders high-definition images. Compression combines Gaussian pruning and fine-tuning, harmonics reduction, and vector quantization, shrinking models about 52 times with small quality loss. Its frame pipeline merges culling and projection with tile sorting and rasterization, skips zero-Jacobian multiplications, and sorts tiles without comparisons. It delivers 1080p at 129 frames per second under a quarter watt. Against prior accelerators, it is six times smaller, six times faster, and seven and a half times more efficient.

Catalog record

CREATORFang-Chi Chang, and Tian-Sheuan Chang, Senior Member, IEEE
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47728
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10223

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A 129FPS Full HD Real-Time Accelerator for 3D Gaussian Splatting This work was supported by the National Science and Technology Council, Taiwan, under Grant 111-2622-8-A49-018-SB, 110-2221-E-A49-148-MY3, 113-2221-E-A49-078-MY3, and 113-2640-E-A49-005. The authors are affiliated with the Institute of Electronics, National Yang Ming Chiao Tung University, Taiwan. (e-mail: [email protected], [email protected]) This paper has been accepted to be published in IEEE Transactions on Visualization and Computer Graphics. Manuscript received XXXX XX, 2025; revised XXXX XX, XXXX. Fang-Chi Chang, and Tian-Sheuan Chang, Senior Member, IEEE Abstract Rendering large-scale, unbounded scenes on AR/VR-class devices is constrained by the computation, bandwidth, and storage cost of 3D Gaussian Splatting (3DGS). We propose a low-power, low-cost 3DGS hardware accelerator that renders full-HD images in real time, together with a hardware-friendly compression pipeline that combines iterative Gaussian pruning and fine-tuning, progressive spherical harmonics (SH) degree reduction, and vector quantization of all SH coefficients and colors. The scheme achieves a 51.6 × 51.6\times model-size reduction with a 0.743 dB PSNR loss. The accelerator uses a frame-level pipeline that integrates point-based culling and projection with tile-based sorting and rasterization, skips zero-Jacobian matrix multiplications (reducing processing elements by 63% and computation by 53%), and adopts comparison-free tile-based sorting with deterministic latency. Implemented in a TSMC 28-nm process at 800 MHz, the design occupies 0.66 ​ mm 2 0.66~\text{mm}^{2} with 1.1438 M gates and 120 kB SRAM, consumes 0.219 W, and delivers 1219 Mpixels/J at 267.5 Mpixels/s, enabling 1080p at 129 FPS. Overall, it is 5.98 × 5.98\times smaller in area, 5.94 × 5.94\times higher throughput, and delivers 7.5 × 7.5\times higher energy efficiency than prior 3DGS accelerators. I Introduction Recently, 3D Gaussian Splatting (3DGS) has emerged [ 9 ] as an popular novel view synthesis method, using a set of 3D Gaussians to represent scenes, enabling high-quality real-time rendering on GPUs. However, 3DGS models typically require larger memory footprints, making them difficult to deploy on resource-constrained portable devices used in augmented reality (AR) and virtual reality (VR) applications. Thus, hardware acceleration coupled with effective model compression techniques becomes critical. A large body of work have been proposed to reduce the storage and bandwidth footprint of 3DGS through attribute compression, structural reparameterization, and pruning [ 17 , 13 , 4 , 19 , 5 , 16 , 25 , 20 ] . However, most of the software acceleration methods are targeted for GPU-based execution, which is not hardware-friendly. Beyond software acceleration on GPUs, hardware accelerators have also attracted attention in recent years [ 14 , 12 , 22 ] . However, these designs incur high area and power costs because they implement the original 3DGS algorithm, and they cannot meet the low-power and high-throughput requirements of real-time full-HD rendering for AR/VR edge devices [ 18 ] . To overcome these limitations, this paper introduces a dedicated hardware accelerator for real-time rendering of 3DGS models on portable AR/VR devices, targeting full-HD resolution at 120 FPS per eye [ 18 ] . First, for model compression, we modify the lightweight model, LightGaussian [ 4 ] , to higher compression ratios (51.6 \times vs 15 \times ) by iterative Gaussian pruning and SH degree reduction, and by applying VQ to all SH coefficients and colors. Second, our hardware architecture proposes a frame-level pipeline with mixed granularity for high hardware utilization and low latency: point-based processing in preprocessing stages and tile-based processing during rendering. This design further reduces complexity by skipping zero-Jacobian matrix multiplications in the projection path and adopts tile-based, comparison-free sorting with \langle\text{tile-id},\text{depth}\rangle keys to match front-to-back \alpha -blending at fixed per-output latency. The final hardware implementation achieves full-HD rendering at 129 FPS, which is 3 \times higher throughput and 5.98 \times smaller area when compared to the prior 3DGS accelerator [ 14 ] . The remainder of the paper is organized as follows. Section II shows the related work. Section III details the software-level compression methods used for 3DGS. Section IV describes the proposed hardware accelerator design. Experimental results are provided in Section V, followed by concluding remarks in Section VI. II Related Work The rapid adoption of 3D Gaussian Splatting (3DGS) has driven research to reduce storage, rendering-time computation, and latency for interactive and streaming scenarios. Existing efforts mainly trade off reducing bytes per Gaussian (e.g., coding, quantization, and codebooks/VQ) versus reducing Gaussian count (e.g., pruning/compaction and SH-degree reduction). The survey in [ 1 ] summarizes these trade-offs and their impact on representation size and runtime costs such as sorting and blending. Many works compress attributes while largely preserving Gaussian count. Morgenstern et al. [ 17 ] reorganize attributes into 2D grids for codec efficiency. EAGLES [ 7 ] uses an encoding–decoding scheme for memory reduction. Compact3DGS [ 13 ] applies VQ to color and SH parameters, while LightGaussian [ 4 ] reduces SH degree and quantizes low-salience components. ContextGS [ 23 ] introduces an anchor-level context model, HAC [ 3 ] incorporates hash-grid-assisted context, and CodecGS [ 15 ] compress 3DGS using optimized feature planes with standard video codecs. Related budgeting methods also include resolution-aware pruning and SH allocation [ 19 ] . Complementary to byte reduction, compaction and pruning reduce Gaussian count to lower compute, memory traffic, and sorting complexity. Mini-Splatting [ 5 ] combines densification and simplification with constrained SH to ease queuing and sorting, and TAMING 3DGS [ 16 ] uses budget-constrained training to produce predictable model sizes suitable for fixed on-chip buffering. Progressive and streaming-oriented methods improve time-to-first-paint and perceived latency. PRoGS [ 25 ] prioritizes high-contribution splats under bandwidth limits, aligning with termination strategies that stop blending when remaining contributions are negligible. StopThePop [ 20 ] reduces popping via hierarchical resorting; hardware implementations often prefer tile-local approximations to avoid global synchronization and excess traffic. Training-time optimizations can also improve compactness and inference behavior; for example, Kheradmand et al. [ 10 ] formulate training as MCMC sampling to enhance Gaussian relocation. Several works propose hardware accelerators for 3DGS and related pipelines. GSCore [ 14 ] introduces shape-aware tests and hierarchical sorting. Lee et al. [ 12 ] study group sorting, reconfigurable matrices, and importance-aware SH handling. GSNorm [ 22 ] uses normalization and LUT-quantized rendering to decouple dependencies. Earlier tile-reordered heaps [ 24 ] also relate to coherence-friendly ordering, although modern 3DGS places stricter bandwidth and buffering demands. Our work adopts a hardware–compression co-design for resource-constrained devices. We tailor compression to hardware constraints and design the accelerator around tile-level ordering and buffering under limited on-chip SRAM. Unlike accelerators that assume unmodified models or rely on complex global ordering and preprocessing, the proposed co-design achieves higher throughput and energy efficiency with lower area cost. III Proposed Algorithm III-A Background: 3D Gaussian Splatting (3DGS) III-A 1 Pipeline Overview Novel view synthesis (radiance field rendering) generates photorealistic images of a scene from its underlying 3D representation. 3DGS models the scene using a set of Gaussian ellipsoids, enabling fast rendering by rasterizing these Gaussians into high-quality images. The inference process in 3DGS consists of four main steps: near-plane culling, projection, Gaussian sorting, and rasterization. • Near-plane culling: The process begins with a lightweight near-plane visibility test that discards Gaussians behind the camera near plane to avoid unnecessary projection and rendering computations. • Projection: After visibility filtering, the surviving Gaussians, along with view

Read the complete text