Image-Conditioned 3D Gaussian Splat Quantization

Xinshuang Liu, Runfa Blark Li, Keito Suzuki, Truong Nguyen · 2025

Read Image-Conditioned 3D Gaussian Splat Quantization by Xinshuang Liu, Runfa Blark Li, Keito Suzuki, Truong Nguyen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15372.txt.

Introduction

Xinshuang Liu and co-authors wrote this computer graphics paper, published in two thousand twenty-five, on shrinking 3D scene files for storage. The method at issue builds scenes from millions of soft dots, called Gaussian splats, which render fast but take much room.

Their tool squeezes those dots far harder than past methods, down to kilobytes rather than megabytes. It shares one codebook across all scenes instead of storing a fresh one per scene. It also reads fresh photos at playback time, so a scene that changed after archiving can still be rebuilt as it now looks. Training ties the squeeze and rebuild steps together so nothing vital is lost. Tests on scene storage and scene updating beat prior methods on both counts. The team plans to release its code, model, and data in public.

Catalog record

CREATORXinshuang Liu, Runfa Blark Li, Keito Suzuki, Truong Nguyen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-15372

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Image-Conditioned 3D Gaussian Splat Quantization Xinshuang Liu, Runfa Blark Li, Keito Suzuki, Truong Nguyen Abstract 3D Gaussian Splatting (3DGS) has attracted considerable attention for enabling high‑quality real‑time rendering. Although 3DGS compression methods have been proposed for deployment on storage‑constrained devices, two limitations hinder archival use: (1) they compress medium‑scale scenes only to the megabyte range, which remains impractical for large‑scale scenes or extensive scene collections; and (2) they lack mechanisms to accommodate scene changes after long‑term archival. To address these limitations, we propose an Image‑Conditioned Gaussian Splat Quantizer ( ICGS‑Quantizer ) that substantially enhances compression efficiency and provides adaptability to scene changes after archiving. ICGS‑Quantizer improves quantization efficiency by jointly exploiting inter‑Gaussian and inter‑attribute correlations and by using shared codebooks across all training scenes, which are then fixed and applied to previously unseen test scenes, eliminating the overhead of per‑scene codebooks. This approach effectively reduces the storage requirements for 3DGS to the kilobyte range while preserving visual fidelity. To enable adaptability to post‑archival scene changes, ICGS‑Quantizer conditions scene decoding on images captured at decoding time. The encoding, quantization, and decoding processes are trained jointly, ensuring that the codes—quantized representations of the scene—are effective for conditional decoding. We evaluate ICGS‑Quantizer on 3D scene compression and 3D scene updating. Experimental results show that ICGS‑Quantizer consistently outperforms state‑of‑the‑art methods in compression efficiency and adaptability to scene changes. Our code, model, and data will be publicly available on GitHub. 1 Introduction 3D Gaussian Splatting (3DGS) Kerbl et al. 2023 has gained considerable attention for its real-time rendering capabilities and high-quality visual outputs. However, 3DGS representations typically require substantial storage, which limits their applicability to large-scale scenes and extensive scene collections. This storage overhead is particularly problematic on storage-constrained devices such as robots, AR/VR headsets, and smartphones. A promising solution is to compress 3D Gaussians before storage. Recent state-of-the-art methods Navaneet et al. 2024 ; Lee et al. 2024 achieve compression by quantizing 3D Gaussians into discrete codes and storing associated codebooks for decoding. These codebooks are learned individually per scene via vector quantization van den Oord et al. 2017 ; Lee et al. 2022 . While effective, these approaches face two fundamental limitations, which we address with ICGS-Quantizer —an Image-Conditioned Gaussian Splat Quantizer. Limitation 1: Inefficient storage at scale. Existing 3DGS quantizers typically reduce medium-scale scenes only to the megabyte range, which remains impractical for large-scale environments or extensive collections. Two key factors drive this inefficiency: (1) they train separate codebooks for each scene, incurring considerable storage overhead because each scene must store its own floating-point codebooks in addition to the quantized codes; and (2) they quantize each Gaussian and its attributes ( e.g. , rotation, scale) independently, missing correlations both across Gaussians and among attributes. To address this, we train ICGS‑Quantizer on large‑scale data to learn shared codebooks across all training scenes, then fix and apply them to previously unseen test scenes, eliminating per‑scene codebook storage. Furthermore, we jointly compress all Gaussians and their attributes to capture inter-Gaussian and inter-attribute correlations. Precisely, the scene is partitioned into sparse 3D blocks, each containing multiple Gaussians, and each block is compressed into residual discrete codes. Within each block—which preserves local spatial structure—the Gaussians and their attributes are jointly encoded into latent representations before quantization. This design reduces 3DGS storage requirements to the kilobyte range while preserving high visual fidelity. Figure 1 : Image-conditioned scene quantization. (a) At time t_{0} , the scene is encoded and quantized as discrete codes. However, after a prolonged period, the scene may have changed. (b) Conventional methods decode the scene from the codes, but can only recover the original scene at time t_{0}{<}t . (c) Our method decodes the scene from the codes conditioned on its image(s) captured at time t , adapting the scene to its current illumination and appearance. We recommend watching our videos to observe the dynamic results. Limitation 2: No post-archival scene adaptation. In real-world applications, achieving a high compression ratio for 3D scenes is not the sole objective. Over time, scenes may change after archival—for example, lighting can shift from day to night, or colors can fade. Therefore, a critical goal during scene decoding is to adapt scenes to their current illumination and appearance. We achieve this by conditioning scene decoding on one or a few current images (Figure 1 ). Specifically, our conditional decoder extracts multi-scale image features from a pre-trained DINOv2 model Oquab et al. 2024 and fuses them into sparse 3D latents via visibility-aware, coarse-to-fine aggregation. This design enables the decoded scene to adapt to its current illumination and appearance. Furthermore, the encoding, quantization, and decoding processes are trained jointly, so that the quantized codes are optimized for conditional decoding. When no scene update is needed, our ICGS-Quantizer can still decode the archived scene without requiring any conditioning images. We evaluate ICGS-Quantizer on both 3D scene compression and 3D scene updating. For compression, our shared-codebook and joint quantization strategies achieve higher compression ratios and superior rendering quality compared to state-of-the-art methods; for fairness, we do not use conditioning images for our method in this setting. For scene updating, conditioning on current views provides strong adaptability to changes, achieving the highest rendering quality for updated scenes. Additional experiments validate the effectiveness of our coarse-to-fine image-conditioning strategy and demonstrate that the quantizer effectively exploits multiple views while maintaining robustness with only a few input views. Our contribution can be summarized as follows: • Shared codebooks for all scenes. We learn codebooks that are shared across all training scenes and fix them at test time, eliminating per-scene codebook storage and improving generalization. • Joint Gaussian–attribute quantization. We jointly quantize multiple Gaussians and their attributes to exploit both inter-Gaussian and inter-attribute correlations, enabling highly efficient compression. • Image-conditioned scene decoding. We condition scene decoding on a few—or even a single—current view of the scene to adapt it to its current state, in a coarse-to-fine manner. The encoding, quantization, and decoding processes are trained jointly, so that the quantized codes are optimized for conditional decoding. Figure 2 : An overview of our proposed ICGS-Quantizer. (a) Quantization module: geometry and texture are decoupled and quantized independently. (b) Joint quantization of 3D Gaussians and their associated attributes. (c) Image-conditioned decoding of 3D Gaussians. Auxiliary neural network branches and training objectives are omitted here for simplicity. 2 Related Work 2.1 Vector Quantization Vector quantization (VQ) Buzo et al. 1980 ; Gray 1984 approximates a vector with its nearest entry in a learned codebook of representative vectors. The objective of VQ is to minimize the discrepancy between the original vectors and their quantized counterparts. Various extensions have been proposed to enhance quantization accuracy and efficiency. Residual quantization Juang and Jr. 1982 ; Chen et al. 2010 ; Martinez et al. 2014 iteratively quantizes residuals across multiple stages, with each stage quantizing the difference between the original vector and its reconstruction from previous stages. Product quantization Sabin and Gray 2003 ; Jégou et al. 2011 ; Ge et al. 2014 partitions vectors into sub-vectors, quantizing each independently. However, traditional vector quantization methods remain inherently non-differentiable, restricting their compatibility with end-to-end neural network training. VQ-VAE van den Oord et al. 2017 addressed this by copying gradients from the quantizer’s outputs to its inputs.

Read the complete text