Post-Training Quantization via Residual Truncation and Zero Suppression for Diffusion Models
Donghoon Kim, Dongyoung Lee, Ik Joon Chang, Sung-Ho Bae · 2025
Read Post-Training Quantization via Residual Truncation and Zero Suppression for Diffusion Models by Donghoon Kim, Dongyoung Lee, Ik Joon Chang, Sung-Ho Bae (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-26436.txt.
Introduction
Donghoon Kim and co-authors present QuaRTZ in a 2025 paper on compressing diffusion models after training. Such models generate high quality images but cost much compute. Eight bit quantization matches full precision, yet four bit settings lose fine texture because rounding errors grow in dense, small activations.
Their scheme first quantizes at eight bits to handle outliers, then compresses to four bits by suppressing leading zeros, preserving low order bits that carry texture. Theory and tests support the method across activation patterns. On a fast image model, four bit QuaRTZ beat a rival that needed extra full precision branches.
Catalog record
| CREATOR | Donghoon Kim, Dongyoung Lee, Ik Joon Chang, Sung-Ho Bae |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-26436
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Post-Training Quantization via Residual Truncation and Zero Suppression for Diffusion Models Donghoon Kim, Dongyoung Lee, Ik Joon Chang, Sung-Ho Bae Abstract Diffusion models achieve high-quality image generation but face deployment challenges due to their high computational requirements. Although 8-bit outlier-aware Post-Training Quantization (PTQ) matches full-precision performance, extending PTQ to 4 bits remains challenging. Larger step sizes in 4-bit quantization amplify rounding errors in dense, low-magnitude activations, leading to the loss of fine-grained textures. We hypothesize that not only outliers but also small activations are critical for texture fidelity. To this end, we propose Quantization via Residual Truncation and Zero Suppression (QuaRTZ), a 4-bit PTQ scheme for diffusion models. QuaRTZ applies 8-bit min–max quantization for outlier handling and compresses to 4 bits via leading-zero suppression to retain LSBs, thereby preserving texture details. Our approach reduces rounding errors and improves quantization efficiency by balancing outlier preservation and LSB precision. Both theoretical derivations and empirical evaluations demonstrate the generalizability of QuaRTZ across diverse activation distributions. Notably, 4-bit QuaRTZ achieves an FID of 6.98 on FLUX.1-schnell, outperforming SVDQuant that requires auxiliary FP16 branches. Figure 1: Qualitative comparison on PixArt- \Sigma using different quantization settings and our method. 1 Introduction Diffusion models have emerged as the state-of-the-art in generative modeling, achieving remarkable performance in text-to-image synthesis, super-resolution, and inpainting ( Ho et al. 2020 ; Rombach et al. 2022 ; Podell et al. 2023 ; Peebles & Xie 2023 ; Black-Forest-Labs 2024 ) . However, the iterative refinement process is computationally expensive, which limits its deployment in latency- and resource-constrained environments such as mobile devices, on-device AI assistants, or large-scale cloud serving with strict throughput demands. Low-bit quantization reduces memory footprint, bandwidth demand, and arithmetic cost, while enabling efficient execution on modern accelerators ( Han et al. 2015 ) . Post-training quantization (PTQ) is particularly attractive for diffusion models where fine-tuning is costly, as it requires neither retraining nor access to the original dataset ( Nagel et al. 2020 ; Li et al. 2021 ) . While 8-bit and even 6-bit PTQ for diffusion models have proven to be effective ( Li et al. 2023 ; Huang et al. 2024 ; Ryu et al. 2025 ) , pushing to 4-bit precision (W4A4) remains challenging due to error propagation; quantization noise accumulates over hundreds of timesteps, degrading image texture quality. Existing PTQ methods address this issue by focusing on outlier preservation through temporal alignment ( Huang et al. 2024 ; He et al. 2023 ; Chen et al. 2024b ) or condition-aware scaling ( Ryu et al. 2025 ; Li et al. 2023 ) ; however, this overlooks the dominant error source at extremely low precision — the loss of Least Significant Bits (LSBs). Diffusion models refine subtle variations over many iterations ( Ho et al. 2020 ; Peebles & Xie 2023 ) , making them especially sensitive to rounding errors near zero. With activations densely concentrated around small values, truncating LSBs discards critical fine-grained information and leads to collapsed generations. Tackling the conflicting challenge of preserving both outliers and LSBs is essential to making 4-bit quantization practical for diffusion models. To this end, we propose Quantization via Residual Truncation and Zero suppression (QuaRTZ), a novel two-stage 4-bit quantization scheme for diffusion models. The first stage minimizes rounding error through 8-bit min-max uniform quantization, resulting in a fine-grained integer representation of the original value. The second stage compresses integer representations to a targeted 4 bits using a Leading Zero Suppression (LZS) kernel, which preserves a salient 4-bit representation beginning from the top-most activated bit. This process allows for high entropy of the compressed representation, where the magnitude of the outliers is retained and LSBs are preserved without information loss. Our two-stage design simultaneously protects outliers and LSBs, directly addressing the two dominant sources of error at low precision. Our contributions are as follows: • We propose QuaRTZ, a novel 4-bit quantization scheme that successfully balances outlier preservation and LSB precision. • Our QuaRTZ scheme demonstrates state-of-the-art performance in various diffusion architectures, including UNet and DiT backbones. Notably, our W4A4-quantized model outperforms SVDQuant counterparts even without an error compensation module, enabling 3.8x reduction in memory footprint compared to the 16-bit baseline. • We illustrate the effectiveness of QuaRTZ in theoretical, information, empirical and hardware perspectives in depth, providing core reasoning behind the insight of preserving LSBs. 2 Related Works Diffusion models have established state-of-the-art performance in a wide range of image generation tasks, including unconditional generation ( Ho et al. 2020 ; Rombach et al. 2022 ) and text-to-image synthesis ( Rombach et al. 2022 ; Podell et al. 2023 ; Sauer et al. 2024 ; Chen et al. 2024a ; Black-Forest-Labs 2024 ) . Recent extensions integrate transformer backbones, further scaling model capacity and controllability ( Peebles & Xie 2023 ; Chen et al. 2024a ; Black-Forest-Labs 2024 ) . Despite these advances, the inherently iterative denoising process results in slow inference, posing a significant barrier to deployment in latency- or resource-constrained environments. Quantization has emerged as a promising direction to accelerate diffusion models by reducing memory footprint and enabling efficient low-precision arithmetic. Two main paradigms exist: Quantization-Aware Training (QAT), which jointly learns task objectives and quantization parameters ( Esser et al. 2019 ) , and Post-Training Quantization (PTQ), which applies quantization to pretrained models without retraining. While QAT generally achieves higher accuracy at low bit-widths, it requires complete training data and considerable computational resources. PTQ, in contrast, does not require complete data or finetuning, making it a practical path for scaling large generative models where retraining is often infeasible. Consequently, the main focus of recent PTQ research has been primarily on developing methods for handling outliers. Approaches such as PTQ4DM ( Shang et al. 2023 ) , Q-Diffusion ( Li et al. 2023 ) , and TFQM-DM ( Huang et al. 2024 ) mitigate large-magnitude errors through temporal alignment, calibration strategies, or condition-aware scaling ( Li et al. 2023 ; He et al. 2023 ; Chen et al. 2024b ) . Building on these foundations, DGQ ( Ryu et al. 2025 ) achieved W4A6 quantization for text-to-image models, and SVDQuant ( Li et al. 2024b ) reached 4-bit precision by introducing 16-bit LoRA ( Hu et al. 2022 ) branches to absorb quantization error. However, prior work largely fails to maintain image quality below 6-bit precision. While SVDQuant is successful at maintaining image quality, it limits the efficiency gains of full low-bit quantization because it relies on auxiliary FP16 branches. These branches introduce additional parameters, modify the original architecture, and require mixed-precision fusion, which increases implementation complexity and reduces deployment efficiency. In this work, we propose a 4-bit post-training quantization method for diffusion models that preserves fine-grained texture quality while improving efficiency without auxiliary, higher-precision branches. We focus on both outliers and LSBs simultaneously, departing from prior methods that emphasize only outlier preservation. 3 Quantization via Residual Truncation and Zero suppression Figure 2: Illustration of the proposed two-stage quantization. Stage 1 applies 8-bit integer quantization to capture outliers with small step size, and Stage 2 compresses activations to 4 bits via subgroup-based leading-zero suppression. The green color indicates entropy (from low to high across values), while the blue block represents the FLAG bits assigned per