Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantization

Peilin Sun, Jianxin Wu · 2026

Read Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantization by Peilin Sun, Jianxin Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16423.txt.

Introduction

Sun and Wu propose Nonlinear Bipolar Compensation, a post-training quantization approach that adds nonlinear fixes to reduce the effect of outliers. Current quantization methods suffer a speed-accuracy tradeoff and limited general use, while recent compensation methods add lightweight linear layers yet fail on outliers. They further design Bipolar Logarithmic Transformation, which maps the quantized input and its error into a new space where a simple linear layer applies compensation. Experiments across tasks, models, and quantization methods confirm its effectiveness, robustness, and generality.

Catalog record

CREATORPeilin Sun, Jianxin Wu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53245
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.16423

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantization Peilin Sun, Jianxin Wu Abstract Network quantization has emerged as one of the most practical model compression techniques, which significantly reduces a model’s memory and compute consumption by mapping floating-point numbers to low-bit representations. However, existing quantization methods typically suffer from the speed-accuracy tradeoff and limited generalization. To address these issues, recent compensation-based methods offer an efficient yet general solution by introducing additional lightweight linear layers into the quantized network. However, the accuracy of these methods suffers from their limited compensation capability and high sensitivity to outliers. In this paper, we propose Nonlinear Bipolar Compensation (NBC), a post-training quantization approach that introduces nonlinear compensation to reduce the effect of outliers. We further design Bipolar Logarithmic Transformation (BLT), which compresses outliers by mapping both the quantized input and the quantization error into a transformed space. A simple linear layer is then applied for compensation in the transformed space, preserving the efficiency of our method. Extensive experiments across various tasks, models, and quantization methods confirm the effectiveness, efficiency, robustness, and generality of our NBC approach. 1 Introduction In recent years, with the development of deep learning, deep neural networks have demonstrated outstanding performance across diverse fields, including vision [ 1 , 2 ] , language [ 3 , 4 ] , and multimodal [ 5 ] tasks, simultaneously raising an increasing demand for resources including computation, storage, memory, and bandwidth. Therefore, model compression [ 6 ] not only becomes a research focus in academia, but also plays a crucial role in real-world application scenarios with resource constraints and real-time inference requirements. Network quantization [ 7 ] stands out among various model compression techniques for its high efficiency and hardware-friendliness. Based on whether retraining is required to recover accuracy after quantization, existing quantization methods can typically be divided into the following two categories: Quantization Aware Training (QAT) [ 8 , 9 , 10 , 11 ] and Post-Training Quantization (PTQ) [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ] . QAT and optimization-based PTQ methods can typically achieve higher accuracy, but their high computational cost and long training process limit their practical deployment. In contrast, PTQ methods require only a small amount of unlabeled data for fast calibration but suffer a severe accuracy drop under low-bit quantization. Therefore, current quantization methods face a difficult tradeoff between accuracy and efficiency. At the same time, the generality of existing quantization approaches is limited, requiring the customization of quantization methods for different model architectures and tasks. To summarize, the field of network quantization still urgently needs an efficient, general, and practical method. Recently, compensation-based methods [ 20 , 21 ] have introduced lightweight linear layers into quantized networks to mitigate quantization errors. While these approaches have achieved promising improvements in low-bit accuracy, their reliance on linear transformations introduces critical bottlenecks: • Limited compensation capability. These methods apply simple linear layers to compensate for the quantization loss. However, the information loss caused by quantization in deep neural networks is inherently highly nonlinear, making it difficult to be fully compensated by linear transformations. As a result, this has become the main reason that limits their accuracy. Figure 1(a) illustrates the importance of nonlinearity in compensation by comparing nonlinear compensation and linear compensation. • High sensitivity to outliers. In many modern networks (especially large language models and transformers), activations often contain outliers that significantly exceed the range of the majority of values [ 22 , 23 , 24 ] . In existing compensation-based methods, linear compensation modules are typically initialized by minimizing the residual sum of squares. Since this approach magnifies the effect of outliers, the resulting regression line tilts excessively toward them to minimize their large squared errors. This sacrifices optimal fit for the majority of the data, making these methods highly sensitive to outliers. Figure 1(b) illustrates how outliers can drag the Ordinary Least Squares (OLS) solution away from the majority of normal activations. We formally show that this failure mode is structural to linear OLS optimization in Appendix B . (a) Nonlinearity improves accuracy (b) Outlier drag in OLS compensation Figure 1 : The left figure (a) shows the ImageNet Top-1 accuracy of 4-bit quantized models, comparing nonlinear compensation and linear compensation. The MLP is a nonlinear compensation method that replaces the linear compensation module with a multilayer perceptron. Although it improves accuracy by introducing nonlinear compensation, it requires training and involves a large number of hyperparameters that must be adjusted. Please refer to the Supplementary Material for details on its implementation. The right figure (b) illustrates how outliers can pull the OLS-fitted linear compensation toward a small set of extreme activations, using a single (x^{q},y-y^{q}) channel pair in block 11 of W4A4 RepQ-ViT ViT-B. Outliers are defined as activations whose absolute value exceeds 10 . In the original space, OLS on all tokens is dragged opposite to OLS on inliers only, with a slope gap of 0.55 ; after the BLT transform, the two lines become nearly parallel with a slope gap of 0.18 , motivating NBC’s compensation in the BLT space. To address these issues, we propose a novel post-training quantization approach called Nonlinear Bipolar Compensation (NBC). Our NBC method introduces nonlinearity to better handle complex quantization loss and outliers, while keeping simplicity and efficiency. NBC introduces a nonlinear function f(x) and its inverse f^{-1}(x) . f(x) maps the quantized input x^{q} and the quantization error y-y^{q} into a new space. A linear layer is then utilized to compensate for the quantization error in this new space. Finally, the result is mapped back to the original space via f^{-1}(x) to achieve error compensation. To compress outliers, we propose the Bipolar Logarithmic Transformation (BLT), which serves as the specific implementation of the nonlinear function f(x) . BLT employs a linear mapping near the origin and applies logarithmic compression to values farther from zero. BLT addresses the limitation of the standard log transformation for data containing negative values, while preserving its benefit of mitigating outliers. Our NBC method offers four key advantages: • Effectiveness. NBC delivers substantial accuracy gains in the low-bit regime, e.g., +4.6\% average top-1 over the RepQ-ViT [ 13 ] baseline at W4A4 ImageNet classification, and consistently outperforms the linear compensation method QwT [ 20 ] . • Efficiency. The compensation parameters are obtained via a closed-form solution, requiring no backpropagation. Calibration completes within a few minutes. Our NBC module adds only roughly 4% overhead relative to the full-precision backbone. • Robustness. By transforming both the quantized input and the quantization error through BLT, NBC reduces compensation error on both outlier and non-outlier activations. • Generality. NBC consistently improves accuracy across diverse tasks (vision, language, and multimodal recognition) and PTQ methods (RepQ-ViT [ 13 ] , AdaLog [ 25 ] , Q-DiT [ 26 ] , DuQuant [ 27 ] ). We summarize the main contributions of this paper as follows. • We analyze existing compensation-based PTQ methods and identify two key limitations: their limited capability to compensate for nonlinear quantization loss and their high sensitivity to outliers in activations . • To overcome the limitations of existing compensation-based methods, we propose the Nonlinear Bipolar Compensation (NBC)

Read the complete text