Activation Sensitivity as a Unifying Principle for Post-Training Quantization

Bruce Changlong Xu · 2026

Read Activation Sensitivity as a Unifying Principle for Post-Training Quantization by Bruce Changlong Xu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-11663.txt.

Introduction

Bruce Changlong Xu unifies post-training quantization under one principle. The 2026 preprint names that principle activation sensitivity. Quantization shrinks LLM weights to fewer bits for cheap runs. Post-training methods must guess which weight channels matter most.

Two paradigms dominate the field's heuristics today. Activation-aware methods like AWQ favor channels with large activations. Second-order methods like GPTQ use input covariance structure. They spread quantization error by that structure instead. Xu argues both rest on one shared idea. Sensitivity of activations is what each method estimates. A unifying view could yield simpler, stronger quantizers. Smaller models with less quality loss are the prize.

Catalog record

CREATORBruce Changlong Xu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-11663

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Activation Sensitivity as a Unifying Principle for Post-Training Quantization Bruce Changlong Xu Abstract Post-training quantization (PTQ) methods for large language models rely on heuristics that implicitly estimate which weight channels matter most to model behavior. Two dominant paradigms have emerged: activation-aware methods such as AWQ prioritize channels with large activation magnitudes, while second-order methods such as GPTQ allocate quantization error according to input covariance structure. Despite their empirical success, these approaches remain conceptually fragmented, and it is unclear what underlying quantity they are approximating. In this work, we present a unified theoretical framework for PTQ by formalizing activation sensitivity: the expected impact of channel-wise perturbations on the loss. Using a first-order Taylor expansion of the loss, we show that sensitivity emerges naturally as the squared norm of gradient-weighted activations, providing a principled measure of channel importance that simultaneously captures activation magnitude and downstream error propagation. Within this framework, AWQ and GPTQ arise as complementary approximations that recover sensitivity under distinct simplifying assumptions—uniform downstream gradients and activation-agnostic covariance, respectively. We analyze the design space of sensitivity metrics, connecting gradient-based saliency, Fisher information, and Hessian-based criteria, and clarify their relationships to classical pruning methods such as Optimal Brain Damage and Optimal Brain Surgeon. This perspective exposes fundamental limitations of layer-local reconstruction objectives and highlights open challenges in post-training quantization, including cross-layer error accumulation, calibration distribution mismatch, and task-conditional sensitivity. Rather than proposing a new quantization algorithm, this work provides a conceptual foundation for understanding, comparing, and extending PTQ methods through the lens of sensitivity. 1 Introduction The deployment of large language models (LLMs) is increasingly constrained by memory bandwidth and storage capacity, particularly in low-latency and cost-sensitive inference settings. Post-training quantization (PTQ) has therefore emerged as a central tool for compressing pretrained models to low bit-widths without retraining. While recent methods enable aggressive weight-only quantization at INT4 and below, their behavior remains uneven across models and layers, and the principles governing their success are not yet well understood. A common thread across modern PTQ methods is the use of layer-local proxy objectives to guide quantization decisions. These proxies implicitly estimate which weights or channels are most important, and allocate quantization error accordingly. Two influential paradigms exemplify this approach. Activation-aware methods such as AWQ ( 8 ) posit that channels processing large-magnitude activations disproportionately influence model behavior, and should therefore be protected from quantization error. Second-order methods such as GPTQ ( 4 ) instead frame quantization as a quadratic optimization problem, using input covariance as a surrogate Hessian to redistribute error along directions of low sensitivity. Both approaches have proven effective in practice, yet they rely on fundamentally different—and only partially overlapping—notions of importance. Despite their empirical success, these methods offer incomplete and sometimes conflicting views of quantization sensitivity. Activation magnitude is a first-order heuristic that captures how computation is used, but ignores how errors propagate through downstream computation. Second-order covariance captures error amplification within a layer, but is agnostic to activation scale and structure. As a result, existing PTQ methods may succeed or fail depending on architectural details, activation distributions, and layer function, without a clear explanation of why. More broadly, no existing framework makes explicit what quantity reconstruction-based PTQ objectives are attempting to approximate. In this work, we argue that the central object underlying PTQ is neither activation magnitude nor input covariance alone, but rather activation sensitivity : the expected impact of channel-wise perturbations on the loss. We formalize this notion by analyzing how small weight perturbations affect the loss through a first-order Taylor expansion. This derivation yields a simple and interpretable saliency metric—the squared norm of gradient-weighted activations—that captures both how strongly a channel is used and how errors in that channel propagate downstream. Importantly, this quantity arises directly from the structure of the loss, rather than from heuristic considerations. This perspective provides a unifying lens on existing PTQ methods. Activation-aware approaches such as AWQ can be interpreted as approximating sensitivity under the assumption of uniform downstream gradients, reducing sensitivity to activation magnitude. Second-order methods such as GPTQ emerge when sensitivity is approximated through unweighted input covariance, effectively marginalizing over activation structure. More sophisticated reconstruction-based methods, including block-wise objectives ( 7 ) , can be viewed as partial attempts to account for interactions that layer-local approximations neglect. From this viewpoint, existing PTQ techniques differ not in whether they model sensitivity, but in which assumptions they make in approximating it. Beyond unification, formalizing activation sensitivity clarifies the limitations of layer-local proxy objectives. Because sensitivity is defined with respect to the loss, it depends on downstream computation, calibration distributions, and cross-layer error accumulation—factors that cannot be fully captured by reconstruction objectives optimized independently per layer. This helps explain why PTQ methods often exhibit layer-dependent behavior, and why improvements to proxy objectives do not necessarily translate to improvements in end-to-end performance. Rather than proposing a new quantization algorithm, this paper provides a theoretical framework for understanding post-training quantization through the lens of sensitivity. We analyze the design space of sensitivity metrics, connecting gradient-based saliency, Fisher information, and Hessian-based criteria, and relate modern PTQ methods to classical second-order pruning techniques such as Optimal Brain Damage ( 6 ) and Optimal Brain Surgeon ( 5 ) . The goal is to clarify what existing PTQ methods are approximating, when their assumptions are valid, and where fundamental challenges remain. 2 Sensitivity as the Object of Post-Training Quantization Post-training quantization can be viewed as the introduction of structured, irreversible perturbations to a pretrained parameter vector. From this perspective, the central question is not how to round weights, but how the resulting perturbations interact with the loss. In this section, we formalize this interaction and argue that sensitivity —the response of the loss to channel-wise perturbations—is the fundamental quantity that post-training quantization objectives implicitly approximate. 2.1 Quantization as Structured Perturbation Let W denote the pretrained parameters of a model and let \widehat{W}=W+\Delta W denote the quantized parameters, where \Delta W represents quantization error. For weight-only PTQ, \Delta W is fixed after calibration and applied uniformly at inference time. The effect of quantization on model behavior is therefore governed entirely by how these perturbations propagate through the network. Let \mathcal{L}(W) denote the expected loss under the deployment distribution. For sufficiently small perturbations, a first-order expansion yields \mathcal{L}(W+\Delta W)\;\approx\;\mathcal{L}(W)+\langle\nabla_{W}\mathcal{L}(W),\Delta W\rangle. (1) Equation ( 1 ) makes explicit that the impact of quantization error depends not only on its magnitude, but on its alignment with directions of high loss sensitivity. Any PTQ objective that operates by minimizing a proxy reconstruction loss can therefore be interpreted as implicitly approximating the inner product in ( 1 ) under simplifying assumptions. 2.2 Channel-Wise Structure in Linear Layers We focus on a single linear layer with weight matrix W\in\mathbb{R}^{d_{\mathrm{out}}\times d_{\mathrm{in}}} and input activations X\in\mathbb{R}^{n\times d_{\mathrm{in}}} collected on a calibration set. The layer output is given by Y=XW^{\top} . A perturbation to input channel j corresponds to modifying the column W_{:,j} , inducing an output perturbation \Delta Y=X_{:,j}(\Delta W_{:,j})^{\top}. (2) Many PTQ methods define channel importance in terms of the magnitude of this output perturbation. Averaging over calibration samples yields the quantity \mathbb{E}\!\left[\|\Delta Y\|_{F}^{2}\right]\;\propto\;\|\Delta W_{:,j}\|_{2}^{2}\,\mathbb{E}\!\left[\|X_{:,j}\|_{2}^{2}\right], (3) which recovers the activation-magnitude criterion employed by activation-aware methods. This measure captures how frequently a channel is used, but remains

Read the complete text