GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers

Guang Liang, Xinyao Liu, Jianxin Wu · 2025

Read GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers by Guang Liang, Xinyao Liu, Jianxin Wu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11784.txt.

Introduction

Vision transformers read images well, but their size makes them slow and hungry on small devices. Guang Liang, Xinyao Liu, and Jianxin Wu shrink them to 4-bit numbers in a 2025 paper. Their method, GPLQ, trains the shrunk model in a way that keeps both accuracy and range.

Most loss comes from shrinking the signals, not the weights, they found. So stage one shrinks signals alone for a single pass, keeping the model in its old comfort zone. Stage two then shrinks the weights with a fast method. The result ran 100 times faster than older training routes. At 4 bits, scores stayed near full-size levels across many tasks.

Catalog record

CREATORGuang Liang, Xinyao Liu, Jianxin Wu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-11784

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers Guang Liang, Xinyao Liu, Jianxin Wu Abstract Vision Transformers (ViTs) are essential in computer vision but are computationally intensive, too. Model quantization, particularly to low bit-widths like 4-bit, aims to alleviate this difficulty, yet existing Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT) methods exhibit significant limitations. PTQ often incurs substantial accuracy drop, while QAT achieves high accuracy but suffers from prohibitive computational costs, limited generalization to downstream tasks, training instability, and lacking of open-source codebase. To address these challenges, this paper introduces General, Practical, and Lightning Quantization (GPLQ), a novel framework designed for efficient and effective ViT quantization. GPLQ is founded on two key empirical insights: the paramount importance of activation quantization and the necessity of preserving the model’s original optimization “basin” to maintain generalization. Consequently, GPLQ employs a sequential “activation-first, weights-later” strategy. Stage 1 keeps weights in FP32 while quantizing activations with a feature mimicking loss in only 1 epoch to keep it stay in the same “basin”, thereby preserving generalization. Stage 2 quantizes weights using a PTQ method. As a result, GPLQ is 100x faster than existing QAT methods, lowers memory footprint to levels even below FP32 training, and achieves 4-bit model performance that is highly competitive with FP32 models in terms of both accuracy on ImageNet and generalization to diverse downstream tasks, including fine-grained visual classification and object detection. We will release an easy-to-use open-source toolkit supporting multiple vision tasks. 1 Introduction Vision Transformer (ViT) dosovitskiy2020image ; vaswani2017attention has emerged as the mainstream backbone network in computer vision, but it demands substantial computational and memory resources. Model quantization is one of the key techniques to address this challenge by reducing the numerical precision of model parameters and/or activation values lang2024comprehensive ; li2022q . However, existing quantization methods still faces challenges, especially in low-bit (e.g., 4-bit) quantization. Mainstream methods include Post-Training Quantization (PTQ) liu2021post and Quantization-Aware Training (QAT) esser2019learned . PTQ has fast speed and low resource consumption, but often leads to large accuracy drop under 4-bit quantization li2023repq . On the other hand, QAT simulates quantization operations during training and enables higher accuracy than that of PTQ, or even higher than that of floating-point models . Nevertheless, in this paper we will show that existing QAT methods have inherent limitations: • High Computational Costs. QAT requires lengthy fine-tuning of the entire model. Training time and GPU memory required in QAT often far exceed those for training the FP32 model lang2024comprehensive . This makes QAT cumbersome and very slow for deployment in real-world applications. • Limited Generalization Ability. QAT methods often boast higher accuracy than their FP32 counterparts. However, in this paper we will show that such models are generalizing worse than FP32 or PTQ quantized models in downstream tasks. That is, they are likely non-generalizable beyond ImageNet deng2009imagenet , the dataset on which they were trained. • Training Instability and Complexity. QAT is prone to training instability huang2023quantization , and complex Knowledge Distillation (KD) techniques li2022q ; huang2023quantization severely increase memory footprint. Some also rely on external, extremely powerful teacher models, which are not available in practical scenarios. In short, existing QAT methods are not practical . • Classification Only and Code Missing. Open-source code for QAT is rare, and is only for classification when it exists. This further makes QAT impractical for real-world applications. To this end, we propose GPLQ (General, Practical, and Lightning Quantization). The core objective of GPLQ is to provide a quantization solution that is far more training-efficient than traditional QAT, superior to PTQ in accuracy and generalization, easy to use, and highly practical. As a result, Figure 1 demonstrates 3 core advantages of GPLQ. • General. GPLQ exhibits excellent average accuracy on multiple downstream tasks : close to or even surpassing FP32 models, and significantly outperforming existing QAT methods. • Practical. GPLQ has very small training memory footprint (far lower than existing QAT methods), which avoids out-of-memory (OOM) issues in many applications and enables quantization of larger models. GPLQ’s design allows it to be conveniently applied to other tasks such as object detection . • Lightning. GPLQ is blazingly fast: hundreds of times faster than existing QAT methods. Figure 1 : Core advantages of our GPLQ: Generality, Practicality, and Lightning efficiency. GPLQ is based on our empirical findings. First, activations are far more important than weights in low-bit quantization. Second, quantization should not change its optimization “basin” (i.e., avoid jumping out of the current local minimum ) in order to keep the generalization ability. Based on these findings, GPLQ adopts a sequential quantization paradigm. First, activations are quantized with weights kept at FP32. To maintain generalization, we draw inspiration from TCS zhou2025all and employ a PCA-based feature mimicking loss to guide the quantized model’s feature outputs to approximate those of the original FP32 model (i.e., stay in the same basin). Second, after activations are quantized, existing efficient PTQ methods are used to quantize the weights. This “activation-first, weights later” strategy not only drastically reduces QAT training time from days to 1-2 hours and with memory footprint even lower than FP32 training , but also allows a 4-bit model to achieve both accuracy and generalization nearly identical to the original FP32 model. The main contributions are: 1. Insights. We reveal that activation quantization is the main bottleneck in QAT, and staying in the original optimization basin is crucial for generalization. 2. GPLQ. We propose “activation-first” sequential quantization: first optimize activations then quantize weights via PTQ. 3. Code. GPLQ provides an easy-to-use quantization tool supporting classification, detection and other downstream tasks. We will open-source GPLQ upon paper acceptance. 2 Related Work Model quantization aims to enhance model efficiency by reducing the numerical precision of weights and activations in neural networks papa2024survey . Post-Training Quantization (PTQ) . PTQ operates without retraining, requires only a small calibration set, and is very fast. Various techniques have been proposed: AIQViT jiang2025aiqvit , GPTQ frantar2022gptq , PTQ4ViT yuan2022ptq4vit ), SmoothQuant xiao2023smoothquant , AWQ lin2024awq . More methods like RepQ-ViT li2023repq and QwT fu2024quantization perform optimization through scale reparameterization and lightweight compensation modules, respectively. Accuracy degradation remains a severe challenge in low-bit scenarios. The second stage of GPLQ employs PTQ to quantize weights. Since activations have been quantized via QAT in the first stage, PTQ’s duty changes from W32A32 \rightarrow W4A4 to W32A4 \rightarrow W4A4. Quantization-Aware Training (QAT) . QAT introduces simulated quantization during training or fine-tuning, and achieves higher accuracy than PTQ. It often uses a Straight-Through Estimator (STE) to handle gradient esser2019learned . Research directions include learning quantization scales esser2019learned , improving training stability) (OFQ liu2023oscillation , Quantization-Variation huang2023quantization ), enhancing efficiency (EfficientQAT chen2024efficientqat ), and specific optimizations for ViTs (e.g., Q-ViT li2022q , PackQViT dong2023packqvit ). The bottlenecks of QAT are high computational cost, training instability, and potential degradation in generalization ability. GPLQ, with an extremely short QAT stage ( only 1 epoch ) focused solely on activations, effectively alleviates the cost, stability, and generalization issues of traditional QAT. Knowledge Distillation (KD) in QAT . KD hinton2015distilling ; gou2021knowledge ; wang2021distilling ; zhu2023quantized is often used in QAT to learn from the FP32 model or even a much stronger, external teacher. Researchers have proposed methods (DGD

Read the complete text