TTAQ: Towards Stable Post-training Quantization in Continuous Domain Adaptation

Junrui Xiao, Zhikai Li, Lianwei Yang, Yiduo Mei, Qingyi Gu · 2024

Read TTAQ: Towards Stable Post-training Quantization in Continuous Domain Adaptation by Junrui Xiao, Zhikai Li, Lianwei Yang, Yiduo Mei, Qingyi Gu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-09899.txt.

Introduction

Junrui Xiao and co-authors present TTAQ in 2024, a method to keep shrunk neural models accurate when test data keep changing. Shrinking a model to fewer bits cuts hardware cost, but standard methods fail when data streams shift over time.

TTAQ adds three parts. One limits how input noise spreads through the weights, one trains the model to give stable answers for the same sample, and one adjusts outputs for class imbalance in the stream. Tests on several datasets show gains over rivals, with a large drop in error for two-bit models on a corrupted image set.

Catalog record

CREATORJunrui Xiao, Zhikai Li, Lianwei Yang, Yiduo Mei, Qingyi Gu
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-09899

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TTAQ: Towards Stable Post-training Quantization in Continuous Domain Adaptation Junrui Xiao, Zhikai Li, Lianwei Yang, Yiduo Mei, Qingyi Gu Abstract Post-training quantization (PTQ) reduces excessive hardware cost by quantizing full-precision models into lower bit representations on a tiny calibration set, without retraining. Despite the remarkable progress made through recent efforts, traditional PTQ methods typically encounter failure in dynamic and ever-changing real-world scenarios, involving unpredictable data streams and continual domain shifts, which poses greater challenges. In this paper, we propose a novel and stable quantization process for test-time adaptation (TTA), dubbed TTAQ, to address the performance degradation of traditional PTQ in dynamically evolving test domains. To tackle domain shifts in quantizer, TTAQ proposes the Perturbation Error Mitigation (PEM) and Perturbation Consistency Reconstruction (PCR). Specifically, PEM analyzes the error propagation and devises a weight regularization scheme to mitigate the impact of input perturbations. On the other hand, PCR introduces consistency learning to ensure that quantized models provide stable predictions for same sample. Furthermore, we introduce Adaptive Balanced Loss (ABL) to adjust the logits by taking advantage of the frequency and complexity of the class, which can effectively address the class imbalance caused by unpredictable data streams during optimization. Extensive experiments are conducted on multiple datasets with generic TTA methods, proving that TTAQ can outperform existing baselines and encouragingly improve the accuracy of low bit PTQ models in continually changing test domains. For instance, TTAQ decreases the mean error of 2-bit models on ImageNet-C dataset by an impressive 10.1%. 1 Introduction In recent years, deep neural networks (DNNs) have achieved remarkable progress in various computer vision tasks, such as image classification Chen et al. 2021 ; Dosovitskiy et al. 2020 ; Lanchantin et al. 2021 , object detection Carion et al. 2020 ; Xiao et al. 2022 ; Xiao et al. 2023a ; Liang et al. 2020 , and semantic segmentation Strudel et al. 2021 ; Zheng et al. 2021 ; Liu et al. 2021a , thanks to the success of large-scale models trained on extremely large datasets. However, increasing demands for computation, memory, and storage present challenges in deploying these models in resource-limited devices or environments. Consequently, various model compression techniques have been proposed to address these resource constraints and accelerate inference, including pruning He et al. 2018 ; Yu et al. 2022 , knowledge distillation Lin et al. 2022 ; Shi et al. 2020 ; Romero et al. 2015 ; Xu et al. 2022 , structured factorization and quantization Li et al. 2022a ; Li et al. 2022b ; Liu et al. 2021c ; Xiao et al. 2023b ; Xu et al. 2023 . Figure 1: Illustration of the problem: (a) The distribution of the 1st to 10th channels of the activation in ResNet-18’s first convolutional layer shows a significant shift. The quantization parameters calibrated based on the original distribution are inaccurate, resulting in performance degradation; (b) A toy example illustrating the bias introduced by class-imbalanced samples. The solid gray line represents the unbiased classifier with the optimal decision boundary, while the dotted line depicts the learned biased classifier on imbalanced streaming samples, which also hinders the accuracy. Model quantization has garnered significant attention as an effective technique that converts the weights and activations of large models from full-precision to low-bit fixed-point representations. Most quantization works can be divided into two categories: quantization-aware training (QAT) Nagel et al. 2022 ; Choi et al. 2018 ; Esser et al. 2019 and post-training quantization (PTQ) Li et al. 2022b ; Lin et al. 2021 ; Liu et al. 2021c ; Yuan et al. 2022 ; Liu et al. 2021b . By retraining the entire labeled dataset, QAT typically possesses higher accuracy but requires significant memory consumption and GPU efforts. On the contrary, PTQ only requires a tiny unlabeled calibration set to quantize the pre-trained parameters without retraining, and thus is regarded as a practical solution in widespread scenarios. Existing PTQ methods demonstrate great potential in terms of both accuracy and efficiency, as they can achieve good prediction accuracy on domain-invariant Wang et al. 2022a test sets even when the quantized weights are reduced to 4 bits. However, in real-world applications, domain shifts frequently occur at test-time due to corruptions, changes in weather conditions, camera sensor differences, and other factors. When there is an inconsistency between the distributions of the test and training data, the conventional PTQ method, which is calibrated on a fixed distribution, experiences severe performance degradation. Moreover, the data collected from edge devices, such as Internet of Things (IoT) products or wearable devices, arrives in a sequential manner, which presents additional challenges. This is due to the limited computational resources and the need for real-time processing, which makes it impractical to retrain all parameters using streaming data. To perform online adaptation on the test stream characterized by an unpredictable distribution and provide instant predictions for incoming test samples, Continual Test-Time Adaptation (CTTA) Wang et al. 2022b ; Gong et al. 2022 ; Döbler et al. 2023 ; Song et al. 2023 has emerged as a robust solution. CoTTA introduces an Exponentially Moving Average (EMA) Cai et al. 2021 teacher-student architecture to self-train all parameters using pseudo labels, which necessitates substantial memory during adaptation. To enhance efficiency, EATA Niu et al. 2022 employs an active sample selection criterion to identify reliable and non-redundant samples for updating the parameters of the batch normalization layer. However, current TTA approaches have focused primarily on full-precision models. The issue of how to achieve stable quantization in CTTA has not yet been explored thoroughly. In this work, we first empirically reveal the potential problems of PTQ in the CTTA setting from two perspectives. Firstly, we observe that domain shifts directly cause alterations in the activation distribution, resulting in inaccurate calibrated quantization parameters (e.g., scale and zero point) and an increase in quantization errors, as illustrated in Figure 1 (a). Secondly, when adapting on streaming data with an unpredictable distribution, imbalanced class distributions can easily arise, as shown in Figure 1 (b), leading to a decrease in the model’s generalization capability and resulting in catastrophic forgetting of the minor class McCloskey and Cohen 1989 . Motivated by the analysis above, we propose a stable PTQ scheme in continuous domain adaptation, namely TTAQ, which considers both the robustness of quantization parameters and the imbalance of classes in adaptation. First, we conduct an analysis of the error propagation stemming from perturbations, considering the signal-to-noise ratio, and then propose the Perturbation Error Mitigation (PEM), which introduces a weight-regularization scheme to mitigate the effect of input perturbations. Second, to further enhance the robustness, we propose Perturbation Consistent Reconstruction (PCR), which adds small perturbations during the block-wise reconstruction process and encourages quantized model can have the same output through perturbation consistency loss. Finally, we introduce Adaptive Balanced Loss (ABL) to address the class imbalance issue caused by unpredictable data streams. ABL reweights the samples based on class-wise frequency and adjusts logits by the accumulated gradients of each class, which effectively promotes the learning of unbiased classifiers. We summarize our main contributions as follows: • To the best of our knowledge, the proposed TTAQ is the first attempt to achieve stable post-training quantization in continuous domain adaptation by considering both quantization robustness and class

Read the complete text