Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling
Jinhee Kim, Jae Jun An, Kang Eun Jeon, Jong Hwan Ko · 2025
Read Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling by Jinhee Kim, Jae Jun An, Kang Eun Jeon, Jong Hwan Ko (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20673.txt.
Introduction
Jinhee Kim, Jae Jun An, Kang Eun Jeon, and Jong Hwan Ko tackle the training cost of networks that must run at several numeric precision levels in one shared model. Past training repeats full data passes for each level, so cost grows in step with the number of levels offered.
Two fixes drive the work, a bias fix that aligns layer outputs across levels so shared scaling layers work without extra fine tuning. A smart sampling plan that trains each level on a small but rich subset of data. Tests on image tasks with two major network types match or beat prior accuracy while cutting training time by up to near eight fold. Code is shared for others to use and check.
Catalog record
| CREATOR | Jinhee Kim, Jae Jun An, Kang Eun Jeon, Jong Hwan Ko |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-20673
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling Jinhee Kim, Jae Jun An, Kang Eun Jeon, Jong Hwan Ko Abstract Multi-bit quantization networks enable flexible deployment of deep neural networks by supporting multiple precision levels within a single model. However, existing approaches suffer from significant training overhead as full-dataset updates are repeated for each supported bit-width, resulting in a cost that scales linearly with the number of precisions. Additionally, extra fine-tuning stages are often required to support additional or intermediate precision options, further compounding the overall training burden. To address this issue, we propose two techniques that greatly reduce the training overhead without compromising model utility: (i) Weight bias correction enables shared batch normalization and eliminates the need for fine-tuning by neutralizing quantization-induced bias across bit-widths and aligning activation distributions; and (ii) Bit-wise coreset sampling strategy allows each child model to train on a compact, informative subset selected via gradient-based importance scores by exploiting the implicit knowledge transfer phenomenon. Experiments on CIFAR-10/100, TinyImageNet, and ImageNet-1K with both ResNet and ViT architectures demonstrate that our method achieves competitive or superior accuracy while reducing training time up to 7.88×. Our code is released at this link . 1 Introduction With the explosion of highly capable yet computationally demanding deep learning models, quantization has emerged as an effective strategy for balancing performance and efficiency [ 1 , 2 , 3 , 4 ] . Despite its advantages, most existing quantization methods are optimized for a single fixed quantization precision configuration, which limits their ability to adapt dynamically to changing resource availability and deployment across various platforms of diverse memory, compute, and power specifications. This has led to a line of recent work focused on training a single model capable of supporting multiple precisions [ 5 , 6 , 7 , 8 ] , thereby enabling instant adaptation to varying resource budgets at runtime without the need for further training. In such multi-bit quantization networks, henceforth multi-bit networks , a single full-precision parent model generates multiple reduced-precision child models, thereby neutralizing the overhead of maintaining separate models for inference. By supporting multiple quantization precisions- referred to as the model’s switchable bit range , these networks enable adaptive deployment across a wide range of compute-constrained devices [ 9 , 10 , 11 , 12 ] . Figure 1 : A conceptual diagram of i) Dedicated, ii) Any-Precision, and iii) our training pipelines. \mathcal{D} and \mathcal{S} indicate the full training dataset and coreset, respectively. Although the multi-bit networks provide flexibility during inference/deployment, this advantage comes at the cost of substantial training overhead, limiting their practical adoption. The prevalent multi-bit training approach, known as Any-Precision [ 13 ] , jointly optimizes the model across a small subset of selected bit-widths, termed the trained bit range . While this approach is more efficient than training individual models across the switchable range ( Dedicated ), it still introduces considerable overhead due to the additional calibration required to enable inference at bit-widths outside the trained range ( calibrated bit range ), as shown in Fig. 1 . Specifically, calibration demands extensive computation using large amounts of training data to preserve the accuracy of untrained bit models in the calibrated range, by aligning their mismatched activation distributions. We identify that these activation mismatches across different bit-width models stem from biases in the weight distribution induced by quantization. Based on this observation, we propose a novel bias correction technique that directly controls the shift and scaling biases in the quantized weights to align distributions across the entire switchable bit-widths. This alignment enables multiple bit-width sub-networks to share a common set of batch normalization (BN) parameters, effectively eliminating the need for costly post-training calibration. Another major source of the significant training overhead is the use of the entire dataset for updating models in the trained bit range. Although coreset selection methods have been introduced to reduce the training overhead by identifying a subset of important data samples [ 14 , 15 , 16 , 17 ] , these approaches have primarily targeted single-precision model training with fixed coresets. Extending this idea to a multi-bit quantization setting, we observe that each bit-width child model can benefit from training on distinct and smaller data subsets due to implicit gradient alignment across bit-widths. Leveraging this insight, we propose a bit-wise coreset sampling method that dynamically selects informative samples individually for each child model, based on the gradients computed per bit-width. Furthermore, since sample importance changes throughout training, we periodically re-sample these coresets to reflect evolving model dynamics. The proposed sampling approach effectively reduces per-epoch computational costs while preserving strong performance through implicit cross-bit-width knowledge transfer, a phenomenon we discover for the first time. To summarize, our contributions are as follows: • Weight bias correction for activation alignment : We correct quantization-induced biases in the weight space instead of the activation space, enabling multiple child models to share normalization parameters, and in turn eliminating the need for an extra training stage. • Bit-wise coreset sampling : We propose a novel per-bit-width coreset sampling strategy that computes bit-wise importance scores using gradient-based methods, thereby reducing training redundancy in multi-bit quantization networks. • Extensive empirical validation : We demonstrate that our method consistently improves or maintains accuracy while significantly reducing training cost across diverse datasets (e.g., CIFAR-10, CIFAR-100, TinyImageNet, and ImageNet-1K) and architectures (e.g., ResNet, DeiT, Swin). Our method achieves up to 7.88× GPU hour reduction without sacrificing model utility, validating the scalability and generality of our approach. 2 Backgrounds & Related Works Multi-bit quantization networks. Algorithm 1 Batch-wise training scheme Input: Data X , label Y Output: Multi-bit network G 1: for epoch = 1, …, T do 2: for batch from X , Y do 3: for bit b in \mathcal{B} do 4: Set all layers in G to b -bit 5: Compute forward pass of G 6: Calculate gradients of G 7: end for 8: Update parameters with \sum_{\mathcal{B}}\mathcal{L}_{b} 9: end for 10: end for Unlike traditional quantized networks that are optimized for a single reduced numerical precision, multi-bit quantization networks [ 13 , 5 , 7 , 18 , 8 , 19 ] are capable of supporting multiple quantization precisions, enabling adaptive and versatile inference/deployment across a wide spectrum of compute-constrained devices [ 9 , 10 , 11 , 12 ] . The mainstream approach to training these networks involves optimizing the model for multiple precisions simultaneously, typically by the sum of loss functions corresponding to each bit-width. Formally, this is stated as shown below: \min_{\theta}\sum_{(\mathbf{x},y)\in\mathcal{S}}\sum_{b\in\mathcal{B}}\ \ \mathcal{L}(\mathbf{x},\,y,\,Q(\theta,b)), (1) where \theta\in\mathbb{R}^{d} denotes the learnable model parameter which is shared across multiple precisions; \mathcal{L}(\mathbf{x},y,Q(\theta,b)) is the loss on training sample (\mathbf{x},y) in training set \mathcal{S} ; Q(\theta,b)\in\mathbb{Z}^{d} is the quantized version of \theta at b -bit precision; and \mathcal{B} , referred to as the trained range , represents the set of all trained bit-widths . To perform this optimization in practice, the batch-wise training scheme, which interleaves parameter updates across child models corresponding to different bit-widths in a batch-wise manner (see Algorithm 1 ), is commonly adopted to promote generalization across the entire training range. Training overhead of multi-bit networks. While multi-bit training is generally more efficient than training multiple single-precision