WINDQuant: Weight-Informed Neural Decision-Making for Global Mixed-Precision LLM Quantization

Phong Nam Huu Nguyen, Khoi M. Le, Cong-Duy T Nguyen, Anh Tuan Luu, Thong Thanh Nguyen, Tho Quan · 2026

Read WINDQuant: Weight-Informed Neural Decision-Making for Global Mixed-Precision LLM Quantization by Phong Nam Huu Nguyen, Khoi M. Le, Cong-Duy T Nguyen, Anh Tuan Luu, Thong Thanh Nguyen, Tho Quan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26660.txt.

Introduction

Nguyen and five others present WINDQuant, a controller that assigns bit widths for squeezing large language models. Ultra-low-bit squeezing usually wrecks accuracy or demands costly retraining, while coarse sensitivity maps miss fine variation inside weight matrices. WINDQuant learns, chunk by chunk under a global storage budget, which widths and treatments each column slice gets, using reinforcement learning with activation-aware calibration. On LLaMA models it holds competitive accuracy at tiny bit widths with less tuning cost than retraining.

Catalog record

CREATORPhong Nam Huu Nguyen, Khoi M. Le, Cong-Duy T Nguyen, Anh Tuan Luu, Thong Thanh Nguyen, Tho Quan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55884
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26660

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

WINDQuant: Weight-Informed Neural Decision-Making for Global Mixed-Precision LLM Quantization Phong Nam Huu Nguyen, Khoi M. Le, Cong-Duy T Nguyen, Anh Tuan Luu, Thong Thanh Nguyen, Tho Quan Abstract Quantization is an effective approach to reduce the memory footprint and inference cost of large language models (LLMs), yet maintaining performance in the ultra-low-bit regime remains challenging. Existing post-training methods often suffer from severe accuracy degradation, while quantization-aware training requires costly retraining and additional resources. Moreover, most mixed-precision strategies rely on coarse-grained or heuristic sensitivity analysis that overlooks fine-grained variations within weight matrices. We propose WINDQuant, a reinforcement-learning-based allocation controller for ultra-low-bit LLM quantization. Rather than introducing another low-level quantization operator, WINDQuant learns how to assign bit-widths and quantization treatments to fine-grained column chunks under a global storage budget. By operating at the column-chunk level, WINDQuant enables flexible and fine-grained precision assignment within layers under a global target bit-width. The implementation combines PPO with activation-aware calibration, lightweight per-unit quantizer fitting, and explicit effective-bit accounting of the learned mixed-precision plan. Experiments on LLaMA models demonstrate that WINDQuant achieves competitive performance in ultra-low-bit settings while reducing optimization overhead relative to retraining-based approaches, highlighting reinforcement learning as a practical controller for adaptive mixed-precision quantization. † † † Corresponding author. 1 Introduction Large Language Models (LLMs) have achieved remarkable performance across reasoning, language understanding, and generation tasks 4 ; 7 . However, this progress comes at the cost of massive computational and memory requirements. Modern frontier models contain hundreds of billions to trillions of parameters, requiring terabytes of VRAM for full-precision deployment 14 . Such resource demands make direct deployment on edge devices or resource-constrained environments infeasible. Quantization has emerged as one of the most promising solutions to reduce inference cost by representing model weights and activations with lower-precision data types 10 . Despite its success, low-bit quantization remains highly challenging, especially in the sub-3-bit regime 2 . Post-Training Quantization (PTQ) methods are efficient and fast but tend to suffer severe degradation at ultra-low precision 26 . Quantization-Aware Training (QAT) methods improve robustness but require expensive retraining and often rely on knowledge distillation, significantly increasing computational overhead 17 . Moreover, most existing approaches apply uniform or coarse-grained quantization strategies, overlooking the fact that not all weights contribute equally to model performance 11 . Recent empirical findings indicate that a small fraction of salient weights disproportionately affects model accuracy 21 . Furthermore, sensitivity exhibits structured locality across layers and channels, suggesting that different substructures within a layer respond differently to quantization 11 . This observation motivates a more fine-grained and adaptive allocation strategy. Instead of treating each tensor, group, or unit as an independent local quantization problem, we view ultra-low-bit mixed-precision quantization as a global budgeted allocation problem, where assigning more precision to one unit necessarily reduces the budget available to others. In this work, we introduce WINDQuant (Weight-Informed Decision-making for Reinforcement Learning in Quantization), a reinforcement learning framework for mixed-precision quantization. We define a unit as a column-level chunk within a layer and formulate quantization as a finite-horizon sequential decision problem with a global target bit budget. At each step, the agent observes statistical, structural, and activation-aware features of a unit and selects one of seven actions: skip or quantize at 8, 4, 3, 2, 1.58, or 1 bit. The training objective combines PPO 29 with budget-aware action masking, dense step-level shaping, and an episode-end quality penalty computed from a fast perplexity proxy. This distinction is central to our formulation. Conventional PTQ and QAT methods primarily specify how a given tensor, group, or block should be quantized once the target precision is fixed. WINDQuant instead addresses the complementary allocation problem: given a pretrained model decomposed into thousands of sub-layer units, the policy decides which units should be kept at higher precision, aggressively quantized, or assigned ternary/binary operators under a global bit budget. Unlike prior RL-based approaches that operate at layer granularity 32 , WINDQuant performs fine-grained unit-level allocation, enabling more expressive and adaptive mixed-precision policies. Our framework integrates activation-aware protection mechanisms and supports multiple quantization operators spanning 1-bit to 8-bit regimes. Experiments on LLaMA models demonstrate that WINDQuant achieves competitive performance in ultra-low-bit settings while significantly reducing computational and memory overhead compared to training-based approaches. Overall, the contributions of our research can be articulated as follows: • We propose WINDQuant , a reinforcement learning–based framework that reformulates fine-grained mixed-precision quantization for large language models as a sequential decision-making problem, enabling adaptive bit-width allocation beyond static heuristic sensitivity ranking. • We demonstrate the effectiveness of the proposed approach across a range of modern large language models, including LLaMA3 (1B, 3B, 8B, 70B), showing its generality and applicability to contemporary architectures. • We show that WINDQuant reaches competitive ultra-low-bit performance on LLaMA models up to 70B parameters while avoiding full model retraining, requiring only lightweight policy optimization and local per-unit quantizer fitting. 2 Related Works 2.1 Quantization of Large Language Models Quantization methods for LLMs are commonly divided into post-training quantization (PTQ) and quantization-aware training (QAT) 18 ; 19 . PTQ approaches compress pretrained models without additional training by applying calibration and reconstruction techniques 21 ; 33 . Their primary strength lies in efficiency, but performance degrades significantly in ultra-low-bit settings, where fixed or heuristic precision assignments cannot adequately compensate for accumulated quantization error. QAT methods address this by incorporating quantization effects during training 22 , yielding stronger accuracy retention at the cost of substantial retraining overhead. Despite their complementary strengths, both paradigms typically treat precision allocation as a static optimization problem, motivating adaptive approaches that learn allocation policies automatically. 2.2 Reinforcement Learning for Neural Network Compression Reinforcement learning has emerged as a promising framework for automated model compression, framing design decisions as sequential optimization problems. HAQ 32 pioneered the use of deep deterministic policy gradient (DDPG) to determine layer-wise bit-widths for hardware-aware mixed-precision quantization without manual heuristics. ReLeQ 13 applied Q-learning to learn heterogeneous bit-width assignments, demonstrating that RL-based policies can approach near-lossless accuracy across different architectures. Beyond quantization, AMC 16 uses DDPG to automate channel pruning ratios, and N2N learning 1 applies policy gradient methods to jointly optimize architecture and compression decisions. These works share a common insight: compression decisions involve complex, interdependent trade-offs that are naturally suited to sequential decision-making formulations. However, existing RL-based quantization methods face several limitations when applied to large language models. First, operating at layer granularity leads to coarse allocation that ignores within-layer sensitivity variations. Second, reward evaluation typically requires repeated full-model inference, creating computational bottlenecks at the billion-parameter scale. Third, the action spaces and state representations used in prior work were designed for smaller vision models and do not account for the structural properties of transformer-based LLMs, such as the distinct sensitivity profiles of attention and MLP components. These limitations motivate our approach: a fine-grained, sub-layer RL formulation with efficient reward evaluation, specifically designed for large-scale transformer quantization. Unlike HAQ and ReLeQ, which assign one bit-width per layer, WINDQuant operates at the column-chunk level, enabling more precise adaptation to within-layer sensitivity. 2.3 Mixed-Precision and Sensitivity-Aware Quantization Empirical findings show that only a small fraction of salient weights dominate model accuracy, with sensitivity structured along channels rather than uniformly distributed 21 ; 11 . Recent elastic quantization operators–such as Learned Step-Size

Read the complete text