Tequila: Trapping-free Ternary Quantization for Large Language Models
Hong Huang Decheng Wu Rui Cen Guanghua Yu Zonghang Li, Kai Liu Jianchen Zhu Peng Chen Xue Liu Dapeng Wu · 2025
Read Tequila: Trapping-free Ternary Quantization for Large Language Models by Hong Huang Decheng Wu Rui Cen Guanghua Yu Zonghang Li, Kai Liu Jianchen Zhu Peng Chen Xue Liu Dapeng Wu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23809.txt.
Introduction
Shrinking a language model's weights to three values, minus one, zero, or one, makes edge hardware happy and accuracy sad. Hong Huang and 9 co-authors trace the damage to what they call deadzone trapping. Many weights sit stuck at the deadzone boundary, fed only noisy gradients that cannot push them free. Their method, Tequila, reuses those trapped weights as dynamic biases instead.
Repurposed weights then carry a continuous signal forward and receive direct gradients backward, at nearly zero inference cost. Across five benchmarks Tequila beat leading ternary methods. On ARC it gained over four points on the best rival and nearly matched full precision, with a threefold inference speedup.
Catalog record
| CREATOR | Hong Huang Decheng Wu Rui Cen Guanghua Yu Zonghang Li, Kai Liu Jianchen Zhu Peng Chen Xue Liu Dapeng Wu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-23809
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Tequila: Trapping-free Ternary Quantization for Large Language Models Hong Huang Decheng Wu Rui Cen Guanghua Yu Zonghang Li, Kai Liu Jianchen Zhu Peng Chen Xue Liu Dapeng Wu Abstract Quantization techniques are essential for the deployment of Large Language Models (LLMs) on edge devices. However, prevailing methods often rely on mixed-precision multiplication that lacks efficient hardware support, making it not feasible. Ternary weight quantization addresses this by constraining weights to {-1, 0, 1}, replacing expensive multiplications with hardware-efficient additions. However, such aggressive compression leads to significant accuracy degradation, even after costly quantization-aware training with massive data. We identify the core issue as deadzone trapping: a large number of weights are trapped at the deadzone boundary. This occurs because these weights receive only noisy, uninformative gradients, preventing stable escape from the deadzone and severely impeding model capacity and optimization. To address this issue, we propose Tequila , a trapping-free quantization optimization method that reactivates deadzone-trapped weights by repurposing them as dynamic biases. This allows the repurposed weights to provide a continuous signal in the forward pass and, critically, receive direct, meaningful gradient signals during backpropagation, thereby enhancing model capacity and optimization with nearly zero inference overhead. Extensive evaluations demonstrate that Tequila outperforms state-of-the-art (SOTA) ternary quantization methods across five benchmarks. Specifically, on the ARC benchmark, it achieves > 4 % >4\% accuracy gain over the SOTA baseline, nearly matching full-precision performance (within < 1 % <1\% gap) with an 3.0 × 3.0\times inference speedup. Consequently, Tequila offers a highly practical and efficient implementation for the deployment of advanced LLMs in resource-constrained environments. The code is available at https://github.com/Tencent/AngelSlim . 1 Introduction Recent advancements in large language models (LLMs) ( Wu et al. 2023 ; Floridi & Chiriatti 2020 ; Zhang et al. 2022 ) have demonstrated remarkable success across a wide range of applications, from conversational chatbots to creative writing. However, growing concerns over data privacy, the need for offline functionality, and the high cost of large-scale cloud deployment ( Yao et al. 2024 ; Liagkou et al. 2024 ) have necessitated the deployment of these models directly on edge devices, which are usually resource-constrained. Quantization ( Dettmers et al. 2021 ; Dettmers et al. 2022 ; Lin et al. 2023 ; Frantar et al. 2022 ) has emerged as a promising technique to achieve this goal, reducing model size and computational requirements by representing model weights with lower precision. However, most existing quantization methods ( Kwon et al. 2022 ; Dettmers et al. 2024 ; Liu et al. 2023 ; Frantar et al. 2022 ) are primarily designed for server-grade GPUs that support specialized hardware features, such as mixed-precision multiplication ( Lin et al. 2023 ) . These methods are often incompatible with a wide range of edge and mobile hardware, highlighting a critical need for hardware-friendly quantization approaches that remain effective across diverse and resource-constrained devices. Ternary quantization ( Li et al. 2016 ; Liu & Liu 2023 ; Ma et al. 2025 ; Wang et al. 2023 ; Wang et al. 2025a ) offers a promising path for on-device deployment of LLMs by constraining weights to \{-1,0,+1\} . This method reduces matrix multiplication to efficient additions, as illustrated in Fig. 2 , which are widely supported by most hardware. However, such aggressive compression introduces significant information loss, often leading to severe accuracy degradation even after costly Quantization-Aware Training (QAT) on massive datasets. For instance, BitNet ( Ma et al. 2025 ; Wang et al. 2025a ) consumes 4T tokens during QAT but still fails to match full-precision performance. Thus, the dual problems of performance degradation and prohibitive training overheads persist as the fundamental barriers to the development of effective ternary LLMs. Figure 1: (Top) Deadzone Trapping in Ternary Quantization : Dead weights are trapped in a cycle of ineffective oscillation around the deadzone boundary due to noisy and uninformative gradients, significantly impeding model capacity and optimization, causing a significant accuracy drop ( >5\% ) versus the full-precision. (Bottom) Reactivation Strategy of Tequila : Our Tequila reactivates dead weights as dynamic biases, providing direct and meaningful gradients for stable escapes, enhancing model capability and optimization, achieving only a minor accuracy gap ( <1\% ). In this paper, we identify the key source of these challenges as deadzone trapping , where a large number of weights are trapped at the deadzone boundary. Deadzone trapping arises from the aggressive nature of ternary quantization, which creates a vast deadzone that quantizes a large proportion of weights to zero. During training, these ”dead” weights receive only noisy, uninformative gradients from the Straight-Through Estimator (STE), preventing effective optimization. Lacking consistent directional signals, these weights cannot escape the deadzone stably and are accumulated at the deadzone boundary, as shown in Fig. 1 (top). This results in a cycle of ineffective oscillation, rendering these weights permanently inactive and severely impeding model capacity and optimization. To address the deadzone trapping issue, we propose Tequila , a trapping-free Te rnary qu antization method for la rge language models. Our key idea is to reactivate dead weights by repurposing them as dynamic biases. This provides continuous signals to the output, significantly enhancing the model capacity, as shown in Fig. 2 (c). More importantly, these weights receive direct and meaningful gradients via the bias terms, enabling them to escape the deadzone stably, as shown in Fig. 1 (bottom). Crucially, these biases can be computed offline, introducing nearly zero inference overhead. We evaluate the effectiveness and efficiency of Tequila on five common benchmarks using LLaMA 3.2 models ( Touvron et al. 2023 ) . Our experiments demonstrate that Tequila outperforms all state-of-the-art (SOTA) ternary methods across all benchmarks while requiring only limited training data. For instance, when trained on just 10B tokens, Tequila achieves a >4\% accuracy gain over the SOTA baseline on the ARC benchmark, nearly matching full-precision performance (within <1\% gap). Furthermore, it delivers a significant 3\times inference speedup on an Intel 8263C CPU, verifying that Tequila fully preserves the hardware efficiency of ternary quantization. Therefore, Tequila offers a practical and efficient solution for deploying LLMs on resource-constrained devices. 2 Background and Challenge 2.1 Ternary Quantization Ternary quantization is an extreme compression technique that constrains model weights to ternary values \{-1,0,+1\} . This representation converts the computationally expensive weight-input matrix multiplication into input-inner addition, as shown in Fig. 2 (a), offering significant hardware advantages. Given a full-precision weight vector W=(w_{1},\dots,w_{n}) , the general form of the ternary quantization function Q(\cdot) is defined as: Q(W)=\hat{W}\alpha,\quad\hat{w}_{i}=\begin{cases}+1,&\text{if }w_{i}\geq\Delta;\\ 0,&\text{if }|w_{i}|<\Delta;\\ -1,&\text{if }w_{i}\leq-\Delta,\end{cases} (1) where \hat{W}=(\hat{w}_{1},\dots,\hat{w}_{n}) is ternary weights, \alpha is a scaling factor and \Delta is a threshold parameter. A significant body of research focuses on determining optimal values for \alpha and \Delta . For instance, the TWN ( Li et al. 2016 ) assumes the weight distribution follows a standard Gaussian distribution. It approximates the optimal threshold as \Delta^{*}\approx\frac{0.75}{n}\sum_{i=1}^{n}|w_{i}| and derives a closed-form solution for \alpha by minimizing ||W-\alpha\hat{W}||^{2} . Subsequent methods ( Chen et al. 2024 ; Liu et al. 2025 ; Zhu et al. 2016 ) forgo this distributional hypothesis and instead treat \alpha or \Delta as trainable parameters learned during optimization. In recent open-source ternary LLMs ( Ma et al. 2025 ; Team et al. 2025 ; Kaushal