BitRL: Reinforcement Learning with 1-bit Quantized Language Models for Resource-Constrained Edge Deployment
Md. Ashiq Ul Islam Sajid, Mohammad Sakib Mahmood, Md. Tareq Hasan, Md Abdur Rahim, Rafat Ara, Md. Arafat Hossain · 2026
Read BitRL: Reinforcement Learning with 1-bit Quantized Language Models for Resource-Constrained Edge Deployment by Md. Ashiq Ul Islam Sajid, Mohammad Sakib Mahmood, Md. Tareq Hasan, Md Abdur Rahim, Rafat Ara, Md. Arafat Hossain (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-24273.txt.
Introduction
Sajid and five colleagues introduce BitRL, a framework for reinforcement learning agents on 1-bit models for edge devices. Large models confine agents to the cloud, raising latency and privacy concerns, while edge hardware cannot carry their memory and energy demands. Using BitNet b1.58 with ternary weights, BitRL cuts memory 10 to 16 times and energy use 3 to 5 times against baselines. It keeps 85 to 98 percent of performance. The paper adds theory on quantization as structured perturbation, convergence bounds for policy gradients, and the exploration-stability trade-off.
Catalog record
| CREATOR | Md. Ashiq Ul Islam Sajid, Mohammad Sakib Mahmood, Md. Tareq Hasan, Md Abdur Rahim, Rafat Ara, Md. Arafat Hossain |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27340 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.24273
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
BitRL: Reinforcement Learning with 1-bit Quantized Language Models for Resource-Constrained Edge Deployment Md. Ashiq Ul Islam Sajid, Mohammad Sakib Mahmood, Md. Tareq Hasan, Md Abdur Rahim, Rafat Ara, Md. Arafat Hossain Abstract The deployment of intelligent reinforcement learning (RL) agents on resource-constrained edge devices remains a fundamental challenge due to the substantial memory, computational, and energy requirements of modern deep learning systems. While large language models (LLMs) have emerged as powerful cognitive architectures for decision-making agents, their multi-billion parameter scale confines them to cloud-based deployment, raising concerns about latency, privacy, and connectivity dependence. We introduce BitRL, a comprehensive framework for building RL agents using 1-bit quantized language models that enables practical on-device learning and inference under severe resource constraints. Leveraging the BitNet b1.58 architecture with ternary weights { − 1 , 0 , + 1 } \{-1,0,+1\} and the bitnet.cpp optimized inference stack, BitRL achieves 10–16 × \times memory reduction and 3–5 × \times energy efficiency improvements over full-precision baselines while maintaining 85–98% of task performance across diverse benchmarks. We provide theoretical analysis characterizing quantization as structured parameter perturbation, derive convergence bounds for quantized policy gradients under frozen-backbone architectures, and identify the exploration-stability trade-off inherent to extreme quantization. Our framework is the first to systematically integrate 1-bit quantized language models with reinforcement learning for edge deployment, with real-world validation on commodity hardware. I Introduction Reinforcement learning (RL) for edge deployment faces fundamental challenges due to the high memory, computation, latency, privacy, and energy costs of cloud-dependent and large-model-based systems [ 18 , 9 ] . While large language models (LLMs) enhance agent reasoning and decision-making [ 17 ] , their scale makes them impractical for resource-constrained devices, motivating extreme model compression. This work introduces BitRL, which leverages a frozen 1-bit quantized language model based on BitNet b1.58, where weights take ternary values \{-1,0,+1\} , drastically reducing memory and compute while serving as a compact state encoder [ 19 , 20 ] . A lightweight trainable head learns the policy and value functions, enabling efficient on-device learning without cloud reliance. Despite severe quantization, BitRL retains 85–98% of full-precision performance while achieving 10–16 \times model size reduction and 3–5 \times energy efficiency gains, though value function learning remains the primary bottleneck due to error accumulation in bootstrapped estimates [ 10 , 13 ] . Extreme quantization introduces biased and noisy gradients that can destabilize RL unless carefully managed through quantization-aware optimization and constrained learning dynamics [ 12 , 8 ] . The key contributions of this work are as follows. First, we present the first systematic framework integrating 1-bit quantized LLMs with RL for edge deployment, going significantly beyond the 4–8 bit range explored in prior work. Second, we provide a theoretical analysis characterizing ternary quantization as structured parameter perturbation with convergence bounds for quantized policy gradients (Section IV ). Third, we identify the value estimation bottleneck as the critical failure mode under extreme quantization, with hybrid-precision architectures as a practical remedy. Fourth, we present comprehensive empirical validation across nine diverse benchmarks with real-world deployment on Raspberry Pi 4 hardware. II Literature Review II-A Neural Network Quantization The quantization of neural networks has evolved from early binary networks [ 24 ] and XNOR-Net [ 25 ] through ternary representations to modern methods. GPTQ and AWQ demonstrate effective 4-bit LLM quantization through layer-wise calibration and activation-aware weight selection [ 21 , 22 ] . Additional approaches include SpQR [ 1 ] ,SqueezeLLM [ 2 ] , OmniQuant [ 3 ] ,and QuIP/QuIP# [ 4 , 7 ] . The BitNet family represents a return to extreme quantization at scale, with BitNet b1.58 achieving near-parity with FP16 models at the 3B+ parameter scale through quantization-aware training, straight-through estimators, and optimized inference infrastructure [ 19 , 20 , 16 ] . II-B Quantization in Reinforcement Learning Despite advances in supervised learning quantization, its application to reinforcement learning has received comparatively less systematic attention. QuaRL provides the most comprehensive early study, finding that RL policies can typically be quantized to 6–8 bits post-training with less than 5% performance loss, while value functions exhibit greater sensitivity to quantization than policies. This asymmetry is attributed to the bootstrapping nature of value estimation, where quantization errors compound across temporal-difference updates. Quantization-aware training during RL was shown to improve robustness, and importantly, quantization noise may increase exploration by maintaining diverse action distributions, potentially improving sample efficiency in sparse-reward environments. Recent investigations show that 4-bit quantized PPO can match or exceed full-precision RLHF baselines, with quantization-induced noise enhancing exploration through increased policy entropy [ 12 ] . The theoretical explanation connects to classical results showing that policy gradient methods with entropy regularization improve exploration. Domain-specific applications include action-quantized offline RL for robotic skill learning [ 5 ] , studies showing that low-bit quantization favors undertrained LLMs [ 11 ] ,analyses of quantization and pruning effects on RL models [ 10 ] , the impact of quantization on large reasoning model RL [ 13 ] , and FPGA-optimized engines [ 14 ] . BitDelta suggests fine-tunes may need only one bit of precision [ 6 ] . However, none of these works address 1-bit quantization with LLM backbones for on-device RL training. Table I shows BitRL’s unique position. TABLE I: Positioning of BitRL vs. Related Work Method Bits Model RL Train Edge HW LLM QuaRL 4–8 MLP/CNN Post-hoc No No ParetoQ [ 12 ] 4 LLM RLHF No Yes QForce-RL [ 14 ] 4–8 MLP Online FPGA No Action-Q [ 5 ] N/A † MLP Offline No No BitDelta [ 6 ] 1 LLM No No Yes BitRL (ours) 1.58 LLM Online RPi4 Yes † Action-Q quantizes the action space, not model weights. II-C LLM-Based Agents and Edge AI Prompting-based agents like ReAct [ 23 ] and Reflexion leverage frozen LLMs for decision-making but require GPT-4-scale models and cloud deployment. Fine-tuning approaches (SayCan, Toolformer, WebGPT) and LLM-augmented RL systems (ELLM, DEPS, Voyager) use LLMs to aid RL in various ways: generating reward functions from task descriptions, decomposing complex tasks into subgoals, and enabling curriculum learning for open-ended exploration. BitRL differs fundamentally by quantizing the LLM itself to 1-bit for edge deployment, treating it as an efficient state encoder rather than using LLMs to aid the RL process. This architectural choice prioritizes on-device deployment and resource efficiency over the reasoning capabilities typically associated with large-scale language models. Edge AI efforts including TinyML [ 9 , 15 ] and federated learning [ 18 ] address resource constraints but do not combine LLM representations with on-device RL training. Efficient architectures such as MobileNets, EfficientNets, and SqueezeNet focus primarily on inference efficiency rather than the representational richness needed for complex decision-making. The frozen backbone with trainable heads architecture in BitRL naturally supports continual learning by allowing task-specific heads to be added without modifying the shared representation, accepting 5–15% accuracy reduction for 10 \times memory compression and 4 \times energy reduction—a favorable trade-off when cloud connectivity is unavailable. III Methodology Fig. 1: BitRL system overview showing text serialization, frozen BitNet backbone, and trainable policy and value heads. III-A Framework Architecture We model all tasks as Markov decision processes (\mathcal{S},\mathcal{A},P,R,\gamma) , optimizing via Proximal Policy Optimization (PPO) with the clipped surrogate objective: \mathcal{L}^{\text{CLIP}}(\phi)=\mathbb{E}_{t}\!\left[\min\!\left(r_{t}(\phi)\hat{A}_{t},\;\operatorname{clip}(r_{t}(\phi),1\!-\!\epsilon,1\!+\!\epsilon)\hat{A}_{t}\right)\right], (1) where r_{t}(\phi)=\pi_{\phi}(a_{t}|s_{t})/\pi_{\phi_{\text{old}}}(a_{t}|s_{t}) and \hat{A}_{t}