Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning

Yoshihiko Fujisawa, Yuma Ichikawa, Yudai Fujimoto, Akira Sakai, Katsuki Fujisawa · 2026

Read Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning by Yoshihiko Fujisawa, Yuma Ichikawa, Yudai Fujimoto, Akira Sakai, Katsuki Fujisawa (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24058.txt.

Introduction

Fujisawa and four co-authors offer LoRDBA, a small adapter for tuning large language models on devices in unmerged mode, where it stays a live side branch. Both low-rank factors become binary sign carriers, sized by light per-channel scales. The dense floating-point branch thus becomes two sign-accumulation products with scaling between them. Tests show it beats low-bit rivals at matched size and matches fp16 LoRA quality in some cases. Prefill delay rises at most 8 percent at rank 16, despite a tenfold smaller footprint.

Catalog record

CREATORYoshihiko Fujisawa, Yuma Ichikawa, Yudai Fujimoto, Akira Sakai, Katsuki Fujisawa
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE113405
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24058

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning Yoshihiko Fujisawa, Yuma Ichikawa, Yudai Fujimoto, Akira Sakai, Katsuki Fujisawa Abstract On-device adaptation of large language models commonly keeps a quantized base model frozen while training and deploying a small, task-specific LoRA adapter. In the unmerged adapter-mode setting, however, the adapter is more than a compact storage module; it introduces an additional dense floating-point branch, maintains a trainable state for local updates, and acts as a unit of communication and hot-swapping. We introduce LoRDBA, a LoRA-compatible adapter that replaces both low-rank factors with binary sign carriers while representing magnitudes through lightweight, channel-wise scales, converting the dense adapter branch into two sign-accumulation matrix multiplications interleaved with channel-wise scaling. A finite-sample analysis shows that reconstruction quality is governed by the residual-to-magnitude ratio of the original LoRA factors. In adapter-mode experiments, LoRDBA outperforms low-bit baselines at matched model sizes while matching fp16 LoRA quality in selected regimes. The unmerged adapter incurs at most 8 % 8\% prefill latency overhead at matched rank r = 16 r{=}16 despite an over 10 × 10\times reduction in adapter footprint, with moderate training memory overhead of approximately 1.6 × 1.6\times that of fp16 LoRA. 1 Introduction Large language models are increasingly adapted using parameter-efficient updates rather than full fine-tuning. LoRA enables this paradigm by freezing pretrained weights and injecting low-rank adapter matrices into Transformer projections, reducing the number of trainable parameters ( Hu et al. 2022 ) . QLoRA extends this approach to low-bit settings by backpropagating through a frozen quantized base model into LoRA adapters ( Dettmers et al. 2023 ) . This base-plus-adapter design is attractive for on-device personalization, privacy-preserving adaptation, and communication-efficient update distribution; a compact low-bit base can remain fixed while task-specific adapters are trained, exchanged, or selected. Recent multi-adapter serving systems, including S-LoRA and Punica, demonstrate that practical deployment often treats adapters as swappable runtime objects rather than as weights that are always merged into the base model ( Sheng et al. 2024 ; Chen et al. 2024 ) . In practical low-bit deployments, merging a trained adapter into a quantized base requires materializing full-precision weights or requantizing per task, forfeiting hot-swap capability; it is therefore preferable to serve the adapter as an unmerged side branch ( Frantar et al. 2022 ; Lin et al. 2024 ) . This unmerged branch creates a distinct bottleneck: the base projection runs on fused low-bit kernels while the adapter uses dense fp16 factors. An adapter compressor for this regime must preserve accuracy, reduce unmerged computation cost, and avoid inflating the local training state. Figure 1 : LoRDBA (red) Pareto-dominates FP16 LoRA and LoRAQuant at matched adapter sizes on LLaMA-2-7B ; cf. Table 1 . Figure 2 : Adapter-mode LoRA versus LoRDBA. The base model is stored in a low-bit codebook and executed with a fused dequantization–matrix multiplication kernel, while the adapter is served in an unmerged form. (a) Standard LoRA introduces two dense fp16 factors and requires a separate fp16 matrix multiplication branch. (b) LoRDBA replaces both factors with sign matrices coupled by three channel-wise scale vectors, reducing the adapter forward pass to two sign-accumulation matrix multiplications. We introduce LoRDBA, a low-rank double-binary adapter designed for unmerged adapter-mode deployment. LoRDBA replaces both LoRA factors with binary sign carriers and augments them with a compact scale envelope along the input, rank, and output axes. Binary signs encode directional information while lightweight channel-wise scales capture magnitudes; the quantized base model remains frozen and unchanged. LoRDBA is trained end-to-end using a smooth-sign straight-through estimator and exported as a near-binary adapter. At inference, it replaces the dense adapter branch with binary-weight accumulation interleaved with scaling. We analyze when this binary reconstruction is accurate, showing that the key quantity is the residual-to-magnitude ratio of the floating-point LoRA factors. Unlike BitDelta ( Liu et al. 2024a ) , which binarizes the full \Theta(NM) delta discarding low-rank structure, and LoRAQuant ( Mirzaei et al. 2025 ) , which retains mixed-precision SVD components, LoRDBA commits both rank- R factors to \pm 1 and trains end-to-end, preserving the low-rank adapter format. We evaluate LoRDBA under the adapter-mode protocol—adapter never merged; storage, latency, and training memory measured separately. Across LLaMA-based math reasoning and summarization benchmarks, LoRDBA improves the accuracy–size trade-off over all evaluated low-bit baselines as previewed in Figure 1 . 2 Preliminaries Notation. Uppercase Roman letters denote matrices; lowercase boldface denotes vectors. \{\pm 1\}^{p\times q} is the set of sign matrices. \|\cdot\|_{F} and \|\cdot\|_{\text{op}} are the Frobenius and operator norms. \diag({\bm{v}}) constructs a diagonal matrix; \sign(U) is entry-wise sign with \sign(0){=}{+}1 . The sub-Gaussian Orlicz norm is \|X\|_{\psi_{2}}\coloneqq\inf\{t>0:{\mathbb{E}}[\exp(X^{2}/t^{2})]\leq 2\} ( Vershynin 2018 ) . LoRA. Given a frozen effective weight matrix W_{0}\in{\mathbb{R}}^{N\times M} , LoRA reparameterizes the fine-tuning increment as \Delta W=AB^{\top}\in{\mathbb{R}}^{N\times M},~~A\in{\mathbb{R}}^{N\times r},~B\in{\mathbb{R}}^{M\times r}, (1) where r\ll\min(N,M) is the adapter rank. Following the column-vector convention, the adapted forward pass for an input {\bm{x}}\in{\mathbb{R}}^{N} is {\bm{y}}=W_{0}^{\top}{\bm{x}}+B(A^{\top}{\bm{x}})\in{\mathbb{R}}^{M} . Thus, each adapted projection incurs two adapter GEMMs with an aggregate arithmetic complexity of \mathcal{O}(r(N+M)) per token and stores 16r(N+M) bits when both factors are kept in fp16. The base weights remain fixed throughout adaptation. Adapter-mode deployment. We assess LoRDBA in a QLoRA-style on-device regime ( Dettmers et al. 2023 ; Sheng et al. 2024 ; Chen et al. 2024 ) . Let W_{\mathrm{fp}} denote the original full-precision pretrained weight, and let W_{Q} be its stored low-bit codebook representation. The effective deployed base weight is \widetilde{W}_{0}\coloneqq\mathrm{dequant}(W_{Q}), which generally differs from W_{\mathrm{fp}} due to quantization error. In this setting, the base model is never materialized in fp16, and the trained adapter is served in an unmerged state. The adapted forward pass is {\bm{y}}=\widetilde{W}_{0}^{\top}{\bm{x}}+\Delta W^{\top}{\bm{x}}~=~\mathrm{dequant}(W_{Q})^{\top}{\bm{x}}+\Delta W^{\top}{\bm{x}}. (2) This factorizes the operation into a base GEMM—executed by a fused dequantize-matmul kernel when the base is quantized—and an independent adapter side branch. The device-side workload is characterized by the following four conditions: (A1) Frozen base: the base model is held fixed and is not updated. In typical on-device deployments it is stored in a low-bit format such as NF4 or GPTQ, but the conditions themselves do not mandate a specific base precision. (A2) Adapter-only training: optimizer states for the base weights are never materialized. (A3) unmerged serving: the adapter is evaluated as a side branch rather than being merged into a task-specific base weight. (A4) Hot-swappability: multiple task-specific adapters can share the same on-device base ( Sheng et al. 2024 ; Chen et al. 2024 ) . The unmerged regime decouples base and adapter kernels: the base GEMM runs at the codebook’s native precision while the adapter GEMM runs at its own bit width. Reducing the adapter bit width is thus the primary mechanism for improving the bandwidth-compute envelope without violating the no-merging constraint. Bits per weight (BPW). Following standard practice ( Mirzaei et al. 2025 ) , we quantify adapter compression in bits per weight (BPW) relative to the reference LoRA parameter count r_{0}(N+M) . An fp16 LoRA adapter of rank r thus has BPW =16r/r_{0} , and the full-size reference at rank r_{0} has BPW =16 . The LoRDBA-specific BPW metrics are defined in Section 3.1 once the adapter parameterization is introduced. 3 Method 3.1 Low Rank Double Binary Adapter LoRDBA parameterizes the LoRA delta with two binary factors and

Read the complete text