Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training

Chengqian Zhang, Wei Zhu, Kyumin Lee · 2026

Read Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training by Chengqian Zhang, Wei Zhu, Kyumin Lee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18822.txt.

Introduction

Chengqian Zhang, Wei Zhu, and Kyumin Lee bridge full fine-tuning and low-rank adaptation for post-training language models. Full tuning works well for reasoning but eats graphics memory and training cost. Low-rank methods save cost yet trail full tuning on hard reasoning tasks. Their Hybrid-LoRA mixes the two approaches across training stages. Tests show it closes much of the gap, beating the best low-rank baseline by about four percent on average.

Catalog record

CREATORChengqian Zhang, Wei Zhu, Kyumin Lee
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40759
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.18822

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training Chengqian Zhang, Wei Zhu, Kyumin Lee Abstract Post-training has become essential for adapting large language models (LLMs) to complex downstream behaviors, including instruction following, preference alignment, and multi-step reasoning. Reinforcement learning with verifiable rewards (RLVR) has recently emerged as a particularly effective post-training paradigm for improving reasoning capabilities, with critic-free algorithms such as GRPO and GSPO enabling scalable optimization. However, RLVR post-training with full fine-tuning (FFT) requires substantial GPU memory and incurs high training costs. Although parameter-efficient fine-tuning (PEFT) methods, such as Low-Rank Adaptation (LoRA), effectively reduce computational costs, they often suffer from a noticeable performance gap compared to full fine-tuning in post-training for complex reasoning tasks. In this paper, we propose Hybrid-LoRA, an efficient hybrid post-training framework that selectively applies full fine-tuning to a small subset of modules less suited to low-rank adaptation, while adapting the remaining components with LoRA. We introduce a novel Hybrid-LoRA Score to rank candidate modules according to their sensitivity to low-rank adaptation under a fixed parameter budget. Experiments show that Hybrid-LoRA closely matches full fine-tuning performance under a 10% full fine-tuning module budget, with the remaining candidate modules adapted by LoRA, consistently outperforming four state-of-the-art PEFT post-training baselines, achieving improvements of up to 5.65% and on average 4.36% over the best baseline. 1 Introduction Large language models (LLMs) have demonstrated remarkable proficiency in complex reasoning tasks through post-training, including supervised fine-tuning (SFT) and reinforcement learning (RL)-based approaches such as reinforcement learning from human feedback (RLHF) and reinforcement learning with verifiable rewards (RLVR) ( 11 ; 14 ) . Recent critic-free RLVR algorithms, such as Group Relative Policy Optimization (GRPO) and Group Sequence Policy Optimization (GSPO), have become widely used for reasoning-oriented post-training, as they optimize policies using verifiable rewards without requiring a separate critic model. These methods have shown strong performance across downstream tasks, including mathematical reasoning, code generation, and scientific question answering. In the post-training stage of general-purpose large language models, existing RLHF and RLVR pipelines commonly rely on full fine-tuning (FFT), where all model parameters are updated, to adapt pre-trained or mid-trained foundation models for alignment and downstream tasks ( 11 ) . However, FFT requires substantial computational resources and is often bottlenecked by GPU memory and training cost. Although parameter-efficient fine-tuning (PEFT) methods such as LoRA substantially reduce the cost of adapting large foundation models ( 3 ; 4 ) , their low-rank update space can limit model expressiveness, leading to a performance gap compared to full fine-tuning. Several studies have sought to improve the expressiveness and parameter allocation of LoRA-based adaptation. For example, AdaLoRA ( 22 ) decomposes the LoRA update into singular-value components and dynamically allocates the parameter budget by pruning less important components during training. AdaLoRA treats every singular value in the middle matrix with its two corresponding vectors as a triplet, computes a importance score for each of the triplets by multiplying its sensitivity and uncertain scores, and then prunes the ranks with lower importance scores based on the computation budget. Another approach, AutoLoRA ( 23 ) propose a bi-level optimization method to find the important rank among evenly distributed ranks in LoRA. AutoLoRA provides a contribution weight for each rank, and then update these weights based on the loss evaluated on validation set after every training step. The high contributing ranks (with higher weights) will be preserved and the model will be retrained when the ranks selection weights converge. Both approaches improve LoRA by allocating its limited rank budget more effectively across modules. However, they still operate entirely within the low-rank adaptation space, leaving a persistent gap from full fine-tuning. With these insights in mind, we aim to bridge this gap by breaking the binary opposition between full fine-tuning and LoRA: rather than choosing one exclusively, we develop a hybrid fine-tuning framework that combines full-parameter updates with low-rank adaption. Our key hypothesis is that Transformer decoder modules are not equally amenable to low-rank adaptation. Some modules require full-parameter updates to preserve post-training performance, whereas others can be effectively adapted with LoRA. This idea introduces two major technical problems: (1) How can we efficiently estimate each module’s suitability for low-rank adaptation and allocate full fine-tuning and PEFT under a fixed parameter budget? (2) How to achieve better performance of RL post-training in the hybrid fine-tuning framework with limited trainable parameters? To address these challenges, we develop a unified framework that integrates suitability estimation with stable RL optimization. In this paper, our contributions are threefold: • We propose Hybrid-LoRA , a hybrid parameter optimization framework for LLM post-training that selectively assigns Transformer modules to FFT while adapting the remaining modules with LoRA. • We introduce Hybrid-LoRA Score , an efficient module-level evaluation metric based on first-order gradient-weighted sensitivity, to estimate each module’s suitability for low-rank adaptation. • We evaluate our framework across six complex reasoning tasks and various scales, demonstrating its effectiveness on a wide range of downstream benchmarks, including mathematical reasoning, scientific question answering, and code generation tasks. It outperforms four state-of-the-art PEFT baselines, achieving improvements of up to 5.65% and on average 4.36% relative improvement over the strongest baseline. 2 Related Works Large Language Model Post-Training. The evolution of LLM post-training has progressed from supervised fine-tuning (SFT) to reinforcement learning from human feedback (RLHF), and more recently to reinforcement learning with verifiable rewards (RLVR). Proximal Policy Optimization (PPO) ( 13 ) has been widely used in early RLHF pipelines, where an additional value model is trained to estimate token-level value functions and compute advantages. However, maintaining this critic introduces substantial memory and computational overhead during large-scale LLM post-training. To reduce this overhead, recent RLVR methods have adopted critic-free optimization. 14 propose Group Relative Policy Optimization (GRPO), which samples multiple responses for each prompt and normalizes rewards within the group to estimate relative advantages without a separate critic. 21 introduce DAPO, which improves large-scale long-CoT RL training with decoupled clipping, dynamic sampling, and other system-level and algorithmic refinements. 24 propose Group Sequence Policy Optimization (GSPO), which replaces token-level importance ratios with sequence-level importance ratios and performs sequence-level clipping to improve training stability. Parameter-Efficient Fine-Tuning and LoRA Variants. To reduce the substantial computational and memory burden of full-parameter adaptation, parameter-efficient fine-tuning (PEFT) methods, particularly LoRA ( 4 ) and its variants, have emerged as a practical alternative for adapting large language models. Several LoRA variants improve efficiency by automatically reallocating the limited rank budget across modules. AdaLoRA ( 22 ) introduces a dynamic rank allocation framework that parameterizes LoRA updates with singular-value components and progressively prunes less sensitive components based on a sensitivity-based importance score. SoRA and AutoLoRA ( 1 ; 23 ) are closely related in their parameterization, as both decompose LoRA updates into sums of rank-one components formed by paired down- and up-projection vectors. SoRA introduces learnable gates over these rank-one components and optimizes them jointly with the LoRA parameters using proximal gradient updates under L_{0} regularization, driving some gates exactly to zero so that the corresponding rank components can be pruned. AutoLoRA adopts a bi-level meta-learning framework, where continuous selection variables over rank-one components are optimized according to validation performance, and discrete layer-wise ranks are obtained by thresholding these variables. Different from the approaches above, where

Read the complete text