Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks?

Zi Liang, Haibo Hu, Qingqing Ye, Yaxin Xiao, Ronghua Li · 2025

Read Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks? by Zi Liang, Haibo Hu, Qingqing Ye, Yaxin Xiao, Ronghua Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-12871.txt.

Introduction

Zi Liang and four co-authors asked whether a cheap way to tune large models leaves them open to attack during training, in a 2025 paper. The method, called LoRA, tweaks only a small low-rank part of the model. Its speed is prized, but its safety under poisoned data was little studied.

Their theory, built on a tool that tracks training and on information measures, gives a split answer. Against hidden backdoor tricks, the tuned model held up better than a fully tuned one. Against broad poisoning of the data, it fared worse, due to its simpler shape. Tests bore out both claims. The authors say users should weigh which threat matters most before choosing the method.

Catalog record

CREATORZi Liang, Haibo Hu, Qingqing Ye, Yaxin Xiao, Ronghua Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-12871

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks? Zi Liang, Haibo Hu, Qingqing Ye, Yaxin Xiao, Ronghua Li Abstract Low rank adaptation (LoRA) has emerged as a prominent technique for fine-tuning large language models (LLMs) thanks to its superb efficiency gains over previous methods. While extensive studies have examined the performance and structural properties of LoRA, its behavior upon training-time attacks remain underexplored, posing significant security risks. In this paper, we theoretically investigate the security implications of LoRA’s low-rank structure during fine-tuning, in the context of its robustness against data poisoning and backdoor attacks. We propose an analytical framework that models LoRA’s training dynamics, employs the neural tangent kernel to simplify the analysis of the training process, and applies information theory to establish connections between LoRA’s low rank structure and its vulnerability against training-time attacks. Our analysis indicates that LoRA exhibits better robustness to backdoor attacks than full fine-tuning, while becomes more vulnerable to untargeted data poisoning due to its over-simplified information geometry. Extensive experimental evaluations have corroborated our theoretical findings. 1 Introduction With the rapid growth in the parameter size of large language models (LLMs), parameter-efficient fine-tuning (PEFT) ( Xu et al., 2023 ; Han et al., 2024 ) has gained increasing attention in both research and industry communities. Among various PEFT strategies, low-rank adaptation (LoRA) ( Hu et al., 2021 ) has emerged as the de facto standard for fine-tuning LLMs thanks to its computational efficiency and minimal performance degradation. To compare LoRA with full fine-tuning across various dimensions, recently many studies have emerged. For example, researchers have investigated LoRA’s expressive capacity ( Zeng & Lee, 2024 ) , the smoothness ( Jang et al., 2024 ) of its convergence, the asymmetry ( Zhu et al., 2024 ) in its submatrices, the impact of initialization ( Hayou et al., 2024 ) , and so on ( Wang et al., 2024a ; Koubbi et al., 2024 ; Mao et al., 2024 ) . While these analyses have shed light on many properties of LoRA, one important aspect, i.e., its potential security risks, remains largely overlooked. Existing studies in this area either use LoRA as a tool to facilitate backdoor attacks ( Yin et al., 2024 ; Liu et al., 2024 ) , adversarial attacks ( Ji et al., 2024 ) , and model stealing attacks ( Horwitz et al., 2024 ; Liang et al., 2025 ) , or focus merely on the benefits ( Xu et al., 2024b ) of LoRA in differential privacy and federated learning. None of these works directly investigate the security vulnerabilities inherent in LoRA itself , which leaves behind potential hazards and vulnerabilities in LoRA-fine-tuned LLMs that are deployed across millions of devices ( Gunter et al., 2024 ) . To fill this gap, in this paper, we attempt to answer the question whether LoRA-based fine-tuning is more vulnerable than full fine-tuning (FF) under mainstream training-time attacks (e.g., data poisoning ( Fan et al., 2022 ; Ramirez et al., 2022 ; He et al., 2024 ) ). We introduce the concept of training-time robustness (TTR) for characterizing a model’s resistance to training-time attacks and propose an analytical framework to theoretically examine the security implications of LoRA’s low-rank structure. The main challenges are two-folded. First, the TTR of a model significantly depends on the specific training tasks and the complex dynamics of the training process. Second, the effectiveness of attacks is heavily influenced by hyperparameters (e.g., learning rate) and attack strategies (e.g., poisoning rate or backdoor triggers), both of which increase the complexity of analysis. To address these challenges, we introduce two novel simplifications when modeling the training dynamics of LoRA. First, we reformulate TTR by measuring the similarity of gradients before and after data poisoning, which enables a neural tangent kernel ( Jacot et al., 2021 ) (NTK)-based analysis to simplify the modeling of a training procedure. Second, we further introduce information theory ( Amari, 2016 ; Nielsen, 2020 ) to connect the model’s structural properties with its TTR, thereby decoupling the influences of different training datasets and attack methods. Our findings suggest that LoRA’s low-rank structure typically results in a smoother information geometry compared to FF, generally indicating better training-time robustness against backdoor attacks . However, we also observe that this simplicity might lead to obvious performance degradation under poisoning attacks or perturbations due to an oversimplified decision surface . We further quantify the key factors within LoRA that influence its TTR, demonstrating that initialization variance and rank are crucial determinants. Additionally, our analysis uncovers previously unexplained characteristics of LoRA, including the asymmetry and initialization of its submatrices, as well as the effects of various hyperparameters, such as the learning rate. We summarize our contributions as follows: \bullet We propose a novel theoretical framework to analyze the security of LoRA, revealing how its low-rank structure influences training-time robustness during fine-tuning. To our best knowledge, this is the first work to investigate LoRA’s intrinsic security vulnerabilities. \bullet We identify key factors within LoRA that influence its security and explain to what extent LoRA can be theoretically equivalent to full fine-tuning from a security perspective. Based on this analysis, we offer practical guidance for improving LoRA’s security. \bullet We provide a comprehensive evaluation of LoRA and FF under poisoning and backdoor attacks. Experimental results substantiate the correctness of these findings and explanations. Following a top-down structure, this paper is organized as follows. Section 2.1 introduces the basic notations and provides an overview of neural network training and the formulation of LoRA. Section 2.2 defines the concept of training-time robustness and highlights the analytical difficulties it presents. Sections 2.3 and 2.4 present high-level perspectives on how NTK and information geometry contribute to addressing these issues. Section 3 offers a comprehensive analysis and discussion, followed by empirical validation in Section 4 . Our source code is available at: https://github.com/liangzid/LoRA-sSecurity . 2 Preliminary 2.1 Notations Training Procedure. Without loss of generality, we begin our analysis with an L -layer artificial neural network (ANN) F_{\Theta}:\mathbb{R}^{n_{0}}\rightarrow\mathbb{R}^{n_{L}} which aims to map the input data x\in\mathbb{R}^{n_{0}} into corresponding output representations y\in\mathbb{R}^{n_{L}} . Given a training dataset \mathcal{D}=\{(x_{i},y_{i})\}_{i=1,2,...,N_{tr}} with N_{tr} finite training samples, we define the input matrix as X=[x_{1,},x_{2},...,x_{N_{tr}}]\in\mathbb{R}^{N_{tr}\times n_{0}} and the corresponding output matrix as Y=[y_{1},...,y_{N_{tr}}]\in\mathbb{R}^{N_{tr}\times n_{L}} . The objective of the neural network F_{\Theta} is to learn the mapping from X to Y by minimizing the following empirical risk function: \hat{\mathcal{L}}(\Theta;X,Y)=\sum_{i}^{N_{tr}}\mathcal{L}(F_{\Theta}(x_{i}),y_{i}), (1) where \Theta\in\mathbb{R}^{P} represents the set of P learnable parameters, and \mathcal{L} is the loss function. Linear Layers. Each layer F^{(l)}:\mathbb{R}^{n_{l}}\rightarrow\mathbb{R}^{n_{l+1}} in F_{\Theta} with l\in\{0,1,...,L-1\} is defined as a linear transformation: \displaystyle{\displaystyle y}^{(l)}(x_{i})=W^{(l)}\cdot x_{i}^{(l)}+b^{(l)}, (2) \displaystyle y_{a}^{(l)}=\mathbf{\sigma}({y}^{(l)}), where {y}^{(l)}\in\mathbb{R}^{n_{l+1}} is the preactivation output, which maps the l -th layer’s input x^{(l)}\in\mathbb{R}^{n_{l}} through the learnable matrix W^{(l)}\in\mathbb{R}^{N_{l+1}\times N_{l}} . The activation function \mathbf{\sigma}(\cdot) produces the output y_{a}^{(l)}\in\mathbb{R}^{n_{l+1}} at the l -th layer. LoRA Adapter. LoRA ( Hu et al., 2021 ) introduces a mechanism to reduce the number of trainable parameters by freezing the original matrix W^{(l)} and learn a low-rank update \Delta W^{(l)} . This update is factorized as the product of two low-rank submatrices, \Delta W^{(l)}=B^{(l)}A^{(l)}, (3) where A^{(l)}\in\mathbb{R}^{r\times n_{l}} and B^{(l)}\in\mathbb{R}^{n_{l+1}\times r} are learnable matrices, and r\ll\min\{n_{l},n_{l+1}\} . We define the intermediate state in

Read the complete text