LoRA and Privacy: When Random Projections Help (and When They Don’t)
Yaxi Hu ∗, Johanna Düngler, Bernhard Schölkopf, Amartya Sanyal · 2026
Read LoRA and Privacy: When Random Projections Help (and When They Don’t) by Yaxi Hu ∗, Johanna Düngler, Bernhard Schölkopf, Amartya Sanyal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21719.txt.
Introduction
Yaxi Hu, Johanna Düngler, Bernhard Schölkopf, and Amartya Sanyal study privacy in LoRA. LoRA is a widely used way to fine-tune large models with small added parts. Their 2026 preprint asks when random projections help protect the data used in training, and when they do not. The tool they build for the job is a randomized method they call the Wishart projection mechanism.
Catalog record
| CREATOR | Yaxi Hu ∗, Johanna Düngler, Bernhard Schölkopf, Amartya Sanyal |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 126 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-21719
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LoRA and Privacy: When Random Projections Help (and When They Don’t) Yaxi Hu ∗, Johanna Düngler, Bernhard Schölkopf, Amartya Sanyal Abstract We introduce the (Wishart) projection mechanism , a randomized map of the form S ↦ M f ( S ) S\;\mapsto\;Mf(S) with M ∼ 𝖶 d ( 1 / r I d , r ) M\;\sim\;\mathsf{W}_{d}(1/rI_{d},\,r) and study its differential privacy properties. For vector-valued queries f f , we prove non-asymptotic DP guarantees without any additive noise, showing that Wishart randomness alone can suffice. For matrix-valued queries, however, we establish a sharp negative result: in the noise-free setting, the mechanism is not DP, and we demonstrate its vulnerability by implementing a near perfect membership inference attack (AUC > 0.99 >0.99 ). We then analyze a noisy variant and prove privacy amplification due to randomness and low rank projection, in both large- and small-rank regimes, yielding stronger privacy guarantees than additive noise alone. Finally, we show that LoRA-style updates are an instance of the matrix-valued mechanism, implying that LoRA is not inherently private despite its built-in randomness, but that low-rank fine-tuning can be more private than full fine-tuning at the same noise level. Preliminary experiments suggest that tighter accounting enables lower noise and improved accuracy in practice. 1 Introduction Differential Privacy (DP) ( Dwork et al. 2006 ) is widely regarded as the gold standard for protecting training data privacy in machine learning. Intuitively, DP limits the influence of any single example on the output, making it difficult to infer whether that example appeared in the training set. The most widely used DP algorithm in modern ML is DP-SGD ( Abadi et al. 2016 ) , the private counterpart of the de facto standard optimization algorithm Stochastic Gradient Descent (SGD). However, DP-SGD is computationally demanding and often incurs a substantial utility loss, especially for large models. While it remains one of the few viable choices for training machine learning models from scratch, in many practical deployments sensitive data enters primarily during fine-tuning, e.g. when an organization adapts a public pre-trained model on proprietary data. This motivates a simple strategy: start from a large public pre-trained model and enforce privacy only during fine-tuning. In the non-private setting, parameter-efficient fine-tuning (PEFT) ( Han et al. 2024 ) updates only a small set of parameters while freezing the base model, substantially reducing memory and compute. This naturally raises the question: can PEFT similarly reduce the cost of DP fine-tuning? Low-Rank Adaptation (LoRA) ( Hu et al. 2022 ) is a widely used PEFT method that often matches full-parameter fine-tuning on downstream tasks ( Hu et al. 2022 ; Dettmers et al. 2023 ) . It freezes the pre-trained weights and inserts randomly initialised trainable low-rank matrices, dramatically shrinking the number of trainable parameters. Variants include adaptive-rank methods ( Zhang et al. 2023b ) , quantization-aware tuning for low-bit backbones ( Dettmers et al. 2023 ; Li et al. 2024 ) , stability/initialization refinements ( Hayou et al. 2024 ; Meng et al. 2024 ) , and structural decompositions ( Liu et al. 2024 ) , each targeting better accuracy under tight compute and memory budgets. Approaches to privatising LoRA have also been proposed, including DP-LoRA ( Liu et al. 2025 ) . Several LoRA variants ( Sun et al. 2024 ; Hao et al. 2024 ) already incorporate substantial randomness (e.g., repeated re-initialization of component weight matrices). Yet most privatisation algorithms treat LoRA updates as deterministic given the data and do not explicitly leverage this inherent randomness. At the same time, empirical studies report reduced memorisation under LoRA ( Hong et al. 2025 ) even without any explicit privatisation. Following Hao et al. 2024 , we note that in certain LoRA variants (e.g. LoRA-FA ( Zhang et al. 2023a ) , the update behaves as if it applies a random Wishart projection to the gradient. We formalize this behaviour through the following abstraction, (which allows us to rigorously analyse the privacy guarantee of these methods) Definition 1 (Projection mechanism) . Let {\mathcal{S}}\subset{\mathcal{X}}^{n} be a dataset collection and f:\mathcal{S}\to\mathbb{R}^{n\times d} a query function. For r\in\mathbb{N} , the (Wishart) projection mechanism is defined by \mathcal{A}_{r}(S)\coloneqq f(S)M,\qquad M:=Z^{\top}Z, where Z\in\mathbb{R}^{r\times d} has i.i.d rows z_{k}\overset{i.i.d}{\sim}\mathcal{N}\left({0,\frac{1}{r}I_{d}}\right) or equivalently, M\sim\mathsf{W}_{d}(\frac{1}{r}I_{d},\,r) 1 1 1 All of our analysis extends to Gaussian noise Z\sim{\mathcal{N}}(0,\sigma^{2}I_{d}) for any \sigma^{2}>0 . We set \sigma^{2}=1/r to match the usual normalization of Wishart random projection (e.g. in LoRA initializations). . We therefore analyse \mathcal{A}_{r} as a proxy for LoRA’s built-in randomness. Unlike the Johnson–Lindenstrauss ( Johnson and Lindenstrauss 1984 ) transform, which is a random embedding that preserves norms, the Wishart projection is a random positive semidefinite reweighting that is isotropic in expectation but can distort norms in a single draw. Recent work ( Malekmohammadi and Farnadi 2024 ) conjectures that LoRA training dynamics may inherently possess privacy properties. Specifically, they show that individual rows of the gradient matrix asymptotically resembles the noise distribution of DP-SGD as the dimension d\to\infty . In contrast, we establish non-asymptotic differential privacy guarantees for the vector-valued projection mechanism in 1 . However, LoRA updates are matrix-valued: in our abstraction, they correspond to applying the projection mechanism to gradient matrices. We show that this matrix-valued projection mechanism, and hence LoRA, is not private in general ( Proposition 2 ): in the setting without additional additive noise, neighbouring datasets can yield non-overlapping support, enabling (near) perfect membership inference with AUC \geq 0.99 as illustrated in Table 1 . Nevertheless, LoRA’s built-in randomness is not irrelevant: it is simply insufficient on its own. We therefore consider a noisy variant of the projection mechanism, obtained by adding a small amount of additive noise. This enforces overlap for output distributions of neighbouring datasets, and the above attack no longer succeeds. In this setting we extend our guarantees from vectors to matrices and show that Wishart projection can amplify privacy. For fixed noise, composing with the projection yields strictly stronger privacy bounds than the noise level alone would suggest, both for large r ( Section 4.1 ) and in the practically relevant small- r regime ( Section 4.2 ). A key implication is that low rank is beneficial beyond utility: at a fixed noise multiplier, projection can amplify privacy and yield smaller \varepsilon than full fine-tuning, a point not made explicit in prior DP-LoRA analyses. We summarize our contributions below. (i) We introduce the Wishart projection mechanism and prove non-asymptotic DP guarantees in the vector case. (ii) We show the corresponding matrix mechanism is not DP, and demonstrate its vulnerability via a near-perfect MIA (AUC >0.99 ). (iii) For a variant involving additive noise, we prove privacy amplification for both large and small ranks. Organization. In Section 2 we review DP preliminaries and give a brief overview of why LoRA is an instance of the Wishart projection mechanism. Section 3 proves privacy for vector-valued functions and shows why it fails for matrices. Section 4 analyses the noisy Wishart projection mechanism for matrix-valued functions and establishes privacy amplification in both rank regimes. Finally, Section 5 covers related work, limitations, and open questions. 2 Preliminaries We first review DP preliminaries, then introduce LoRA and show how it instantiates the projection mechanism. Differential privacy