On the Robustness Tradeoff in Fine-Tuning
Kunyang Li, Jean-Charles Noirot Ferrand, Ryan Sheatsley, Blaine Hoak · 2025
Read On the Robustness Tradeoff in Fine-Tuning by Kunyang Li, Jean-Charles Noirot Ferrand, Ryan Sheatsley, Blaine Hoak (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-14836.txt.
Introduction
Kunyang Li and co-authors chart the trade-off between accuracy and robustness when fine-tuning models in a 2025 paper. Fine-tuning adapts a pretrained model to a new task, and it is now standard practice. What it does to a model's ability to resist attacks and odd inputs was murky.
Seven tuning strategies were tested across six benchmark data sets. A consistent trade-off appeared: gains in accuracy cost adversarial robustness. Light-touch methods that update only small bias terms won on simple image tasks. Methods that tune information-heavy attention layers won on harder ones. Robustness to shifted data tracked accuracy closely, unlike attack robustness. The authors argue tuning recipes should be picked with robustness in mind, not accuracy alone.
Catalog record
| CREATOR | Kunyang Li, Jean-Charles Noirot Ferrand, Ryan Sheatsley, Blaine Hoak |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-14836
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On the Robustness Tradeoff in Fine-Tuning Kunyang Li, Jean-Charles Noirot Ferrand, Ryan Sheatsley, Blaine Hoak Abstract Fine-tuning has become the standard practice for adapting pre-trained models to downstream tasks. However, the impact on model robustness is not well understood. In this work, we characterize the robustness-accuracy trade-off in fine-tuning. We evaluate the robustness and accuracy of fine-tuned models over 6 benchmark datasets and 7 different fine-tuning strategies. We observe a consistent trade-off between adversarial robustness and accuracy. Peripheral updates such as BitFit are more effective for simple tasks—over 75% above the average measured by the area under the Pareto frontiers on CIFAR-10 and CIFAR-100. In contrast, fine-tuning information-heavy layers, such as attention layers via Compacter, achieves a better Pareto frontier on more complex tasks—57.5% and 34.6% above the average on Caltech-256 and CUB-200, respectively. Lastly, we observe that the robustness of fine-tuning against out-of-distribution data closely tracks accuracy. These insights emphasize the need for robustness-aware fine-tuning to ensure reliable real-world deployments. 1 Introduction Pre-training and fine-tuning can efficiently transfer knowledge from upstream data to downstream tasks [ 2 , 44 ] . Models can be fine-tuned in various ways [ 27 ] —full fine-tuning, linear probing (training the classification head), and parameter-efficient fine-tuning (PEFT) [ 17 , 30 , 5 , 10 , 14 , 16 , 28 , 29 , 34 , 49 ] . Specifically, PEFT strategies selectively update parameters to achieve high accuracy on target tasks with significantly reduced computation and storage costs. While fine-tuning optimizes efficiency and accuracy, its impact on model robustness remains underexplored–and in fact relatively unknown. Attackers actively generate adversarial examples [ 36 , 40 ] by adding crafted perturbations to cause model misclassification. A model’s ability to resist these samples, as well as perform well when presented with out-of-distribution (OOD) data, is called model robustness. Prior studies on adversarial robustness [ 40 , 36 , 11 , 3 , 7 ] focus on attacking models that are trained from scratch. Models learn to use high-dimensional features during training. Since test data have similar features to training data, models are able to achieve high accuracy at evaluation. However, attackers can generate adversarial examples to largely degrade accuracy by perturbing those features, which we refer to as highly predictive, non-robust features [ 20 , 45 ] in this paper. In comparison, while attacks are targeted at downstream phenomena [ 4 , 22 , 6 , 18 , 39 ] , two data phenomena are involved in pre-training and fine-tuning—upstream and downstream data, respectively. Here, a key question arises: how does adversarial robustness vary as the model is fine-tuned? We hypothesize that features learned from pre-trained data are more robust against downstream attacks. As models fit to downstream phenomena, they learn non-robust features to gain accuracy while sacrificing adversarial robustness. Additionally, the fundamental trade-off between accuracy and adversarial robustness [ 45 , 31 , 51 ] may still exist but is potentially shifted. This work is the first to investigate deeply how robustness is impacted by fine-tuning. Here, we focus on three questions: (1) does the adversarial robustness-accuracy trade-off exist during fine-tuning? (2) how sensitive is this trade-off to different fine-tuning strategies and downstream data distributions? and (3) do findings on adversarial robustness generalize to OOD robustness? We begin by formally exploring the potential interaction between robustness and fine-tuning ( i.e . , the impact of mechanisms and data phenomena). Thereafter, we evaluate trade-offs empirically by fine-tuning a test suite of models using a range of fine-tuning strategies (see Figure 1 ). The experiments are performed over 231 models, 7 fine-tuning methods, and 6 benchmark datasets (5 for adversarial robustness and 1 with 6 domains for OOD robustness), resulting in approximately 2,100 adversarial and 2,000 OOD robustness assessments. Figure 1 : Continuous robustness evaluation during fine-tuning. The evaluation finds: (1) a consistent adversarial robustness-accuracy trade-off early (within the first 3 epochs) in fine-tuning across all methods—initially, robustness improves together with accuracy, then it peaks and declines as fine-tuning continues; (2) the Pareto frontiers ( i.e . , optimal trade-off curves) are sensitive to fine-tuning methods and downstream task complexity—fine-tuning methods modifying intermediate information-intense layers, such as attention layers ( e.g . , Compacter [ 37 ] ), achieve better balances than those updating excessively ( e.g . , full fine-tuning) or only peripheral layers ( e.g . , linear probing, BitFit [ 49 ] ); and (3) OOD robustness does not exhibit a trade-off with accuracy but remains relatively stable and closely tracks accuracy, suggesting that different underlying mechanisms drive robustness in security and safety contexts. These findings deepen our understanding of designing robustness-aware fine-tuning strategies facing different tasks and risks. 2 Background 2.1 Fine-tuning Strategies ViT backbone. The transformer architecture [ 46 ] has become the state-of-the-art across many fields [ 9 , 8 , 2 ] . It typically serves as the backbone structure in the pre-training and fine-tuning paradigm [ 44 ] , where general knowledge of pre-trained models is transferred to solve specific tasks through fine-tuning on (often small) downstream datasets. In this study, we focus on vision transformers (ViT), thus on images. Here, an input image is divided into fixed-size patches, flattened, and projected into embeddings. Then attention scores are calculated to determine the relationship between patches to capture global dependencies. The outputs are then passed through a feedforward network (FFN), in-between two layer normalizations (LN) for stability. Fine-tuning Methods. Full fine-tuning, which updates all model parameters, is widely used to transfer knowledge from pre-trained models to downstream tasks in computer vision [ 50 , 27 ] . While effective, it can be computationally expensive, especially for large-scale models. In contrast, linear probing, which fine-tunes only the final classification layer, is more efficient but often fails to match the performance of full fine-tuning [ 27 ] . To bridge this gap, parameter-efficient fine-tuning (PEFT) methods have been developed, aiming to achieve comparable or higher accuracy with fewer trainable parameters and reduced memory and storage overhead [ 15 , 32 ] . PEFT techniques primarily retain the pre-trained parameters while introducing a small set of trainable parameters to adapt the model to the downstream task. A general formulation can be expressed as: \hat{y}\leftarrow W_{0}x+\Delta Wx, (1) where W_{0} represents the frozen pre-trained weights, and \Delta W denotes the task-specific learnable parameters introduced by a given PEFT method. The input x is processed through the model, and the prediction \hat{y} is computed based on all parameters. Several PEFT techniques have been adapted for ViTs from the original transformer models in NLP [ 1 ] . LoRA [ 17 ] reduces computational cost by factorizing weight updates into low-rank matrices within attention layers, effectively capturing task-specific adaptations. BitFit [ 49 ] takes a more selective approach by updating only bias terms, leaving all other weights unchanged. Adapter-based methods [ 16 ] introduce small trainable modules between transformer layers to inject task-specific information without modifying the backbone. Compacter [ 37 ] further improves efficiency by using Kronecker-based parameterization within adapters, reducing the number of additional parameters needed. Finally, (IA) 3 [ 34 ] fine-tunes the model by learning per-layer multiplicative reweighting factors, modifying activations without directly changing pre-trained weights. A detailed visualization of how