Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models
Junjiao Tian, Chengyue Huang, Zsolt Kira · 2024
Read Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models by Junjiao Tian, Chengyue Huang, Zsolt Kira (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01713.txt.
Description
A 2024 arXiv research paper by Junjiao Tian, Chengyue Huang, Zsolt Kira. Modern optimizers such as AdamW, equipped with momentum and adaptive learning rate, are designed to escape local minima and explore the vast parameter space. This exploration is beneficial for finding good loss basins when training from scratch. It is not necessarily ideal when resuming from a powerful foundation model because it can lead to large deviations from the pre-trained initialization and, consequently, worse robustness and generalization.
Catalog record
| CREATOR | Junjiao Tian, Chengyue Huang, Zsolt Kira |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-01713
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models Junjiao Tian, Chengyue Huang, Zsolt Kira Abstract Modern optimizers such as AdamW, equipped with momentum and adaptive learning rate, are designed to escape local minima and explore the vast parameter space. This exploration is beneficial for finding good loss basins when training from scratch. It is not necessarily ideal when resuming from a powerful foundation model because it can lead to large deviations from the pre-trained initialization and, consequently, worse robustness and generalization. At the same time, strong regularization on all parameters can lead to under-fitting. We hypothesize that selectively regularizing the parameter space is the key to fitting and retraining the pre-trained knowledge. This paper proposes a new weight decay technique, Selective Projection Decay (SPD), that selectively imposes a strong penalty on certain layers while allowing others to change freely. Intuitively, SPD expands and contracts the parameter search space for layers with consistent and inconsistent loss reduction, respectively. Experimentally, when equipped with SPD, Adam consistently provides better in-distribution generalization and out-of-distribution robustness performance on multiple popular vision and language benchmarks. Code available at https://github.com/GT-RIPL/Selective-Projection-Decay.git . 1 Introduction Modern optimizers, such as Adam [ 1 ] , LARS [ 2 ] , and LAMB [ 3 ] usually include momentum and adaptive learning rates. They help optimizers avoid local minima and accelerate learning [ 4 , 5 ] to explore wider parameter spaces. However, we hypothesize that this behavior is not always beneficial for fine-tuning from a well pre-trained foundation model, especially when fine-tuning a few layers is already sufficient for fitting the target data [ 6 , 7 , 8 , 9 ] . Several prior works have found that unnecessary exploration will lead to large deviation from the initialization and worse robustness [ 10 , 11 ] , and constraining the deviation can improve a model’s generalization on in-distribution (ID) data and robustness to out-of-distribution (OOD) data [ 12 , 13 , 14 ] 1 1 1 In this paper, ID generalization and OOD robustness refer to test accuracy on the fine-tuning distribution and robust accuracy on other shifted distributions, respectively. . For example, L2-SP [ 13 ] imposes a regularization term on the distance between the current and pre-trained models. More recently, TPGM [ 10 ] and FTP [ 11 ] propose to learn different hard constraints for each layer. These new works have demonstrated impressive results on benchmarks. However, they are either difficult to tune, specialized to specific settings, or require significant computation and storage overhead. This motivates us to ask whether a simple few-liner solution exists for this fundamental problem. We propose re-examining the existing methods and summarizing their findings to find this solution. Starting from the simplest: L2-SP [ 13 ] . Specifically, L2-SP adds an L2 regularization term to the original objective function. Formally, \displaystyle\mathcal{L(\theta)}=\tilde{\mathcal{L}}(\theta)+\frac{\lambda}{2}\|\theta-\theta_{0}\|^{2}_{2} (1) where \theta denotes the model parameters, \theta_{0} the initialization, \tilde{\mathcal{L}}(\theta) the original objective function, and \lambda the hyper-parameter for regularization strength. When \theta_{0}=\mathbf{0} , L2-SP reduces to an ordinary weight decay. This simple method should be effective enough to constrain the model, as our experiments show it can reduce the deviation between the fine-tuned and pre-trained models (Sec. 4.1 ). However, it is held back by an important design choice: the penalty is always applied to all model parameters. Our empirical results identify that a large \lambda prevents every layer from deviating too much and leads to poor fitting, while a small \lambda cannot provide enough regularization. This significantly limits the otherwise effective design (Sec. 4.1 ). So, what is missing in this algorithm? Recent works in robust fine-tuning and parameter-efficient fine-tuning (PEFT) have shown that customizing constraints for each layer and selectively choosing layers for fine-tunning can improve robustness [ 10 , 11 , 6 ] . Inspired by these findings, we hypothesize that selectively imposing the regularization to different layers is the key. Therefore, we propose a simple selective version of L2-SP weight decay: selective projection decay (SPD). This new algorithm innovates in two aspects: a selection condition and a regularization strength ratio . The former determines when to apply regularization to a layer, and the latter determines the strength of regularization for intuitive hyper-parameter tuning. Specifically, we derive the selection condition from hyper-optimization [ 15 , 16 , 17 ] by treating the condition as an optimizable parameter (Sec. 3.3 ), and the regularization strength ratio by re-writing L2-SP as a projection operation (Sec. 3.4 ). Intuitively, when the condition is met, the algorithm imposes large regularizations on selected layers. This allows the algorithm to avoid unnecessary deviation and simultaneously fit into the fine-tuning data. We test SPD on large-scale computer vision, and NLP benchmarks with popular foundation models and test ID and OOD performance on various distribution and domain shifts. SPD achieves SOTA performance while being much simpler than other competing methods. Our contributions are: • We propose a selective projection decay, a selective variant of the popular L2-SP/weight decay regularization methods, for robust fine-tuning of large foundation models. We show that selectivity is important to make regularization effective. • We conduct a detailed study of ID and OOD performance on image classification and semantic segmentation with natural distribution and domain shifts. SPD improves ID and OOD performance on these benchmarks. • We show that SPD consistently improves the performance of PEFT methods (e.g. LoRA [ 7 ] and adapters [ 9 ] ) on 8 common sense reasoning language tasks with LLaMA-7B (-13B). 2 Related Works Robust Fine-Tuning with Distance Regularization. Constraining the distance or deviation between the fine-tuned and pre-trained models has been studied in several prior works. L2-SP [ 13 ] explicitly adds an L2 norm penalty on the deviation and shows improved ID generalization for fine-tuning. MARS-SP [ 12 ] studies different forms of norms as the penalty. It shows that the Matrix Row Sum (MARS) norm can be a superior alternative to the L2 norm. These two methods impose “soft” penalties and can be less effective [ 18 ] . Instead, LCC [ 18 ] proposes constraining the deviation through direct projection on the parameters, which also enforces a hard constraint on the Lipschitz continuity of the fine-tuned model. However, LCC is hard to tune because the projection radius is not an intuitive hyper-parameter. Furthermore, using a single projection constraint for all layers is not an ideal strategy [ 10 ] . More recently, TPGM [ 10 ] proposes to automatically learn the constraints in LCC during fine-tuning, customizing a different projection radius for each layer through a bi-level optimization scheme. FTP [ 11 ] further improves the computation efficiency of TPGM by adopting hyper-optimization [ 15 , 16 , 17 ] in its computation. Nevertheless, FTP is still difficult to control because hyper-optimization requires a secondary optimizer with additional optimization hyper-parameters, and the learned regularization can be too strong with no intuitive way to adjust. In contrast, SPD is a much simpler and more intuitive method, which can be implemented with just a few lines of code. The superior controllability makes SPD potentially applicable to more applications. Parameter Efficient Fine-Tuning (PEFT). PEFT methods such as adapters [ 9 , 8 ] and LoRA [ 7 ] have been