One Communication Round is All It Needs for Federated Fine-Tuning Foundation Models
Ziyao Wang, Bowei Tian, Yexiao He, Zheyu Shen, Luyang Liu, Ang Li1. University of Maryland, College Park 2. Google DeepMind {ziyaow,boweitian,yexiaohe,zyshen,angliece}@umd.edu, [email protected] · 2024
Read One Communication Round is All It Needs for Federated Fine-Tuning Foundation Models by Ziyao Wang, Bowei Tian, Yexiao He, Zheyu Shen, Luyang Liu, Ang Li1. University of Maryland, College Park 2. Google DeepMind {ziyaow,boweitian,yexiaohe,zyshen,angliece}@umd.edu, [email protected] (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04650.txt.
Introduction
Fine-tuning a huge foundation model across many sites without moving the data usually means many rounds of messages, and each round is costly. Ziyao Wang and co-authors argue one round is enough in this 2024 paper. In their setup, each site tunes the model on its own data. The sites then merge their tuned versions once into a shared model.
Theory and tests both support the claim, they report. The one-round merged model matched the quality of models built over many rounds. That result surprised the team because merging is normally repeated to iron out differences. Cutting to a single round slashes the data that must travel between sites. The saving matters most for the largest models, where each round ships billions of values. Privacy is kept, since raw data never leaves its site.
Catalog record
| CREATOR | Ziyao Wang, Bowei Tian, Yexiao He, Zheyu Shen, Luyang Liu, Ang Li1. University of Maryland, College Park 2. Google DeepMind {ziyaow,boweitian,yexiaohe,zyshen,angliece}@umd.edu, [email protected] |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-04650
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
One Communication Round is All It Needs for Federated Fine-Tuning Foundation Models Ziyao Wang, Bowei Tian, Yexiao He, Zheyu Shen, Luyang Liu, Ang Li1. University of Maryland, College Park 2. Google DeepMind {ziyaow,boweitian,yexiaohe,zyshen,angliece}@umd.edu, [email protected] Abstract The recent advancement of foundation models (FMs) has increased the demand for fine-tuning these models on large-scale cross-domain datasets. To address this, federated fine-tuning has emerged as a solution, allowing FMs to be fine-tuned on distributed datasets across multiple devices while ensuring data privacy. However, the substantial parameter size of FMs and the multi-round communication required by federated fine-tuning algorithms result in prohibitively high communication costs, challenging the practicality of federated fine-tuning. In this paper, we are the first to reveal, both theoretically and empirically, that the traditional multi-round aggregation algorithms may not be necessary for federated fine-tuning large FMs. Our experiments reveal that a single round of aggregation ( i.e., one-shot federated fine-tuning) yields a global model performance comparable to that achieved through multiple rounds of aggregation. Through rigorous mathematical and empirical analyses, we demonstrate that large FMs, due to their extensive parameter sizes and pre-training on general tasks, achieve significantly lower training loss in one-shot federated fine-tuning compared to smaller models. Our extensive experiments show that one-shot federated fine-tuning not only reduces communication costs but also enables asynchronous aggregation, enhances privacy, and maintains performance consistency with multi-round federated fine-tuning on both text generation and text-to-image generation tasks. Our findings have the potential to revolutionize federated fine-tuning in practice, enhancing efficiency, reducing costs, and expanding and accessibility for FMs. 1 Introduction Cutting-edge foundation models (FMs) demonstrate remarkable versatility across various domains. Notably, large language models (LLMs) like GPT-4 Achiam et al. 2023 , Gemma Team et al. 2024 , and Llama Touvron et al. 2023b excel in tasks such as translation, question answering (QA), chat assistant, and math. Similarly, stable diffusion models can generate diverse images based on textual descriptions. Achieving such versatility requires fine-tuning these FMs on cross-domain datasets. However, this process faces significant challenges in real-world scenarios due to the valuable datasets residing on devices owned by organizations or individuals, raising privacy concerns. To address these privacy issues, researchers have proposed using federated learning (FL) Zhang et al. 2021 for distributed fine-tuning of FMs, a process known as federated fine-tuning. Federated fine-tuning allows distributed clients to collaboratively fine-tune a global FM on specific tasks without disclosing their private data. Traditional FL requires multiple communication rounds between clients and the server to ensure the global model convergence McMahan et al. 2017 . However, the substantial parameter size of FMs (typically in billions) results in significant communication overhead. Many devices lack the capability to repeatedly communicate model parameters of this scale. While previous works adopt parameter-efficient fine-tuning (PEFT) methods such as low-rank adaptation (LoRA) Hu et al. 2021 to reduce the number of trainable and communicated parameters, the high communication requirements of federated fine-tuning remain a practical limitation. Figure 1: The distinct performances of one-shot federated learning between small models and large FMs. The horizontal axis represents multi-round FL accuracy, while the vertical axis represents one-shot FL accuracy. The ResNet-18 and LSTM are trained and tested on CIFAR-10 and Shakespeare respectively. Other models are fine-tuned on Wizard dataset and tested on ARC Easy. The closer points are to the dashed line means the closer accuracy between one-shot and multi-round FL. Unexpectedly, our recent experiments have discovered an emergent capability of FMs that could fundamentally shift the approach to federated fine-tuning. We find that with sufficient local fine-tuning epochs, a single communication round is all it needs to effectively fine-tune FMs , which is called one-shot federated fine-tuning . Figure 1 highlights the performance comparisons between one-shot FL and traditional multi-round FL, maintaining the same total number of local epochs. While one-shot FL underperforms multi-round FL for smaller models ( e.g., ResNet-18 and LSTM), it achieves comparable performance for larger FMs ( e.g., GPT-2, Llama, etc). This unique discovery challenges the conventional belief that multiple communication rounds are essential for the federated fine-tuning of FMs. Instead, we demonstrate that FMs can achieve convergence with just a single aggregation of well-fine-tuned local models. This paper explores this innovative finding, providing rigorous theoretical analysis and compelling empirical evidence to validate the effectiveness of one-shot FL for federated fine-tuning FMs. The introduction of one-shot FL brings transformative benefits. First , it dramatically reduces communication costs . One-Shot FL slashes communication overhead by a factor of \frac{1}{T} , where T represents the number of communication rounds in traditional federated fine-tuning. This reduction is a game-changer for devices with limited bandwidth. Second , one-shot FL enables seamless asynchronous training . This flexibility removes the bottleneck of server waiting times, ensuring uninterrupted training regardless of client connectivity or resource limitations. The process becomes far more robust and efficient. Third , one-shot FL offers enhanced security against prevalent client-side federated learning attacks. Attacks like client-side model inversion and gradient inversion, which depend on multiple global model updates, are rendered ineffective. This significantly bolsters the integrity of the training process. Our key contributions are listed as follows: • Novel Discovery: To the best of our knowledge, we are the first to discover that one-round aggregation is sufficient for federated fine-tuning large FMs. • Theoretical Analysis: We theoretically demonstrate the relationship between the error of one-shot federated fine-tuning and model smoothness, fine-tuning model update, number of fine-tuning rounds, and the second norm of original model parameters. Our analysis, supported by experiments, reveals that large FMs are smoother, exhibit smaller model updates, and require fewer fine-tuning epochs than smaller models, resulting in significantly lower one-shot federated fine-tuning error. • Experimental Validation: We conduct extensive experiments on six FMs and three tasks, demonstrating that one-shot federated fine-tuning achieves performance comparable to multi-round federated fine-tuning, particularly for models with over 1 billion parameters. Experimental results also surprisingly show that LoRA outperforms full fine-tuning in the context of one-shot federated fine-tuning. 2 Preliminary Federated Learning Paradigm of Small Models. In FL, the primary objective is to optimize a global objective function F(\bm{w}) , which is weighted average of the local objective functions from m clients Wang et al. 2020b : F(\bm{w})=\sum_{i=1}^{m}p_{i}F_{i}(\bm{w}) (1) where \bm{w} represents the model parameters and p_{i} is the scaling factor for averaging. To protect the data privacy of each client, the server cannot access the local dataset. Thus, the local objective function F_{i}(\bm{w}) remains unknown to the server. FedAvg McMahan et al. 2017 algorithm provides a distributed training algorithm to facilitate privacy-conscious training. It allows multiple clients to train the model on their local datasets and aggregates locally trained models on the server at the end of each communication round. In t -th communication round, the global model update rule of FedAvg is: \bm{w}^{(t+1,0)}-\bm{w}^{(t,0)}=\alpha^{(t)}\sum_{i=1}^{m}p_{i}\Delta_{i}^{(t)},t\in[0,T-1] (2) where \bm{w}^{(t,0)} is the model weights in t -th communication round and 0-th local epoch, which represents the global model in t -th round. T is the total number of communication rounds, \alpha^{(t)} is the global learning rate, and \Delta_{i}^{(t)} is the local model update in t -th round. \Delta_{i}^{(t)} is the accumulative model update of k local stochastic gradient descent (SGD) steps: \Delta_{i}^{(t)}=\sum_{j=1}^{k}\beta_{i}^{(t,j)}g_{i}(\bm{w}_{i}^{(t,j)}) (3) where g_{i}(\bm{w}_{i}^{(t,j)}) is the stochastic gradient over a local mini-batch and \beta_{i}^{(t,j)} is the