FedSmoothLoRA: Toward Smoother and Faster Convergence in Federated Low-Rank Adaptation
Zehao Wang, Guanglei Yang, Yihan Zeng, Hang Xu, Hongzhi Zhang, Wangmeng Zuo, Chun-Mei Feng · 2026
Read FedSmoothLoRA: Toward Smoother and Faster Convergence in Federated Low-Rank Adaptation by Zehao Wang, Guanglei Yang, Yihan Zeng, Hang Xu, Hongzhi Zhang, Wangmeng Zuo, Chun-Mei Feng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29460.txt.
Description
Zehao Wang and six colleagues tackle federated fine-tuning with low-rank adaptation, which adapts large models while keeping each client's data local. Pairing federated averaging with low-rank updates narrows the learning space. It breaks continuity between rounds and starts clients from the generic point. Their FedSmoothLoRA design keeps the wider update space from merging updates into the backbone. It preserves round-to-round state with a matching matrix and tailors starting points with local gradient guidance. Tests on image classification and language generation show steadier, faster convergence.
Catalog record
| CREATOR | Zehao Wang, Guanglei Yang, Yihan Zeng, Hang Xu, Hongzhi Zhang, Wangmeng Zuo, Chun-Mei Feng |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90125 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.29460