FedSmoothLoRA: Toward Smoother and Faster Convergence in Federated Low-Rank Adaptation

Zehao Wang, Guanglei Yang, Yihan Zeng, Hang Xu, Hongzhi Zhang, Wangmeng Zuo, Chun-Mei Feng · 2026

Read FedSmoothLoRA: Toward Smoother and Faster Convergence in Federated Low-Rank Adaptation by Zehao Wang, Guanglei Yang, Yihan Zeng, Hang Xu, Hongzhi Zhang, Wangmeng Zuo, Chun-Mei Feng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29460.txt.

Description

Zehao Wang and six colleagues tackle federated fine-tuning with low-rank adaptation, which adapts large models while keeping each client's data local. Pairing federated averaging with low-rank updates narrows the learning space. It breaks continuity between rounds and starts clients from the generic point. Their FedSmoothLoRA design keeps the wider update space from merging updates into the backbone. It preserves round-to-round state with a matching matrix and tailors starting points with local gradient guidance. Tests on image classification and language generation show steadier, faster convergence.

Catalog record

CREATORZehao Wang, Guanglei Yang, Yihan Zeng, Hang Xu, Hongzhi Zhang, Wangmeng Zuo, Chun-Mei Feng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE90125
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.29460

Browse the full library · About the Tower of Babel