FedKRSO: Communication and Memory Efficient Federated Fine-Tuning of Large Language Models
Guohao Yang ∗ \ast , Tongle Wu ⋆ \star , Yuanxiong Guo ∗ \ast , Ying Sun ⋆ \star , and Yanmin Gong ∗ \ast · 2026
Read FedKRSO: Communication and Memory Efficient Federated Fine-Tuning of Large Language Models by Guohao Yang ∗ \ast , Tongle Wu ⋆ \star , Yuanxiong Guo ∗ \ast , Ying Sun ⋆ \star , and Yanmin Gong ∗ \ast (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03019.txt.
Description
Fine-tuning large models across many private devices is made cheaper by Guohao Yang and 4 co-authors in a 2026 paper. Federated learning trains collaboratively without centralizing data, keeping privacy. Full fine-tuning, though, costs too much communication and memory for resource-constrained clients, while lighter setting-efficient methods often lose results compared with the full version.
Their method, FedKRSO, updates models inside shared random low-part subspaces made by the server. Clients send only compact accumulators along those subspaces, not full settings. The server aggregates and redistributes the global model efficiently. Convergence is analyzed rigorously for general federated settings. Tests on the GLUE benchmark across scenarios showed strong results with low overhead, approaching full federated fine-tuning quality.
Catalog record
| CREATOR | Guohao Yang ∗ \ast , Tongle Wu ⋆ \star , Yuanxiong Guo ∗ \ast , Ying Sun ⋆ \star , and Yanmin Gong ∗ \ast |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-03019