DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Models
Jin Liu, Yinbin Miao, Ning Xi, Junkang Liu · 2026
Read DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Models by Jin Liu, Yinbin Miao, Ning Xi, Junkang Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19945.txt.
Description
AdamW speeds training of large models, but under differentially private federated learning it runs into trouble: noise and data differences inflate variance, bias the second-moment estimate, and worsen drift between clients. A 2026 arXiv paper by Jin Liu and three co-authors adapts AdamW to that setting with DP-FedAdamW.
The optimizer steadies second-moment variance and strips out privacy-induced bias. It also lines local updates up with the global descent direction to curb drift. Theory gives an unbiased second-moment estimator and a linearly accelerated convergence rate. It needs no assumption about data homogeneity, and it offers tighter privacy guarantees. Tests across language and vision transformers and a residual network show strong results, including a clear margin over the prior best on a standard image benchmark under a strict privacy budget.
Catalog record
| CREATOR | Jin Liu, Yinbin Miao, Ning Xi, Junkang Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-19945