DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Models

Jin Liu, Yinbin Miao, Ning Xi, Junkang Liu · 2026

Read DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Models by Jin Liu, Yinbin Miao, Ning Xi, Junkang Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19945.txt.

Description

AdamW speeds training of large models, but under differentially private federated learning it runs into trouble: noise and data differences inflate variance, bias the second-moment estimate, and worsen drift between clients. A 2026 arXiv paper by Jin Liu and three co-authors adapts AdamW to that setting with DP-FedAdamW.

The optimizer steadies second-moment variance and strips out privacy-induced bias. It also lines local updates up with the global descent direction to curb drift. Theory gives an unbiased second-moment estimator and a linearly accelerated convergence rate. It needs no assumption about data homogeneity, and it offers tighter privacy guarantees. Tests across language and vision transformers and a residual network show strong results, including a clear margin over the prior best on a standard image benchmark under a strict privacy budget.

Catalog record

CREATORJin Liu, Yinbin Miao, Ning Xi, Junkang Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-19945

Browse the full library · About the Tower of Babel