Taming Preconditioner Drift: Unlocking the Potential of Second-Order Optimizers for Federated Learning on Non-IID Data
Junkang Liu, Fanhua Shang ∗, Hongying Liu ∗, Jin Liu, Weixin An, Yuanyuan Liu ∗ · 2026
Read Taming Preconditioner Drift: Unlocking the Potential of Second-Order Optimizers for Federated Learning on Non-IID Data by Junkang Liu, Fanhua Shang ∗, Hongying Liu ∗, Jin Liu, Weixin An, Yuanyuan Liu ∗ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19271.txt.
Description
Second-order optimizers speed up large-scale training by using curvature, the shape of the loss surface, not just its slope. In federated learning, where clients train locally and a server averages their work, naive versions often wobble or diverge when client data differ sharply. Junkang Liu and five colleagues trace much of that instability to a geometric cause they call preconditioner drift.
Each client builds its own curvature-based coordinate system, so server averaging mixes updates measured in incompatible geometries and corrupts the global direction. Their framework, FedPAC, aligns and corrects those preconditioners before averaging, restoring a coherent descent direction. The paper thus turns a puzzling federated failure into a fixable mismatch, opening second-order methods to settings where data are far from identical.
Catalog record
| CREATOR | Junkang Liu, Fanhua Shang ∗, Hongying Liu ∗, Jin Liu, Weixin An, Yuanyuan Liu ∗ |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 107 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-19271