Taming Preconditioner Drift: Unlocking the Potential of Second-Order Optimizers for Federated Learning on Non-IID Data

Junkang Liu, Fanhua Shang ∗, Hongying Liu ∗, Jin Liu, Weixin An, Yuanyuan Liu ∗ · 2026

Read Taming Preconditioner Drift: Unlocking the Potential of Second-Order Optimizers for Federated Learning on Non-IID Data by Junkang Liu, Fanhua Shang ∗, Hongying Liu ∗, Jin Liu, Weixin An, Yuanyuan Liu ∗ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19271.txt.

Description

Second-order optimizers speed up large-scale training by using curvature, the shape of the loss surface, not just its slope. In federated learning, where clients train locally and a server averages their work, naive versions often wobble or diverge when client data differ sharply. Junkang Liu and five colleagues trace much of that instability to a geometric cause they call preconditioner drift.

Each client builds its own curvature-based coordinate system, so server averaging mixes updates measured in incompatible geometries and corrupts the global direction. Their framework, FedPAC, aligns and corrects those preconditioners before averaging, restoring a coherent descent direction. The paper thus turns a puzzling federated failure into a fixable mismatch, opening second-order methods to settings where data are far from identical.

Catalog record

CREATORJunkang Liu, Fanhua Shang ∗, Hongying Liu ∗, Jin Liu, Weixin An, Yuanyuan Liu ∗
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE107 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-19271

Browse the full library · About the Tower of Babel