From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity

Renaud Gaucher, Aymeric Dieuleveut, Hadrien Hendrikx · 2026

Read From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity by Renaud Gaucher, Aymeric Dieuleveut, Hadrien Hendrikx (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03329.txt.

Description

Byzantine-robust distributed learning is recast as tuning with inexact gradients by Renaud Gaucher, with 2 co-authors, uniting a fragmented theory. Federated learning methods can be attacked through adversarial nodes, called Byzantine failures. Robust aggregations replace simple averaging. But analyses for each method have stayed ad hoc. That hinders development of more complex robust methods, such as accelerated schemes that could cut message costs for large distributed training jobs.

Under standard assumptions, the authors show robust shared tuning fits the inexact gradient oracle system, with additive and multiplicative error terms. Gradient descent on standard robust aggregations then directly obtains optimal asymptotic error. Two faster schemes follow, one a Nesterov-type acceleration, the other using an auxiliary loss approximating the global loss. Both cut communication complexity sharply, shown in theory and in tests.

Catalog record

CREATORRenaud Gaucher, Aymeric Dieuleveut, Hadrien Hendrikx
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-03329

Browse the full library · About the Tower of Babel