From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity
Renaud Gaucher, Aymeric Dieuleveut, Hadrien Hendrikx · 2026
Read From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity by Renaud Gaucher, Aymeric Dieuleveut, Hadrien Hendrikx (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03329.txt.
Description
Byzantine-robust distributed learning is recast as tuning with inexact gradients by Renaud Gaucher, with 2 co-authors, uniting a fragmented theory. Federated learning methods can be attacked through adversarial nodes, called Byzantine failures. Robust aggregations replace simple averaging. But analyses for each method have stayed ad hoc. That hinders development of more complex robust methods, such as accelerated schemes that could cut message costs for large distributed training jobs.
Under standard assumptions, the authors show robust shared tuning fits the inexact gradient oracle system, with additive and multiplicative error terms. Gradient descent on standard robust aggregations then directly obtains optimal asymptotic error. Two faster schemes follow, one a Nesterov-type acceleration, the other using an auxiliary loss approximating the global loss. Both cut communication complexity sharply, shown in theory and in tests.
Catalog record
| CREATOR | Renaud Gaucher, Aymeric Dieuleveut, Hadrien Hendrikx |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-03329