Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance
Hongxu Chen, Ke Wei, Luo Luo · 2026
Read Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance by Hongxu Chen, Ke Wei, Luo Luo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11789.txt.
Description
Decentralized optimization spreads machine learning across a network of nodes that work together through local links. Stochastic gradient noise often differs from node to node, but its effect on design and cost has stayed unclear. Hongxu Chen and two co-authors propose D-NSS, a method with node-specific sampling. Its sample cost depends on the simple average of local standard deviations, not worst-case or root-mean-square measures.
That link gives tighter bounds than existing methods. The authors also derive a matching lower bound under uneven variance, proving their link is optimal. A variance reduction extension, D-NSS-VR, improves sample cost further under mean-squared smoothness while keeping the simple-average link. Tests confirm the theory and show both methods work well for network learning with uneven noise.
Catalog record
| CREATOR | Hongxu Chen, Ke Wei, Luo Luo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-11789