Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance

Hongxu Chen, Ke Wei, Luo Luo · 2026

Read Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance by Hongxu Chen, Ke Wei, Luo Luo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11789.txt.

Description

Decentralized optimization spreads machine learning across a network of nodes that work together through local links. Stochastic gradient noise often differs from node to node, but its effect on design and cost has stayed unclear. Hongxu Chen and two co-authors propose D-NSS, a method with node-specific sampling. Its sample cost depends on the simple average of local standard deviations, not worst-case or root-mean-square measures.

That link gives tighter bounds than existing methods. The authors also derive a matching lower bound under uneven variance, proving their link is optimal. A variance reduction extension, D-NSS-VR, improves sample cost further under mean-squared smoothness while keeping the simple-average link. Tests confirm the theory and show both methods work well for network learning with uneven noise.

Catalog record

CREATORHongxu Chen, Ke Wei, Luo Luo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE102 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-11789

Browse the full library · About the Tower of Babel