When Scaling Fails: Network and Fabric Effects on Distributed GPU Training Performance
Dinesh Gopalan, Ratul Ali · 2026
Read When Scaling Fails: Network and Fabric Effects on Distributed GPU Training Performance by Dinesh Gopalan, Ratul Ali (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-04424.txt.
Description
Gopalan and Ali trace GPU training scaling failures to network and fabric effects. Extra nodes should help as planned, yet real jobs stall early with unstable swings. Across live clusters, topology, congestion, sync, and locality often dominate performance past a few nodes. Identical software can diverge with fabric design and traffic patterns. Failure modes include sync amplification, contention from topology, and locality variance, often misread as framework flaws. Diagnostic rules from the study should improve planning and cut large scale costs.
Catalog record
| CREATOR | Dinesh Gopalan, Ratul Ali |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37452 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.04424