Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training
Guanbin Xu † , ZhenGuo Xu † , Yuzhe Li † , Youhui Bai † , Ping Gong † , Chaoyi Ruan ‡ Cheng Li †,∘ · 2026
Read Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training by Guanbin Xu † , ZhenGuo Xu † , Yuzhe Li † , Youhui Bai † , Ping Gong † , Chaoyi Ruan ‡ Cheng Li †,∘ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-20656.txt.
Description
Overlapping communication with computation is crucial for distributed large-model training, yet tuning that overlap stays hard, above all when computation becomes the bottleneck. Guanbin Xu and six co-authors present Lagom, a system that co-tunes communication settings to balance resource use between computation and communication. A unified cost model plus a priority-based search algorithm cuts optimization complexity from exponential to linear.
Evaluations on high- and low-bandwidth GPU clusters show Lagom runs about seven to thirty-three percent faster than NCCL. It runs about three to twenty-seven percent faster than AutoCCL across varied models and parallelization schemes. The paper presents co-tuned overlap as a practical lever for distributed training speed, with gains that hold across cluster types and model shapes.
Catalog record
| CREATOR | Guanbin Xu † , ZhenGuo Xu † , Yuzhe Li † , Youhui Bai † , Ping Gong † , Chaoyi Ruan ‡ Cheng Li †,∘ |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-20656