Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training

Guanbin Xu † , ZhenGuo Xu † , Yuzhe Li † , Youhui Bai † , Ping Gong † , Chaoyi Ruan ‡ Cheng Li †,∘ · 2026

Read Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training by Guanbin Xu † , ZhenGuo Xu † , Yuzhe Li † , Youhui Bai † , Ping Gong † , Chaoyi Ruan ‡ Cheng Li †,∘ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-20656.txt.

Description

Overlapping communication with computation is crucial for distributed large-model training, yet tuning that overlap stays hard, above all when computation becomes the bottleneck. Guanbin Xu and six co-authors present Lagom, a system that co-tunes communication settings to balance resource use between computation and communication. A unified cost model plus a priority-based search algorithm cuts optimization complexity from exponential to linear.

Evaluations on high- and low-bandwidth GPU clusters show Lagom runs about seven to thirty-three percent faster than NCCL. It runs about three to twenty-seven percent faster than AutoCCL across varied models and parallelization schemes. The paper presents co-tuned overlap as a practical lever for distributed training speed, with gains that hold across cluster types and model shapes.

Catalog record

CREATORGuanbin Xu † , ZhenGuo Xu † , Yuzhe Li † , Youhui Bai † , Ping Gong † , Chaoyi Ruan ‡ Cheng Li †,∘
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-20656

Browse the full library · About the Tower of Babel