AutoOverlap : Enabling Fine-Grained Overlap of Computation and Communication with Chunk-Based Scheduling

Xinwei Qiang, Yue Guan, Zhengding Hu, Yufei Ding, Adnan Aziz · 2026

Read AutoOverlap : Enabling Fine-Grained Overlap of Computation and Communication with Chunk-Based Scheduling by Xinwei Qiang, Yue Guan, Zhengding Hu, Yufei Ding, Adnan Aziz (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20595.txt.

Description

Xinwei Qiang and four colleagues present AutoOverlap, a way to overlap computation and communication more finely in large GPU workloads. Communication has become a first-order bottleneck at scale. Existing distributed compilers overlap whole compute and communication kernels at the stream level, a coarse approach.

That coarseness costs extra kernel launches, forces device-wide synchronization at kernel boundaries, and leaves slack when the slowest piece stretches the communication tail. AutoOverlap schedules in chunks for finer overlap. Shaving this overhead matters for training runs that span thousands of chips. The win shows up as less dead time on chips that cost a great deal per hour of use.

Catalog record

CREATORXinwei Qiang, Yue Guan, Zhengding Hu, Yufei Ding, Adnan Aziz
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-20595

Browse the full library · About the Tower of Babel