AdaPtis : Reducing Pipeline Bubbles with Adaptive Pipeline Parallelism on Heterogeneous Models Conference: ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming; January 31–February 04, 2026; Sydney, Australia

Jihu Guo, Tenghui Ma, Wei Gao, Peng Sun, Jiaxing Li, Xun Chen, Yuyang Jin, Dahua Lin · 2025

Read AdaPtis : Reducing Pipeline Bubbles with Adaptive Pipeline Parallelism on Heterogeneous Models Conference: ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming; January 31–February 04, 2026; Sydney, Australia by Jihu Guo, Tenghui Ma, Wei Gao, Peng Sun, Jiaxing Li, Xun Chen, Yuyang Jin, Dahua Lin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23722.txt.

Description

Training large models by splitting them across machines wastes time whenever one stage waits on another. Jihu Guo and 8 co-authors built OctoPipe to shrink those pipeline bubbles for models whose parts differ in size and shape. Older systems tune only one choice at a time: how to split, where to place, or how to schedule. OctoPipe tunes all three together, guided by a graph-based simulator of the pipeline.

An iterative tuner searches the huge combined space with bubbles in mind. A unified executor then runs irregular orders without deadlock and overlaps communication with computation. Across model shapes and cluster sizes, throughput rose by roughly ten to fifty percent over leading systems.

Catalog record

CREATORJihu Guo, Tenghui Ma, Wei Gao, Peng Sun, Jiaxing Li, Xun Chen, Yuyang Jin, Dahua Lin
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-23722

Browse the full library · About the Tower of Babel