ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism

Alan AboudibRodrigo Lopez Portillo A.Kalei BradySteffen Cruz[1.2ex] Macrocosmos AI {firstname}@macrocosmos.ai · 2026

Read ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism by Alan AboudibRodrigo Lopez Portillo A.Kalei BradySteffen Cruz[1.2ex] Macrocosmos AI {firstname}@macrocosmos.ai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11947.txt.

Description

Aboudib, Lopez, Brady, and Cruz propose ResBM, a Residual Bottleneck Model built for low-bandwidth pipeline training. Large multi-node training leans on data and pipeline parallelism, but splitting work this way still demands ultra-high-bandwidth links. ResBM places a residual encoder-decoder bottleneck across pipeline splits, trainable end to end with a low-rank identity path. It reaches 128-fold activation compression, a state-of-the-art figure, without notable loss in convergence speed or added memory and compute overhead.

Catalog record

CREATORAlan AboudibRodrigo Lopez Portillo A.Kalei BradySteffen Cruz[1.2ex] Macrocosmos AI {firstname}@macrocosmos.ai
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42278
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11947

Browse the full library · About the Tower of Babel