ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism
Alan AboudibRodrigo Lopez Portillo A.Kalei BradySteffen Cruz[1.2ex] Macrocosmos AI {firstname}@macrocosmos.ai · 2026
Read ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism by Alan AboudibRodrigo Lopez Portillo A.Kalei BradySteffen Cruz[1.2ex] Macrocosmos AI {firstname}@macrocosmos.ai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11947.txt.
Description
Aboudib, Lopez, Brady, and Cruz propose ResBM, a Residual Bottleneck Model built for low-bandwidth pipeline training. Large multi-node training leans on data and pipeline parallelism, but splitting work this way still demands ultra-high-bandwidth links. ResBM places a residual encoder-decoder bottleneck across pipeline splits, trainable end to end with a low-rank identity path. It reaches 128-fold activation compression, a state-of-the-art figure, without notable loss in convergence speed or added memory and compute overhead.
Catalog record
| CREATOR | Alan AboudibRodrigo Lopez Portillo A.Kalei BradySteffen Cruz[1.2ex] Macrocosmos AI {firstname}@macrocosmos.ai |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42278 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.11947