Speculative Pipeline Decoding: Higher-Accruacy and Zero-Bubble Speculation via Pipeline Parallelism

Yijiong Yu, Huazheng Wang, Shuai Yuan, Ruilong Ren, Ji Pei · 2026

Read Speculative Pipeline Decoding: Higher-Accruacy and Zero-Bubble Speculation via Pipeline Parallelism by Yijiong Yu, Huazheng Wang, Shuai Yuan, Ruilong Ren, Ji Pei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30852.txt.

Description

Yu and colleagues propose Speculative Pipeline Decoding, splitting the target model into pipeline stages so tokens of a single sequence advance as one. Mainstream methods rely on multi-token prediction, incurring compounding forecast trouble and exposed draft latency. To keep it saturated, a Pipeline Draft Module aggregates multi-depth features to predict the next token and runs with each step. This yields bounded forecast trouble, higher hits, and hidden draft latency. Experiments show it achieves higher speedup than EAGLE-3 at moderate width.

Catalog record

CREATORYijiong Yu, Huazheng Wang, Shuai Yuan, Ruilong Ren, Ji Pei
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48153
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30852

Browse the full library · About the Tower of Babel