Speculative Pipeline Decoding: Higher-Accruacy and Zero-Bubble Speculation via Pipeline Parallelism
Yijiong Yu, Huazheng Wang, Shuai Yuan, Ruilong Ren, Ji Pei · 2026
Read Speculative Pipeline Decoding: Higher-Accruacy and Zero-Bubble Speculation via Pipeline Parallelism by Yijiong Yu, Huazheng Wang, Shuai Yuan, Ruilong Ren, Ji Pei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30852.txt.
Description
Yu and colleagues propose Speculative Pipeline Decoding, splitting the target model into pipeline stages so tokens of a single sequence advance as one. Mainstream methods rely on multi-token prediction, incurring compounding forecast trouble and exposed draft latency. To keep it saturated, a Pipeline Draft Module aggregates multi-depth features to predict the next token and runs with each step. This yields bounded forecast trouble, higher hits, and hidden draft latency. Experiments show it achieves higher speedup than EAGLE-3 at moderate width.
Catalog record
| CREATOR | Yijiong Yu, Huazheng Wang, Shuai Yuan, Ruilong Ren, Ji Pei |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48153 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.30852