EasySpec: Layer-Parallel Speculative Decoding for Efficient Multi-GPU Utilization
Yize Wu, Ke Gao, Ling Li, Yanjun Wu · 2025
Read EasySpec: Layer-Parallel Speculative Decoding for Efficient Multi-GPU Utilization by Yize Wu, Ke Gao, Ling Li, Yanjun Wu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-02493.txt.
Description
Yize Wu, Ke Gao, Ling Li, and Yanjun Wu sped up language model replies on multi GPU rigs. Their paper, from twenty twenty-five, deals with speculative decoding. A small draft model writes a guess line of tokens, and the big model checks it. This trick cuts wait with no loss in quality. On many GPUs, the draft model needs fewer cards than the big one, so cards sit idle while the draft runs.
Their fix, EasySpec, runs the draft model's layers in parallel, not in line. That breaks the chain between layers so more cards stay busy. Readers who serve large models get a way to use their full rig. Less idle time means faster replies at the same cost.
Catalog record
| CREATOR | Yize Wu, Ke Gao, Ling Li, Yanjun Wu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-02493