EAGLE-Pangu: Accelerator-Safe Tree Speculative Decoding on Ascend NPUs
Chang Han, Yijie Hu, Jingling Liu · 2026
Read EAGLE-Pangu: Accelerator-Safe Tree Speculative Decoding on Ascend NPUs by Chang Han, Yijie Hu, Jingling Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08088.txt.
Description
Han and colleagues port tree speculative decoding to Ascend NPUs, where autoregressive decoding still bottlenecks large language model serving. Speculative decoding cuts costly teacher-model calls by verifying several candidate tokens per step, and tree-structured variants add parallelism. But porting them across backends is brittle, since attention masking, KV-cache layouts, and indexing rules differ. Their EAGLE-Pangu system brings EAGLE-3-style tree speculation to a Pangu teacher backend, with an explicit branch and commit cache manager, accelerator-safe tree tensorization, and a fused-kernel teacher verification path.
Catalog record
| CREATOR | Chang Han, Yijie Hu, Jingling Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41019 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.08088