EAGLE-Pangu: Accelerator-Safe Tree Speculative Decoding on Ascend NPUs

Chang Han, Yijie Hu, Jingling Liu · 2026

Read EAGLE-Pangu: Accelerator-Safe Tree Speculative Decoding on Ascend NPUs by Chang Han, Yijie Hu, Jingling Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08088.txt.

Description

Han and colleagues port tree speculative decoding to Ascend NPUs, where autoregressive decoding still bottlenecks large language model serving. Speculative decoding cuts costly teacher-model calls by verifying several candidate tokens per step, and tree-structured variants add parallelism. But porting them across backends is brittle, since attention masking, KV-cache layouts, and indexing rules differ. Their EAGLE-Pangu system brings EAGLE-3-style tree speculation to a Pangu teacher backend, with an explicit branch and commit cache manager, accelerator-safe tree tensorization, and a fused-kernel teacher verification path.

Catalog record

CREATORChang Han, Yijie Hu, Jingling Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41019
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.08088

Browse the full library · About the Tower of Babel