Entropy-Aware Speculative Decoding Toward Improved LLM Reasoning

Tiancheng Su, Meicong Zhang, Guoxiu He · 2025

Read Entropy-Aware Speculative Decoding Toward Improved LLM Reasoning by Tiancheng Su, Meicong Zhang, Guoxiu He (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-23765.txt.

Description

Tiancheng Su, Meicong Zhang, and Guoxiu He tweak speculative decoding, the trick that lets a small draft model propose tokens for a large model to verify, speeding up text generation. Normally the technique is used purely for acceleration, and output quality stays capped by the large target model. Their variant, Entropy-Aware Speculative Decoding, needs no training and aims at quality as well.

The method watches for a suspicious pattern: both models are highly uncertain, yet their top predictions overlap strongly. In those uncertain-agreement cases it rejects the agreed token and resamples from the target model's distribution, stopping a shaky guess from steering everything after it. On hard reasoning benchmarks the approach consistently beat standard speculative decoding and reward-guided variants at comparable speed, and it sometimes outperformed the target model running alone.

Catalog record

CREATORTiancheng Su, Meicong Zhang, Guoxiu He
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-23765

Browse the full library · About the Tower of Babel