RASD: Retrieval-Augmented Speculative Decoding

Guofeng Quan Wenfeng Feng Chuzhan Hao Guochao JiangYuewei Zhang Hao Wang · 2025

Read RASD: Retrieval-Augmented Speculative Decoding by Guofeng Quan Wenfeng Feng Chuzhan Hao Guochao JiangYuewei Zhang Hao Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03434.txt.

Description

Six authors, led by Guofeng Quan, wrote this 2025 paper on a faster way to run large language models. Speculative decoding speeds up a model by drafting words ahead and letting the big model check them. Drafts usually come from a small side model or from extra model parts.

Small draft models fail often outside their training ground, and drafting itself eats time. That caps how many draft words get accepted at each check. The new method, RASD, adds retrieval to the draft stage. It pulls draft material from stored context rather than relying on the small model alone. Two tree tools support it: pruning, which trims a draft tree by the draft model's own scores, and fusion, which joins trees. The design aims for longer accepted runs and thus faster output.

Catalog record

CREATORGuofeng Quan Wenfeng Feng Chuzhan Hao Guochao JiangYuewei Zhang Hao Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-03434

Browse the full library · About the Tower of Babel