RASD: Retrieval-Augmented Speculative Decoding
Guofeng Quan Wenfeng Feng Chuzhan Hao Guochao JiangYuewei Zhang Hao Wang · 2025
Read RASD: Retrieval-Augmented Speculative Decoding by Guofeng Quan Wenfeng Feng Chuzhan Hao Guochao JiangYuewei Zhang Hao Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03434.txt.
Description
Six authors, led by Guofeng Quan, wrote this 2025 paper on a faster way to run large language models. Speculative decoding speeds up a model by drafting words ahead and letting the big model check them. Drafts usually come from a small side model or from extra model parts.
Small draft models fail often outside their training ground, and drafting itself eats time. That caps how many draft words get accepted at each check. The new method, RASD, adds retrieval to the draft stage. It pulls draft material from stored context rather than relying on the small model alone. Two tree tools support it: pruning, which trims a draft tree by the draft model's own scores, and fusion, which joins trees. The design aims for longer accepted runs and thus faster output.
Catalog record
| CREATOR | Guofeng Quan Wenfeng Feng Chuzhan Hao Guochao JiangYuewei Zhang Hao Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-03434