QuickSilver -- Speeding up LLM Inference through Dynamic Token Halting, KV Skipping, Contextual Token Fusion, and Adaptive Matryoshka Quantization
Danush Khanna 1, Aditya Kumar Guru 1, Srivarshinee Sridhar 2 · 2025
Read QuickSilver -- Speeding up LLM Inference through Dynamic Token Halting, KV Skipping, Contextual Token Fusion, and Adaptive Matryoshka Quantization by Danush Khanna 1, Aditya Kumar Guru 1, Srivarshinee Sridhar 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-22396.txt.
Description
Danush Khanna and co-authors present QuickSilver, a 2025 framework that speeds up large language model inference. Running these models eats most of their cost and power. QuickSilver works at token level without retraining or weight changes. Four mechanisms combine to cut wasted compute.
Tokens that converge stop early under dynamic halting. Skipped key-value work and token fusion trim more cost. Adaptive quantization adjusts precision on the fly. The parts are modular and can be mixed. Faster inference lowers bills for anyone serving chat models at scale, without touching the trained model.
Catalog record
| CREATOR | Danush Khanna 1, Aditya Kumar Guru 1, Srivarshinee Sridhar 2 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 151 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-22396