QuickSilver -- Speeding up LLM Inference through Dynamic Token Halting, KV Skipping, Contextual Token Fusion, and Adaptive Matryoshka Quantization

Danush Khanna 1, Aditya Kumar Guru 1, Srivarshinee Sridhar 2 · 2025

Read QuickSilver -- Speeding up LLM Inference through Dynamic Token Halting, KV Skipping, Contextual Token Fusion, and Adaptive Matryoshka Quantization by Danush Khanna 1, Aditya Kumar Guru 1, Srivarshinee Sridhar 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-22396.txt.

Description

Danush Khanna and co-authors present QuickSilver, a 2025 framework that speeds up large language model inference. Running these models eats most of their cost and power. QuickSilver works at token level without retraining or weight changes. Four mechanisms combine to cut wasted compute.

Tokens that converge stop early under dynamic halting. Skipped key-value work and token fusion trim more cost. Adaptive quantization adjusts precision on the fly. The parts are modular and can be mixed. Faster inference lowers bills for anyone serving chat models at scale, without touching the trained model.

Catalog record

CREATORDanush Khanna 1, Aditya Kumar Guru 1, Srivarshinee Sridhar 2
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE151 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-22396

Browse the full library · About the Tower of Babel