Qrita : High-performance Top- k k and Top- p p Algorithm for GPUs using Pivot-based Truncation and Selection

Jongseok Park, Sunga Kim, Alvin Cheung, Ion Stoica · 2026

Read Qrita : High-performance Top- k k and Top- p p Algorithm for GPUs using Pivot-based Truncation and Selection by Jongseok Park, Sunga Kim, Alvin Cheung, Ion Stoica (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01518.txt.

Description

Jongseok Park and co-authors propose Qrita, an efficient GPU sampler for Top-k and Top-p via pivot-based truncation and selection. Sorting burdens GPUs, while stochastic methods alter outputs. Qrita searches by pivots for Top-k and Top-p, using Gaussian sigma-truncation to shrink vocabulary search. Quaternary pivot search handling duplicates halves iterations and guarantees determinism. Built in Triton, tested against SGLang and FlashInfer, it lifts serving throughput up to 1.4 times, halving memory, matching sorting. Qrita is now vLLM's default Top-k and Top-p sampler.

Catalog record

CREATORJongseok Park, Sunga Kim, Alvin Cheung, Ion Stoica
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64008
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.01518

Browse the full library · About the Tower of Babel