Qrita : High-performance Top- k k and Top- p p Algorithm for GPUs using Pivot-based Truncation and Selection
Jongseok Park, Sunga Kim, Alvin Cheung, Ion Stoica · 2026
Read Qrita : High-performance Top- k k and Top- p p Algorithm for GPUs using Pivot-based Truncation and Selection by Jongseok Park, Sunga Kim, Alvin Cheung, Ion Stoica (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01518.txt.
Description
Jongseok Park and co-authors propose Qrita, an efficient GPU sampler for Top-k and Top-p via pivot-based truncation and selection. Sorting burdens GPUs, while stochastic methods alter outputs. Qrita searches by pivots for Top-k and Top-p, using Gaussian sigma-truncation to shrink vocabulary search. Quaternary pivot search handling duplicates halves iterations and guarantees determinism. Built in Triton, tested against SGLang and FlashInfer, it lifts serving throughput up to 1.4 times, halving memory, matching sorting. Qrita is now vLLM's default Top-k and Top-p sampler.
Catalog record
| CREATOR | Jongseok Park, Sunga Kim, Alvin Cheung, Ion Stoica |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64008 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.01518