Min- k k Sampling: Decoupling Truncation from Temperature Scaling via Relative Logit Dynamics

Yuanhao Ding Meimingwei Li Esteban Garces Arias Matthias Aßenmacher, Christian Heumann Chongsheng Zhang, [1.5ex] School of Computer, Information Engineering, Henan University · 2026

Read Min- k k Sampling: Decoupling Truncation from Temperature Scaling via Relative Logit Dynamics by Yuanhao Ding Meimingwei Li Esteban Garces Arias Matthias Aßenmacher, Christian Heumann Chongsheng Zhang, [1.5ex] School of Computer, Information Engineering, Henan University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11012.txt.

Introduction

Ding and five colleagues propose Min-k sampling, a dynamic cut-off rule separating truncation from temperature scaling. Mainstream top-p, top-k, and min-p methods balance diversity and accuracy but are sensitive to temperature. Recent logit-space approaches gain temperature invariance yet rely on global statistics open to long-tail noise. Min-k finds semantic cliffs in the logit distribution, sharp shifts from confident core tokens to uncertain tail tokens. A decay rate sets the boundary each step, and the authors prove strict temperature invariance.

Catalog record

CREATORYuanhao Ding Meimingwei Li Esteban Garces Arias Matthias Aßenmacher, Christian Heumann Chongsheng Zhang, [1.5ex] School of Computer, Information Engineering, Henan University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59634
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11012

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Min- k k Sampling: Decoupling Truncation from Temperature Scaling via Relative Logit Dynamics Yuanhao Ding Meimingwei Li Esteban Garces Arias Matthias Aßenmacher, Christian Heumann Chongsheng Zhang, [1.5ex] School of Computer, Information Engineering, Henan University Abstract The quality of text generated by large language models depends critically on the decoding sampling strategy. While mainstream methods such as Top- k k , Top- p p , and Min- p p achieve a balance between diversity and accuracy through probability-space truncation, they share an inherent limitation: extreme sensitivity to the temperature parameter. Recent logit-space approaches like Top- n ​ σ n\sigma achieve temperature invariance but rely on global statistics that are susceptible to long-tail noise, failing to capture fine-grained confidence structures among top candidates. We propose Min- k k Sampling , a novel dynamic truncation strategy that analyzes the local shape of the sorted logit distribution to identify "semantic cliffs": sharp transitions from high-confidence core tokens to uncertain long-tail tokens. By computing a position-weighted relative decay rate, Min- k k dynamically determines truncation boundaries at each generation step. We formally prove that Min- k k achieves strict temperature invariance and empirically demonstrate its low sensitivity to hyperparameter choices. Experiments on multiple reasoning benchmarks, creative writing tasks, and human evaluation show that Min- k k consistently improves text quality, maintaining robust performance even under extreme temperature settings where probability-based methods collapse. We make our code, models, and analysis tools publicly available. 1 1 1 https://github.com/YecanLee/Mink 1 Introduction The quality of texts generated by large language models (LLMs) crucially depends on the sampling strategy used during decoding. Established methods such as Top- k 11 , Top- p (nucleus sampling) 16 , and Min- p 23 strike a balance between diversity and accuracy by truncating the probability space. However, these probability-based approaches share an inherent limitation: they are still extremely sensitive to the temperature parameter T , whose magnitude controls the probability distribution over candidate tokens and thus directly influences the stochasticity of next-token sampling 1 ; 5 ; 3 . As illustrated in Figure 1 (b, d, f, h), using the LLaMA-3-8B-Instruct model 10 , we evaluate different probability-based sampling methods under both a high-confidence context ("The capital of France is") and a low-confidence context ("The next word could be") . As T increases, the probability distribution becomes flatter, and Top- p as well as Min- p are more likely to (mistakenly) include a large number of low-quality noise tokens within the sampling space. Although Top- k maintains a fixed number of candidates, its reliance on a static k makes it insensitive to dynamic confidence, and its stochasticity remains subject to T , which consequently degrades the overall quality of the generated text. As will be shown quantitatively in Section 5.5 , these methods experience near-total semantic collapse (noise rate > 90%) when temperature exceeds 2.0. \begin{overpic}[hightconfident_4fig.png] \put(12.0,-1.2){(a)} \put(38.0,-1.2){(b)} \put(62.0,-1.2){(c)} \put(87.0,-1.2){(d)} \end{overpic} \begin{overpic}[lowconfident_4fig.png] \put(11.0,-1.2){(e)} \put(37.0,-1.2){(f)} \put(62.0,-1.2){(g)} \put(87.0,-1.2){(h)} \end{overpic} Figure 1: Comparison of decoding methods under different temperature and confidence conditions. See Appendix A for detailed token information. Top: High-confidence case, where Min- k accurately selects correct tokens and remains temperature-invariant. Bottom: Low-confidence case, where Min- k identifies semantic boundaries while reducing noise tokens. To alleviate this issue, recent work has introduced the Top- n\sigma sampling 28 , which operates directly in the logit space. This approach establishes a threshold determined by the maximum logit value and the global standard deviation, thereby achieving invariance to temperature scaling. However, Top- n\sigma relies on the global \sigma , making its truncation decision susceptible to interference from background noise within the long-tail logit distribution. Consequently, it fails to accurately locate the " semantic cliff "—a phenomenon characterizing the sharp drop in probability mass that separates high-confidence candidates from the unreliable long-tail noise 16 ; 15 ; 30 ; 23 . Furthermore, the choice of the hyperparameter n directly influences the trade-off between exploration and precision. As illustrated in Figure 1 (a, c, e, g), under both high-confidence and low-confidence semantic contexts, the default setting of n=1.0 (as in the original paper) introduces semantically irrelevant noise, while n=2.0 (still within the range of the paper) further enlarges the candidate pool and amplifies the presence of noise tokens. Desired properties of an improved sampling method include (1) preserving temperature invariance and reducing sensitivity to hyperparameter tuning, and (2) faithfully reflecting internal confidence cliffs among top candidates. Existing truncation methods conflate two distinct effects of temperature scaling: diversifying among plausible candidates and admitting noise tokens from the tail. An ideal truncation mechanism should decouple these two effects, allowing temperature to control only diversity within the plausible candidate set. To this end, we propose Min- k Sampling , a dynamic truncation strategy that directly analyzes the local shape of the sorted logit distribution to determine the optimal truncation point. Our method computes a position-weighted relative decay rate to detect the most pronounced semantic cliff within the logit sequence and applies truncation at this dynamically determined boundary. Because this method is relative in nature, it achieves temperature invariance. More importantly, by focusing on the local head structure of the distribution, it allows the sampling decision to dynamically adapt to the model’s confidence variation at each generation step, effectively identifying semantic boundaries. Contributions. 1. We propose Min- k sampling , a novel method that analyzes the local shape of the logit distribution to locate the "semantic cliff," enabling adaptive truncation and reducing semantic noise in generated texts. 2. We show that Min- k is temperature-invariant (§ 3.4 ) and exhibits low sensitivity to hyperparameters, maintaining stable performance across different settings (§ B , § 5.4 ). 3. Experiments on multiple datasets and the Alpaca2.0 benchmark demonstrate that Min- k consistently improves text quality and diversity under both low and high temperatures. 2 Related Work We categorize existing sampling-based decoding strategies into three broad families: probability-based, entropy-based, and logit-space-based: Probability-based Sampling. LLMs generate text by sampling from the probability distribution over their vocabulary. Classical temperature scaling 1 aims to balance generation diversity and determinism by adjusting the smoothness of this distribution. To further filter long-tail noise, the most straightforward approach, Top- k sampling 11 , restricts the candidate set to the k most likely tokens, but its fixed size cannot adapt to different contexts. To address this issue, Top- p (nucleus) sampling 16 dynamically selects the smallest set of tokens whose cumulative probability exceeds a given threshold, thereby achieving context adaptivity. However, Top- p is sensitive to temperature and tends to introduce low-quality tokens at high temperatures. More recently, Min- p 23 mitigates this issue to some extent via a dynamic threshold, but still fails to fully overcome the challenges posed by high temperatures 28 . These widely used sampling methods share a common limitation: their decision criteria directly depend on probability values that are themselves temperature-sensitive, leading to unstable performance under extreme temperatures. Moreover, 20 argue that probabilities only capture the relative strength between tokens rather than the model’s absolute confidence, which fundamentally limits the reliability of all these methods. Entropy-based Sampling. Entropy-based sampling methods adopt an information-theoretic perspective, utilizing entropy to guide token selection, thereby aiming to achieve a more principled balance between diversity and quality. Mirostat 2 adaptively adjusts the temperature parameter in

Read the complete text