Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuning Wenhao Yu, Shaohang Wei, Jiahong Liu, Yifan Li, Minda Hu, Aiwei Liu, Hao Zhang, Irwin King Abstract Token-level reweighting is a simple yet effective mechanism for controlling supervised fine-tuning, but common indicators are largely one-dimensional: the ground-truth probability reflects downstream alignment, while token entropy reflects intrinsic uncertainty induced by the pre-training prior. Ignoring entropy can misidentify noisy or easily replaceable tokens as learning-critical, while ignoring probability fails to reflect target-specific alignment. RankTuner introduces a probability–entropy calibration signal, the Relative Rank Indicator , which compares the rank of the ground-truth token with its expected rank under the prediction distribution. The inverse indicator is used as a token-wise Relative Scale to reweight the fine-tuning objective, focusing updates on truly under-learned tokens without over-penalizing intrinsically uncertain positions. Experiments on multiple backbones show consistent improvements on mathematical reasoning benchmarks, transfer gains on out-of-distribution reasoning, and pre code generation performance over probability-only or entropy-only reweighting baselines. 1 Introduction With the remarkable progress of large language models (LLMs) in natural language processing ( 1 ; 7 ) , mathematical reasoning ( 5 ; 11 ) , code generation ( 2 ; 39 ) , and knowledge retrieval ( 20 ) , finetuning has become a pivotal technique ( 18 ; 38 ) for efficiently adapting the general capabilities of LLMs to specific downstream tasks. By selectively updating models with datasets of varying sizes and domains, finetuning significantly enhances accuracy, robustness, and practicality in targeted application areas, while also fulfilling requirements for safety and preference alignment ( 32 ) . Finetuning methodologies encompass a variety of paradigms such as supervised finetuning (SFT) ( 13 ; 12 ) and reinforcement learning (RL) ( 35 ) , and have been widely adopted in academic and industrial settings across multilingual ( 28 ; 19 ) , multitask ( 27 ) , and cross-modal scenarios ( 31 ; 33 ) . Recent token-level reweighting methods for fine-tuning can be broadly categorized into two paradigms based on the statistics they rely on: Prob-dominant weighting that designs w_{t} as a function of the ground-truth probability p_{t} ( 26 ; 40 ; 24 ) , and Entropy-dominant weighting that uses the predictive uncertainty H_{t} as the reweighting signal ( 8 ) . While effective in specific settings, prior work typically treats p_{t} and H_{t} in isolation, which can misidentify what to emphasize . Noisy tokens often fall into a “noise region” where neither one-dimensional signal is reliable: entropy-dominant schemes can up-weight them for high uncertainty, while prob-dominant schemes may overreact to atypical but irrelevant tokens. A controlled noise-insertion diagnostic shows both baselines surface injected noise far more than our indicator, as seen in Tab. 1 (detailed in App. B.3 ). Replaceable tokens (e.g., “essentially” vs. “basically”) are intrinsically ambiguous and high-entropy, so a low p_{t} need not indicate a true error—making prob-dominant weighting overly sensitive and potentially harmful to linguistic flexibility. These regimes motivate a calibrated token-importance signal that contextualizes likelihood by intrinsic uncertainty, down-weighting noisy/replaceable tokens while focusing updates on genuinely critical failures. Table 1: Noise sensitivity. Token noise precision/recall@10% and sequence noise hit@10% (App. B.3 ). Method Tok Prec@10% ( \downarrow ) Tok Rec@10% ( \downarrow ) Seq Hit@10% ( \downarrow ) Entropy-dominant 4.54\% 55.33\% 77\% Prob-dominant 3.25\% 39.65\% 77\% RankTuner (ours) \mathbf{2.16\%} \mathbf{26.39\%} \mathbf{9\%} Based on these findings, we propose RankTuner , a rank-guided token reweighting framework that calibrates downstream alignment by intrinsic uncertainty . The principal contributions of this paper are as follows: 1. We analyze Prob-Dominant vs. Entropy-Dominant token reweighting and show why one-dimensional weighting can over-emphasize noisy and replaceable tokens, since p_{t} (alignment) and H_{t} (intrinsic uncertainty) capture different factors (Sec. 3 ). 2. We introduce a rank-based view with a Relative Rank signal that compares the target-token rank against its expected rank, yielding an uncertainty-aware adaptive token reweighting scheme for fine-tuning (Sec. 4 ). 3. We validate RankTuner across base models and reasoning benchmarks, with ablations supporting the complementary roles of probability- and entropy-aware components (Sec. 5 ). 2 Preliminaries In this section, we establish the formal notation and theoretical foundation for our method. We first introduce a unified weighting framework for fine-tuning, followed by the core concepts of model uncertainty and the guessing problem. 2.1 Problem Formulation and Unified Weighting Consider a dataset \mathcal{D} consisting of prompt-response pairs (x,y) , where x is the input prompt and y=(y_{1},y_{2},\dots,y_{T}) is the target response of length T . Let \pi_{\theta} denote a language model parameterized by \theta . At each decoding step t\in\{1,\dots,T\} , the model generates a probability distribution over the vocabulary \mathcal{V} . We denote the probability assigned to the i -th token v_{i}\in\mathcal{V} as p_{t,i}=\pi_{\theta}(v_{i}\mid y_{0 . Consequently, \mathcal{I}_{t} admits a power-law form that directly connects it to the competence ratio: \mathcal{I}_{t}\;=\;2^{-K(\xi_{t})\cdot\log_{2}(R_{t}/\mathbb{E}[R_{t}])}\;=\;\left(\frac{\mathbb{E}[R_{t}]}{R_{t}}\right)^{K(\xi_{t})}. (7) For typical reasoning tokens where both R_{t} and \mathbb{E}[R_{t}] are small, we have K(\xi_{t})\approx 0.5 (see App. A.4 for analysis). Constructing rank-based surrogates \widehat{\rho} and \widehat{\kappa} . To operationalize C_{t} in terms of rank-based quantities, we define surrogates \widehat{\rho}(p_{t}) and \widehat{\kappa}(H_{t}) by directly exploiting the established bounds together with the coefficient K(\xi_{t}) from the Cauchy analysis. From Eq. ( 4 ) (which gives R_{t}\lesssim 1/p_{t} ) and the power-law structure revealed in Eq. ( 7 ), we set \widehat{\rho}(p_{t})\triangleq p_{t}^{K(\xi_{t})} as a proxy for R_{t}^{-K(\xi_{t})} . Similarly, letting s(H_{t}) denote the right-hand side of Eq. ( 5 ) (a lower bound for \mathbb{E}[R_{t}] ), we set \widehat{\kappa}(H_{t})\triangleq s(H_{t})^{-K(\xi_{t})} as a proxy for \mathbb{E}[R_{t}]^{-K(\xi_{t})} . With these definitions, the surrogate ratio \widehat{C}_{t}\;\triangleq\;\frac{\widehat{\rho}(p_{t})}{\widehat{\kappa}(H_{t})}\;=\;\bigl(p_{t}\,s(H_{t})\bigr)^{K(\xi_{t})} (8) approximates the competence ratio \left(\frac{\mathbb{E}[R_{t}]}{R_{t}}\right)^{K(\xi_{t})} by substituting the rank-based bounds into the power-law form. Relating \mathcal{I}_{t} to the competence score. Combining Eq. ( 7 ) with the surrogate construction above, we observe that the Relative Rank Indicator \mathcal{I}_{t} directly approximates the competence ratio: \mathcal{I}_{t}\;=\;\left(\frac{\mathbb{E}[R_{t}]}{R_{t}}\right)^{K(\xi_{t})}\;\gtrsim\;\left(\frac{s(H_{t})}{1/p_{t}}\right)^{K(\xi_{t})}\;=\;\widehat{C}_{t}. (9) In this manner, our approach bridges probability p_{t} and entropy H_{t} through a unified rank-based framework. The functions \widehat{\rho} and \widehat{\kappa} represent one concrete instantiation of the general template C_{t}=\rho(p_{t})/\kappa(H_{t}) , where the rank-to-probability/entropy correspondences are given by the theoretical bounds (Eqs. ( 4 ) and ( 5 )). This construction is validated by the empirical adherence observed in Fig. 2 . App. A.5 further justifies the surrogate substitution and establishes the boundedness/tightness guarantees. The resulting formulation enables practical application of competence-aware weighting in supervised fine-tuning, as we detail in the following subsection. 4.5 Implementation of Relative-Rank Guided Losses The Relative Rank Indicator \mathcal{I}_{t} measures token-level performance relative to uncertainty. For fine-tuning, we focus on its inverse: assigning larger weights to tokens where the model underperforms relative to expectation, while down-weighting already well-mastered tokens to avoid over-optimization. We term this weighting signal the Relative Scale ( \mathcal{S}_{t} ): \displaystyle\mathcal{S}_{t}\triangleq\mathcal{I}_{t}^{-1}\approx\bigl(p_{t}\cdot s(H_{t})\bigr)^{-K(\xi_{t})}, (10) \displaystyle\xi_{t}\coloneqq\max\!\bigl\{R_{t},\,s(H_{t})\bigr\}, \displaystyle K( \displaystyle\xi_{t})\coloneqq\bigl[\log_{2}(\xi_{t}+1)\bigr]^{-2}. For simplicity and training stability, we omit the multiplicative factor \frac{\xi_{t}}{\xi_{t}+1} in K(\xi_{t}) . Other approximations of \xi are discussed in App. C.5 . We incorporate the Relative Scale \mathcal{S}_{t} into supervised fine-tuning by modulating the token-level weighting coefficients. Following the unified formulation in Sec. 2 , we replace the original weight w_{t} with a variant: \widetilde{w}_{t}=w_{t}\cdot\mathcal{S}_{t}. (11) Algorithm B.1 provides the pseudocode for this procedure. Practically, we set w_{t}=p_{t} for all fine-tuning tasks on math reasoning datasets, and w_{t}=1 for general fine-tuning tasks; the rationale is provided in App. B.6 . 5 Experiments We design experiments to answer the following questions: (RQ1) Effectiveness : Does RankTuner consistently improve mathematical reasoning performance over the original models and representative probability-/entropy-based fine-tuning baselines across benchmarks and decoding budgets (Pass@1/Pass@16)? (RQ2) Out-of-distribution generalization : Does RankTuner generalize beyond mathematical reasoning to diverse reasoning benchmarks? (RQ3) Key ingredients : How do the probability-aware and entropy-aware components contribute to the gains, and how does RankTuner compare to loss-shaping alternatives? Table 2: Performance comparison on mathematical reasoning benchmarks. We report Pass@1 and Pass@16 metrics. Best results for each base model are in bold. The \Delta row shows the improvement of RankTuner over the Original base model. Model Method MATH-OAI Minerva Math OlympiadBench AIME24 AMC23 P@1 P@16 P@1 P@16 P@1 P@16 P@1 P@16 P@1 P@16 Qwen2.5-Math-7B Original 31.79 87.80 7.63 42.28 9.49 47.85 6.25 \mathbf{23.33} 20.47 \mathbf{85.00} SFT 53.52 88.20 17.74 50.00 19.14 55.11 2.08 10.00 24.53 82.50 EAFT 53.94 87.40 20.24 59.19 18.96 54.37 2.50 10.00 24.22 67.50 OverTone 47.00 87.80 18.89 51.47 16.02 51.56 2.50 20.00 25.16 75.00 DFT \mathbf{69.15} 85.00 26.06 40.07 32.62 54.81 4.17 16.67 41.09 72.50 TALR 68.83 87.40 \mathbf{35.68} \mathbf{60.29} 32.87 57.78 6.67 16.67 43.44 77.50 RankTuner 68.60 \mathbf{88.80} 33.30 59.56 \mathbf{32.89} \mathbf{62.07} \mathbf{7.08} \mathbf{23.33} \mathbf{44.53} 82.50 \Delta \uparrow 36.81 \uparrow 1.00 \uparrow 25.67 \uparrow 17.28 \uparrow 23.40 \uparrow 14.22 \uparrow 0.83 \uparrow 0.00 \uparrow 24.06 \downarrow 2.50 Qwen3-8B Original 65.14 87.40 31.39 48.53 27.19 51.11 6.04 \mathbf{26.67} 35.62 75.00 SFT 54.83 88.60 21.42 54.04 20.13 53.33 2.71 16.67 26.25 67.50 EAFT 55.23 \mathbf{90.20} 23.85 63.60 19.97 52.59 3.33 13.33 28.75 80.00 OverTone 35.58 82.80 17.78 57.35 11.43 44.74 1.25 13.33 16.72 67.50 DFT 70.92 86.00 32.42 47.79 35.07 58.22 8.75 16.67 45.78 75.00 TALR 70.12 89.40 \mathbf{40.46} 61.03 34.38 60.00 7.29 \mathbf{26.67} 43.75 80.00 RankTuner \mathbf{72.38} \mathbf{90.20} 38.26 \mathbf{65.44} \mathbf{36.25} \mathbf{64.00} \mathbf{10.21} \mathbf{26.67} \mathbf{46.56} \mathbf{85.00} \Delta \uparrow 7.24 \uparrow 2.80 \uparrow 6.87 \uparrow 16.91 \uparrow 9.06 \uparrow 12.89 \uparrow 4.17 \uparrow 0.00 \uparrow 10.94 \uparrow 10.00 5.1 Experimental Setup Following prior work ( 40 ) , we train on the NuminaMath-CoT dataset ( 17 ) using the first 10k training instances. We run experiments with multiple base models, including Qwen2.5-Math-7B ( 42 ) and Qwen3-8B ( 41 ) . We further report supplementary cross-architecture results in the Tab. 8 . Implementation Details. Our implementation is built on the verl framework ( 36 ) . All experiments can be completed on four NVIDIA A800-SXM4-80GB GPUs. We use the AdamW optimizer with a learning rate of 5\times 10^{-5} for all models. We set the global mini-batch size to 256 and the maximum input length to 2048 tokens. The learning rate follows a cosine decay schedule with a warm-up ratio of 0.1. For evaluation, we generate 16 decoding runs with temperature 1.0 and maximum generation length of 4096 tokens, and report Pass@1 and Pass@16 (see App. C.3 for the Pass@ k definition). We evaluate on Math500 ( 23 ) , Minerva Math ( 21 ) , OlympiadBench ( 10 ) , AIME 2024, and AMC 2023. Baselines and Metrics. We compare RankTuner against standard SFT and representative token-level loss reweighting baselines. In particular, OverTone, DFT, and TALR are probability-dominant weighting schemes driven primarily by the ground-truth token probability p_{t} (possibly with gating/temperature), while EAFT is an entropy-dominant scheme that weights tokens based on (top- K ) predictive entropy (see App. C.2 for more detailed comparisons of the baselines). We report Pass@ k (mainly Pass@1 and Pass@16), i.e., the probability that at least one out of k sampled solutions is correct (see App. C.3 for the Pass@ k definition and computation). Figure 3: Ablations, baselines, and inference entropy on AIME24 and OlympiadBench. Left: We report Pass@1/Pass@16 and compare RankTuner with tuned Alpha Power ( \alpha{=}0.5 ) and Entropy Reg ( \alpha{=}0.02 ). Middle: We plot AIME24 Pass@k and further include two RankTuner ablations (w/o Prob, w/o Entropy), highlighting complementary roles of the probability- and entropy-aware terms. Right: We measure average inference entropy on AIME24 for Qwen2.5-Math-7B; the dashed line indicates the original (pre-finetuning) model and colors group methods by probability orientation ( P-decay , P-neutral , P-boost ). 5.2 RQ1: Effectiveness on Reasoning Tasks Tab. 2 compares RankTuner with representative probability- and entropy-based fine-tuning baselines across five mathematical reasoning benchmarks. Across both backbones, RankTuner delivers consistent improvements over the original models, with particularly strong gains in Pass@1 on MATH-OAI, Minerva Math, and OlympiadBench; meanwhile, it also boosts Pass@16 on most benchmarks, indicating that the improved single-sample accuracy does not come at the expense of multi-sample coverage. Notably, on AIME24 —a comparatively hard benchmark where several baselines exhibit substantial degradation (e.g., reduced Pass@16 and/or Pass@1)— RankTuner maintains the original Pass@16 while still improving Pass@1, suggesting a more robust calibration of learning signals that avoids over-correcting intrinsically uncertain positions. We observe one mild trade-off on Qwen2.5-Math-7B for AMC23 Pass@16, which decreases slightly despite a large Pass@1 gain; overall, RankTuner achieves the best or near-best performance across the majority of benchmark–metric pairs. 5.3 RQ2: Out-of-Distribution Generalization To evaluate the generalization capability of RankTuner beyond mathematical reasoning, we conduct experiments on two diverse reasoning benchmarks: ARC-C ( 4 ) and GPQA ( 34 ) . Following our experimental protocol, we set the sampling temperature to 0.8 , generate 16 candidate responses per query, and use a maximum token budget of 3072 to accommodate comprehensive reasoning traces. Tab. 3 summarizes the Pass@1 performance of various methods on Qwen2.5-Math-7B. Table 3: Out-of-distribution evaluation on ARC-C and GPQA using Qwen2.5-Math-7B. We report Pass@1 accuracy (higher is better). Best results are in bold and second-best results are underlined. Benchmark Original SFT DFT EAFT TALR RankTuner ARC-C 13.46 42.30 26.50 48.57 \underline{52.54} \mathbf{53.58} GPQA 7.86 25.00 27.90 25.63 \underline{29.29} \mathbf{29.64} Overall, Tab. 3 shows that RankTuner achieves the best on ARC-C and GPQA, indicating robust out-of-distribution transfer beyond math reasoning. In contrast, DFT is a probability-dominant reweighting method that prioritizes already-confident tokens, which can induce over-sharpening and hurt generalization under distribution shift. Reweighted by relative rankings rather than a single signal, RankTuner provides a richer and less distribution-specific training objective that preserves general reasoning ability. 5.4 RQ3: Key Ingredients of RankTuner To isolate the effect of each ingredient, we conduct ablations on Qwen2.5-Math-7B. We compare against two strong loss-level baselines that reflect common alternatives to reweighting: Alpha Power Loss reshapes the token loss as (1-p^{\alpha})/\alpha (here \alpha{=}0.5 ), while Entropy Regularization augments CE with an entropy bonus -\alpha H(p) (here \alpha{=}0.02 ) to encourage exploration/diversity. Fig. 3 summarizes the results. The left and middle panels compare RankTuner with the two tuned baselines on AIME24 and OlympiadBench (Pass@1/Pass@16) and show AIME24 Pass@k, where RankTuner achieves consistent gains, especially at Pass@16. The middle panel further includes two ablated variants: RankTuner w/o Prob (dropping the probability term p_{t}^{-K(\xi_{t})} ) can slightly improve Pass@1 but yields weaker improvements as k grows, indicating reduced sample diversity/coverage; in contrast, RankTuner w/o Entropy (dropping the entropy term H_{t}^{-K(\xi_{t})} ) degrades across all k , showing the entropy component is essential for robust Pass@k gains. 5.5 Inference-time Entropy Analysis We study how inference-time token entropy changes after fine-tuning, and how it correlates with probability-oriented weighting designs. On Qwen2.5-Math-7B , we compute the average predictive entropy on AIME24 and then average over tokens. Specifically, we sample 8 decoding runs per prompt with temperature 0.2 and report entropy averaged across runs and tokens. The right panel of Fig. 3 shows a striking pattern: the finetuned model’s inference entropy is highly aligned with how weighting “steers” probability, forming three distinct signatures ( P-decay / P-neutral / P-boost ). OverTone ( P-decay ) yields the highest entropy—consistent with the idea that, in a model-strong reasoning setting, over-emphasizing currently-wrong tokens can amplify noisy supervision and make the model more “confused” ( 22 ) . In contrast, SFT/EAFT ( P-neutral ) exhibit a mild entropy rise; notably, EAFT being entropy-weighted does not automatically translate to lower post -finetuning inference entropy. Finally, P-boost methods reduce entropy, but with very different “sharpness”: DFT shows the most aggressive entropy collapse. TALR uses a dynamic exponent, but it is still driven mainly via p_{t} and does not explicitly account for token-type priors; whereas RankTuner stays closest to the original baseline by coupling the probability exponent to an uncertainty-linked term K(\xi_{t}) tied to the rank-based proxy \mathbb{E}[R_{t}] . 6 Conclusion We present RankTuner , a rank-guided token reweighting framework that calibrates downstream alignment by intrinsic uncertainty. By discretizing probability and entropy into a commensurate rank-based pair—the ground-truth rank and its expected rank—we derive a Relative Rank Indicator and use its inverse as a token-wise Relative Scale to focus updates on genuinely under-learned critical tokens, while down-weighting noisy or replaceable positions. Across multiple backbones and reasoning benchmarks, RankTuner achieves consistent gains over probability- or entropy-only reweighting baselines, and our ablations and entropy-based behavioral analysis highlight the complementary roles of both components in improving accuracy without collapsing diversity. These results suggest that probability–entropy calibration offers a simple and effective principle for adaptive fine-tuning, and this perspective is promising to generalize to broader tasks and training paradigms. Impact Statements This paper presents a token-level reweighting method for supervised fine-tuning, aiming to improve training stability and downstream reasoning performance by calibrating probability- and entropy-based signals. As a general optimization technique, our approach may help practitioners build more reliable and sample-efficient models for scientific and educational applications. At the same time, improved fine-tuning procedures can contribute to increased capabilities of language models (e.g., mathematical reasoning or code generation), which may be misused in downstream settings. We therefore recommend that any deployment follow established responsible-release practices (e.g., access control, monitoring, and usage policies) and comply with applicable laws and norms. Our work does not involve human subjects, and we conduct experiments using publicly available datasets and models. Appendix This appendix provides supplementary materials to support the main text, including additional theoretical details, algorithmic analysis, and extended experiments. Below we summarize what each part aims to accomplish: • Theoretical Analysis ( A ): proofs and derivations that connect rank-, probability-, and entropy-based quantities. – Rank–probability bound ( A.1 ). – Expected rank–entropy bound ( A.2 ). – Cauchy Mean Value Theorem derivation ( A.3 ). – Coefficient analysis across different regimes ( A.4 ). – Boundedness/tightness under surrogate substitution ( A.5 ). • Pseudocode and Analysis ( B ): implementation-oriented details, complexity analysis, and diagnostics/visualizations. – Pseudocode ( B.1 ). – Time and memory complexity ( B.2 ). – Noise sensitivity diagnostic ( B.3 ). – Token-level visualization of difficulty and correctness ( B.4 ). – Experimental validation of tightness of bounds ( B.5 ). – Rationale of selections of initial weight for different tasks ( B.6 ). • Supplementary Experiments ( C ): additional experimental details and results. – Dataset statistics ( C.1 ). – Metrics ( C.3 ). – Supplementary cross-architecture results for mathematical reasoning ( C.4 ). – Selections of the \xi approximation ( C.5 ). – Code fine-tuning and evaluation ( C.6 ). Appendix A Theoretical Analysis A.1 Rank–Probability Bound Lemma A.1 (Rank–Probability Bound) . Let the probability distribution at position t be sorted such that p_{t,\hat{1}}\geq p_{t,\hat{2}}\geq\cdots . For the ground-truth token with probability p_{t} and rank R_{t} , we have R_{t}\leq 1/p_{t} (Eq. ( 4 )). Proof. Let the probabilities be sorted in non-increasing order as p_{t,\hat{1}}\geq p_{t,\hat{2}}\geq\cdots . Let the ground-truth token at position t have probability p_{t} , and let R_{t} denote its (1-indexed) rank in this sorted list (breaking ties arbitrarily). Then p_{t,\hat{R_{t}}}=p_{t} , and for every i\leq R_{t} we have p_{t,\hat{i}}\geq p_{t,\hat{R_{t}}}=p_{t} . Therefore, 1=\sum_{i\geq 1}p_{t,\hat{i}}\geq\sum_{i=1}^{R_{t}}p_{t,\hat{i}}\geq\sum_{i=1}^{R_{t}}p_{t}=R_{t}p_{t}, (12) which implies R_{t}\leq 1/p_{t} . A.2 Expected Rank–Entropy Bound Lemma A.2 (Expected Rank–Entropy Bound) . The expected rank \mathbb{E}[R_{t}] satisfies Eq. ( 5 ). Proof. Let the probability distribution over the vocabulary at position t be sorted in non-increasing order, p_{t,\hat{1}}\geq p_{t,\hat{2}}\geq\cdots , and recall that p_{\max,t}\triangleq p_{t,\hat{1}} . Define a random variable R_{t}\in\{1,2,\ldots\} whose distribution is given by this sorted list: \Pr(R_{t}=i)\triangleq p_{t,\hat{i}},\qquad i\geq 1. (13) Then \mathbb{E}[R_{t}]=\sum_{i\geq 1}i\,p_{t,\hat{i}} , and the Shannon entropy (in bits) is H_{t}=-\sum_{i\geq 1}p_{t,\hat{i}}\log_{2}p_{t,\hat{i}}. (14) Case 1: H_{t}\geq 2 . Set A\triangleq\mathbb{E}[R_{t}] . Consider the set of (not necessarily monotone) distributions \{p_{i}\}_{i\geq 1} on \{1,2,\ldots\} with mean constraint \sum_{i\geq 1}i\,p_{i}=A . It is a classical maximum-entropy result, due to Jaynes ( 16 ) and widely used in the guessing literature ( 30 ) , that under a fixed mean (average-energy) constraint the unique entropy maximizer is the geometric (Boltzmann) distribution p^{\mathrm{geom}}_{i}=\frac{1}{A-1}\left(1-\frac{1}{A}\right)^{i},\qquad i\geq 1, (15) which indeed satisfies \sum_{i\geq 1}p^{\mathrm{geom}}_{i}=1 and \sum_{i\geq 1}i\,p^{\mathrm{geom}}_{i}=A . Therefore, for any distribution with mean A (in particular, our \{p_{t,\hat{i}}\} ), H_{t}\leq h\!\left(p^{\mathrm{geom}}\right), (16) where h(\cdot) denotes entropy in bits. For the geometric distribution ( 15 ), a direct calculation gives h\!\left(p^{\mathrm{geom}}\right)=\log_{2}(A-1)+A\log_{2}\!\left(\frac{A}{A-1}\right). (17) The function \phi(A)\triangleq A\log_{2}\!\left(\frac{A}{A-1}\right) is strictly decreasing for A>1 and satisfies \phi(2)=2 and \lim_{A\to\infty}\phi(A)=\log_{2}(e)<2 ; hence for all A\geq 2 , A\log_{2}\!\left(\frac{A}{A-1}\right)\leq 2. (18) Moreover, h(p^{\mathrm{geom}})\geq 2 if and only if A\geq 2 (with equality at A=2 ). Since we are in the regime H_{t}\geq 2 and H_{t}\leq h(p^{\mathrm{geom}}) by ( 16 ), we must have A\geq 2 , and thus ( 18 ) applies. Combining ( 16 ), ( 17 ), and ( 18 ), we obtain H_{t}\leq\log_{2}(A-1)+2. (19) Rearranging yields A=\mathbb{E}[R_{t}]\geq\tfrac{1}{4}\,2^{H_{t}}+1, (20) which is exactly the first case in Eq. ( 5 ). Case 2: H_{t}<2 . This regime follows from a simple decomposition on whether the guess is correct on the first try: \displaystyle\mathbb{E}[R_{t}] \displaystyle=1\cdot p_{t,\hat{1}}+\sum_{i\geq 2}i\,p_{t,\hat{i}} \displaystyle\geq 1\cdot p_{t,\hat{1}}+2\sum_{i\geq 2}p_{t,\hat{i}} \displaystyle=p_{\max,t}+2(1-p_{\max,t})=2-p_{\max,t}, (21) which matches the second case in Eq. ( 5 ). An entropy-only variant for the low-entropy regime. The low-entropy regime ( H_{t}<2 ) in Eq. ( 5 ) can be expressed purely in terms of H_{t} rather than p_{\max,t} . Define h(p)\triangleq H_{b}(p)+(1-p)\log_{2}(|\mathcal{V}|-1), (22) where H_{b}(p)\triangleq-p\log_{2}p-(1-p)\log_{2}(1-p) is the binary entropy. By Fano’s inequality (see, e.g., ( 6 ) ), among all distributions on |\mathcal{V}| outcomes with maximal mass p_{\max,t} , the entropy is maximized by placing the remaining mass uniformly over the other |\mathcal{V}|-1 outcomes. Therefore, \displaystyle H_{t} \displaystyle\leq-p_{\max,t}\log_{2}p_{\max,t}-(1-p_{\max,t})\log_{2}\!\left(\frac{1-p_{\max,t}}{|\mathcal{V}|-1}\right) \displaystyle=H_{b}(p_{\max,t})+(1-p_{\max,t})\log_{2}(|\mathcal{V}|-1) \displaystyle=h(p_{\max,t}), (23) Since h is strictly decreasing on [1/|\mathcal{V}|,1] , this inequality yields an upper bound p_{\max,t}\leq h^{-1}(H_{t}) . Substituting this into the second case of Eq. ( 5 ), we conclude that even in the H_{t}<2 regime, \mathbb{E}[R_{t}] is bounded from below by a function of entropy alone: \mathbb{E}[R_{t}]\geq 2-h^{-1}(H_{t}). (24) A.3 Cauchy Mean Value Theorem Derivation In this section, we provide the detailed derivation of Eq. ( 6 ) from the main text, which connects the difference f(R)-f(\mathbb{E}[R]) to the logarithmic ratio \log_{2}(R/\mathbb{E}[R]) via the Cauchy Mean Value Theorem. Setup and theorem statement. Let u(x)\triangleq f(x)=\frac{1}{\log_{2}(x+1)} be the transformation function used in defining the Relative Rank Indicator, and let v(x)\triangleq\log_{2}(x) be an auxiliary function. The Cauchy Mean Value Theorem states that if u and v are continuous on [\mathbb{E}[R],R] (assuming \mathbb{E}[R]0 and b>0 . On any compact domain bounded away from zero, F is Lipschitz continuous; hence the substitution (a,b)=(1/R_{t},\,1/\mathbb{E}[R_{t}])\mapsto(p_{t},\,1/s(H_{t})) induces a controlled change in \mathcal{I}_{t} that scales linearly with \Delta^{(p)}_{t} and \Delta^{(H)}_{t} (up to a constant depending on the chosen domain). As a future direction, one can further tighten this bound by restricting rank computations to an effective support (e.g., top- k ), which implicitly bounds R_{t} and \mathbb{E}[R_{t}] to a smaller range and can reduce computation from O(|\mathcal{V}|) to O(k) per token. Empirically, App. B.5 shows that both gaps are concentrated near zero on real model outputs, which supports the tightness of the surrogate substitution in Eq. ( 32 ). Appendix B Pseudocode and Analysis B.1 Pseudocode Algorithm B.1 presents the pseudocode for RankTuner-guided supervised fine-tuning. The key distinction from standard SFT lies in the computation of token-wise scale \mathcal{S}_{t} (Lines 4–10), which dynamically reweights each token based on its relative competence. Note that for simplification and training stability, we remove the \frac{\xi_{t}}{(\xi_{t}+1)} multiplier from the original formulation of K(\xi_{t}) . Algorithm 1 RankTuner-Guided Supervised Fine-Tuning 0: Model \mathcal{M}_{\theta} , dataset \mathcal{D} , original token weights \{w_{t}\} 1: for each batch \mathbf{x},\mathbf{y} from \mathcal{D} do 2: \mathbf{z}\leftarrow\mathcal{M}_{\theta}(\mathbf{x}) 3: for each token position t do 4: p_{t}\leftarrow p_{\theta}(y_{t}\mid\mathbf{x}_{0 and a\neq 1 , … Noise sentence (Alpaca). Aerobic and anaerobic exercise are two types of exercises that work differently on the body. Noisy response (excerpt). …therefore, f(x)-g(x) is an odd function. [noise inserted here] From f(x)-g(x)>0 , we get … B.4 Token-Level Visualization of Difficulty and Correctness Figure 4: Two-dimensional view of token difficulty and correctness. (Left) Token-level visualization on a partial reasoning trace from Qwen3-8B on AIME24, reporting p_{t} , H_{t} , and the proposed unified indicator I_{t} (formalized in Sec. 4 ). The three rows correspond to p_{t} , H_{t} , and I_{t} , respectively. Colors encode relative magnitude (blue \rightarrow larger, red \rightarrow smaller); arrows indicate the ascending direction (note H_{t} is reversed). I_{t} is normalized around a neutral value of 1 . This example illustrates how the three signals complement each other on real model text. Most arithmetic and connective tokens in this span have high p_{t} and low H_{t} , so the unified indicator stays close to the neutral level ( I_{t}\approx 1 ), suggesting locally “easy” and confident predictions. In contrast, atypical or formatting-related tokens (e.g., the “Putting” token and the LaTeX macro fragment near the final boxed answer) exhibit sharply reduced p_{t} and increased uncertainty (higher H_{t} ), and are highlighted by a noticeable deviation of I_{t} away from 1 . Overall, I_{t} provides a single, normalized view that surfaces token-level difficulty while still being sensitive to correctness cues from p_{t} . B.5 Experimental Validation of Tightness of Bounds We empirically validate the tightness of the two key bounds used throughout the paper (Sec. 4.3 ) by measuring their approximation errors on chain-of-thought tokens from Minerva Math predicted by Qwen3-8B. Specifically, we examine: (i) the rank–probability gap \frac{1}{R_{t}}-p_{t}\in[0,1) , which probes how well 1/R_{t} serves as a discrete surrogate for the ground-truth probability p_{t} ; and (ii) the inverse expected-rank gap \frac{1}{s(H_{t})}-\frac{1}{\mathbb{E}[R_{t}]}\in[0,1) , where s(H_{t}) is the entropy-based lower bound on \mathbb{E}[R_{t}] defined in Eq. ( 5 ). Fig. 5 and Tab. 5 show that both approximation gaps are concentrated near zero (computed over 4k+ tokens): the median errors are 0.0259 for 1/R-p and 0.0093 for 1/s(H)-1/\mathbb{E}[R] , and even at the 90th percentile the errors remain moderate ( \leq 0.348 and \leq 0.297 , respectively). This supports our use of rank-based surrogates: 1/R_{t} is a practical proxy for p_{t} (consistent with the envelope R\leq 1/p ), and 1/\mathbb{E}[R_{t}] closely tracks its entropy-induced theoretical bound 1/S(H_{t}) , making either quantity a reliable stand-in for the other when constructing uncertainty-aware competence and scaling signals. Figure 5: Error distributions for bound tightness on Qwen3-8B (Minerva Math, tokens 0–29). (Left) Distribution of \frac{1}{R}-p (rank-based approximation of token probability). (Right) Distribution of \frac{1}{s(H)}-\frac{1}{\mathbb{E}[R]} , where s(H) is the entropy-based lower bound in Eq. ( 5 ) (so 1/s(H) is the corresponding theoretical bound on 1/\mathbb{E}[R] ). Table 5: Summary statistics of approximation errors (smaller is better). We report robust central tendency and moderate quantiles to highlight that the errors are typically small. Error Type Mean Median Std P80 P90 1/R-p 0.109776 0.025879 0.151621 0.228027 0.348145 1/s(H)-1/\mathbb{E}[R] 0.084548 0.009272 0.137787 0.167955 0.297379 B.6 Rationale of Selections of Initial Weight for Different Tasks For all fine-tuning tasks on math reasoning datasets, we set w_{t}=p_{t} as the initial weight, which corresponds to the ground-truth probability of the token. For general fine-tuning tasks, we set w_{t}=1 as the initial weight, which represents a uniform weighting scheme. We provide the rationale for these selections from three perspectives. Figure 6: (Left) Illustration of the distinction between knowledge region and noise region when setting w_{t}=p_{t} . For math reasoning tasks, setting w_{t}=p_{t} helps distinguish the knowledge region (high p_{t} ) from the noise region (low p_{t} ). For general tasks, if w_{t}=p_{t} is applied, the knowledge region (which lies in low p_{t} ) would be incorrectly delimited. (Right) Normalized logit-gradient magnitude W_{f}(p) as a function of the ground-truth probability p for three representative loss shapes. 1. A knowledge–noise separation view explains why we initialize w_{t}=p_{t} for math reasoning but w_{t}=1 for general tasks. For math reasoning datasets, most of the knowledge space lies in the high p_{t} region, indicating that the model is already well-aligned with the pretraining math datasets. As illustrated in Fig. 6 , setting w_{t}=p_{t} helps distinguish the knowledge region from the noise region and reduces the contribution of noise. In contrast, for most common tasks, the majority of the knowledge space resides in the low p_{t} region. Therefore, setting w_{t}=1 preserves the basic trend of the NLL loss, and the gradient will be more assigned to the low p_{t} region. 2. An importance-sampling view of SFT suggests w_{t}=p_{t} is a variance-stable starting point, and composes naturally with our scale. Standard SFT takes gradients under a fixed demonstration distribution. Following ( 40 ) , we can rewrite the SFT gradient as an on-policy expectation under the model distribution by inserting the importance ratio between the Dirac-delta action distribution and the model policy: \mathbb{E}_{(x,y)\sim\mathcal{D}}\!\left[-\nabla_{\theta}\log\pi_{\theta}(y_{t}\mid y_{