Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuning

Wenhao Yu, Shaohang Wei, Jiahong Liu, Yifan Li, Minda Hu, Aiwei Liu, Hao Zhang, Irwin King · 2026

Read Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuning by Wenhao Yu, Shaohang Wei, Jiahong Liu, Yifan Li, Minda Hu, Aiwei Liu, Hao Zhang, Irwin King (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01745.txt.

Description

Wenhao Yu and colleagues improve guided fine tuning with token weighting, where single signals mislead. Ground truth probability reflects alignment, while token entropy reflects built in uncertainty from the training prior. Their RankTuner builds a probability entropy tuning signal, the rank signal, matching the ground truth token rank with its expected rank under guesses. Its inverse serves as a scale for weighting, focusing updates on under learned tokens without over punishing unsure spots. Tests on backbones show math tasks and out of domain transfer gains over single indicator baselines.

Catalog record

CREATORWenhao Yu, Shaohang Wei, Jiahong Liu, Yifan Li, Minda Hu, Aiwei Liu, Hao Zhang, Irwin King
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81666
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.01745

Browse the full library · About the Tower of Babel