Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuning
Wenhao Yu, Shaohang Wei, Jiahong Liu, Yifan Li, Minda Hu, Aiwei Liu, Hao Zhang, Irwin King · 2026
Read Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuning by Wenhao Yu, Shaohang Wei, Jiahong Liu, Yifan Li, Minda Hu, Aiwei Liu, Hao Zhang, Irwin King (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01745.txt.
Description
Wenhao Yu and colleagues improve guided fine tuning with token weighting, where single signals mislead. Ground truth probability reflects alignment, while token entropy reflects built in uncertainty from the training prior. Their RankTuner builds a probability entropy tuning signal, the rank signal, matching the ground truth token rank with its expected rank under guesses. Its inverse serves as a scale for weighting, focusing updates on under learned tokens without over punishing unsure spots. Tests on backbones show math tasks and out of domain transfer gains over single indicator baselines.
Catalog record
| CREATOR | Wenhao Yu, Shaohang Wei, Jiahong Liu, Yifan Li, Minda Hu, Aiwei Liu, Hao Zhang, Irwin King |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81666 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.01745