Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning
Tianyi Wu, Jingwei Ni, Bryan Hooi, Jiaheng Zhang, Elliot Ash, See-Kiong Ng, Mrinmaya Sachan, Markus Leippold · 2025
Read Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning by Tianyi Wu, Jingwei Ni, Bryan Hooi, Jiaheng Zhang, Elliot Ash, See-Kiong Ng, Mrinmaya Sachan, Markus Leippold (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11962.txt.
Description
Tianyi Wu and seven co-authors balance two virtues of language models in twenty twenty-five. Instruction fine-tuning makes models more informative. It can also make them less truthful. The reason is that tuning data pushes models to produce long-tail knowledge they never learned well. On unseen tasks, that shows up as confident error.
The authors demonstrate the harm of unfamiliar knowledge in tuning sets. They then introduce two paradigms to counter it. One, UNIT-cut, finds and removes unfamiliar knowledge from the data. The other, UNIT-ref, trains models to flag their own uncertainty at the end of a response. Experiments show the trade-off can be eased. Readers get tuning recipes that protect honesty while keeping answers rich.
Catalog record
| CREATOR | Tianyi Wu, Jingwei Ni, Bryan Hooi, Jiaheng Zhang, Elliot Ash, See-Kiong Ng, Mrinmaya Sachan, Markus Leippold |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-11962