Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation
Hieu Nguyen, Zihao He, Shoumik Atul Gandre, Ujjwal Pasupulety, Sharanya Kumari Shivakumar, Kristina Lerman · 2025
Read Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation by Hieu Nguyen, Zihao He, Shoumik Atul Gandre, Ujjwal Pasupulety, Sharanya Kumari Shivakumar, Kristina Lerman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11306.txt.
Description
Hieu Nguyen and five co-authors curb hallucination in large language models with smoothed knowledge distillation. Models often state false or ungrounded claims, which bars them from high stakes use. Their work came out in twenty twenty-five.
Hard labels in training push fixed answers, breed overconfidence, and ignore the doubt built into language. In their fix, a teacher model gives soft labels to a student during fine tuning on teaching data. That softening trims overconfidence and firms factual grounding. Tests on summary tasks across model families cut hallucination versus standard fine tuning, while general task scores held, the authors report.
Catalog record
| CREATOR | Hieu Nguyen, Zihao He, Shoumik Atul Gandre, Ujjwal Pasupulety, Sharanya Kumari Shivakumar, Kristina Lerman |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-11306