Train for Truth, Keep the Skills: Binary Retrieval-Augmented Reward Mitigates Hallucinations

Tong Chen, Akari Asai, Luke Zettlemoyer, Hannaneh Hajishirzi, Faeze Brahman · 2025

Read Train for Truth, Keep the Skills: Binary Retrieval-Augmented Reward Mitigates Hallucinations by Tong Chen, Akari Asai, Luke Zettlemoyer, Hannaneh Hajishirzi, Faeze Brahman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17733.txt.

Description

Tong Chen and four co-authors work on a common flaw in language models. Models often state false claims that their training data never supported. The authors call this extrinsic hallucination, meaning invented facts beyond the source. Past fixes often weakened writing quality or other skills.

Their method rewards a model only when its whole answer is correct. A retrieval step checks the answer against outside sources first. On open-ended writing, false claims fell by about thirty-nine percent. On short questions, the model learned to say it did not know. Math, code, and instruction tasks kept their earlier quality levels.

Catalog record

CREATORTong Chen, Akari Asai, Luke Zettlemoyer, Hannaneh Hajishirzi, Faeze Brahman
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-17733

Browse the full library · About the Tower of Babel