Exploring the Generalizability of Factual Hallucination Mitigation via Enhancing Precise Knowledge Utilization

Siyuan Zhang, Yichi Zhang, Yinpeng Dong, Hang SuDept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center,THBI Lab, BNRist Center, Tsinghua University, Beijing 100084, China · 2025

Read Exploring the Generalizability of Factual Hallucination Mitigation via Enhancing Precise Knowledge Utilization by Siyuan Zhang, Yichi Zhang, Yinpeng Dong, Hang SuDept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center,THBI Lab, BNRist Center, Tsinghua University, Beijing 100084, China (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19127.txt.

Introduction

Siyuan Zhang and co-authors tackle false facts in large language model answers, in this 2025 paper. Rather than patching single errors, they train the model to use what it already knows with more precision. Their method fine-tunes the model on its own answers to simple, exact factual questions, favoring the precise ones.

They also built a large Chinese factual question set spanning twenty-one domains, for both training and testing. After training, models did better on factual tasks of many forms, and on general tasks too. Gains held across languages. The authors argue that sharpening knowledge use beats narrow fixes for this problem.

Catalog record

CREATORSiyuan Zhang, Yichi Zhang, Yinpeng Dong, Hang SuDept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center,THBI Lab, BNRist Center, Tsinghua University, Beijing 100084, China
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-19127

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Exploring the Generalizability of Factual Hallucination Mitigation via Enhancing Precise Knowledge Utilization Siyuan Zhang, Yichi Zhang, Yinpeng Dong, Hang SuDept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center,THBI Lab, BNRist Center, Tsinghua University, Beijing 100084, China Abstract Large Language Models (LLMs) often struggle to align their responses with objective facts, resulting in the issue of factual hallucinations , which can be difficult to detect and mislead users without relevant knowledge. Although post-training techniques have been employed to mitigate the issue, existing methods usually suffer from poor generalization and trade-offs in other different capabilities. In this paper, we propose to address these by directly augmenting LLM’s fundamental ability to precisely leverage its knowledge and introduce PKUE ( P recise K nowledge U tilization E nhancement), which fine-tunes the model on self-generated responses to precise and simple factual questions through preference optimization. Furthermore, we construct FactualBench , a comprehensive and precise factual QA dataset containing 181k Chinese data spanning 21 domains, to facilitate both evaluation and training. Extensive experiments demonstrate that PKUE significantly improves LLM overall performance, with consistent enhancement across factual tasks of various forms, general tasks beyond factuality, and tasks in different language. 1 Introduction Figure 1: Previous methods on factual hallucination mitigation exhibit poor generalizability across different factual tasks and suffer from degradations on comprehensive abilities and helpfulness, while our method PKUE improves model performance on all seven benchmarks, with a significant advantage on Avg. Factual hallucinations occur when Large Language Models (LLMs) generate inaccurate or entirely fabricated contents in response to queries ( Zhang et al., 2023b ; Huang et al., 2023 ) , which can undermine user trust in models and cause significant harm, especially when LLMs are deployed in high-stake applications ( Ji et al., 2023 ; Ahmad et al., 2023 ; Kang and Liu, 2023 ) . Furthermore, identifying hallucinations is challenging, as the fabricated contents are often presented plausibly and convincingly, making it difficult for both models and users to recognize inaccuracies ( Kaddour et al., 2023 ; Zhang et al., 2023b ) , which emphasizes the essentiality of mitigating hallucinations. Among various approaches to mitigate factual hallucinations, from pre-training ( Gardent et al., 2017 ; Wang, 2019 ) to inference-time techniques ( Nakano et al., 2021 ; Chuang et al., 2024 ) , post-training ( Lin et al., 2024a ; Tian et al., 2024 ) has become popular for not requiring large-scale data manipulation or additional runtime computations. Recent methods typically enhance factuality by training on open-ended questions, e.g., “Tell me a bio of an entity ”, which are broad and imprecise. They leave additional space for models to provide answers with diverse contents, subsequently assessed using average factual precision metrics like FActScore ( Min et al., 2023 ) . However, as shown in Figure 1 , these methods lead to declines in other factuality-related tasks and trade-offs in overall performance. The poor generalization can be attributed to the biased signals of these metrics mixing accuracy with length ( Wei et al., 2024b ) . Besides, wrong judgments on the correctness of atomic facts, which are caused by the entity ambiguity after response decomposition ( Chiang and Lee, 2024 ; Wanner et al., 2024 ) and the lack of standard answers in open-ended QA, degrade training effects. Moreover, the inherent noise in the training pairs, as both factually correct and incorrect atomic facts are mixed within a single response, can ultimately reduce the effectiveness of alignment ( Gu et al., 2025 ) . The trade-offs in general performance are not alleviated with additional training on advanced abilities ( Zhao et al., 2023 ) related to factuality under adversarial queries ( Zhang et al., 2024 ) or complex instructions following ( Lin et al., 2024a ) , as they are not necessary for other tasks and can lead to forgetting of the acquired abilities ( Ouyang et al., 2022 ; Lin et al., 2024b ) . In this paper, we address the above issues from the perspective of knowledge utilization and propose PKUE ( P recise K nowledge U tilization E nhancement) to enhance this capability with the task of precise QA. Since knowledge utilization is a crucial factor for factuality ( Wang et al., 2023 ) and a fundamental ability of LLM Zhao et al. (2023) , its enhancement is expected to bring generalized improvement beyond hallucination mitigation. We take precise fact-seeking QA as a representative task, which is short-form with standard answers and simple without other attributes besides the correctness. These features make it a proper task to reflect factual hallucination as well as knowledge utilization ( Roberts et al., 2020 ; Ji et al., 2023 ; Zhao et al., 2023 ) aside from other abilities and prevent the mentioned issues of imprecise open-ended QA. To better enhance utilization rather than inject external information, we leverage the model’s existing knowledge and conduct Direct Preference Optimization (DPO) ( Rafailov et al., 2023 ) training on sampled data from the model itself, which provides more granular bi-directional controls and better generalization ( Zhang et al., 2023b ; Chu et al., 2025 ) than uni-directional Supervised Fine-tuning (SFT). This self-aligned approach preserves the model’s distribution and limits the post-training shift, avoiding undesirable behaviors ( Gudibande et al., 2023 ; Zhang et al., 2023b ) and additional hallucinations introduced by training on external new knowledge ( Huang et al., 2023 ; Lin et al., 2024a ; Gekhman et al., 2024 ) . However, a precise factual QA dataset with large scale and diverse domains is lacking for training. Existing ones ( Yang et al., 2015 ; Joshi et al., 2017 ; Yang et al., 2018 ; Kwiatkowski et al., 2019 ) are usually outdated and fall short in fine-grained domain annotations, limiting their accuracy and diversity. To this end, we build FactualBench , a large-scale dataset with 181k Chinese QA data spanning 21 domains 1 1 1 We release our dataset in https://github.com/ZSYNOTZSH/FactualBench . Chinese is selected since it is a widely used language with a large community but still lacks high-quality datasets. We extract knowledge from the Internet encyclopedia, a widely used pre-training corpus ( Liu et al., 2024b ; Ando et al., 2024 ) and can be taken as a knowledge base that LLM has already seen. Multiple filtering strategies are adopted to ensure data quality. Evaluations on FactualBench reveal that while the task is not easy for LLMs, sampling with a higher temperature can yield more correct answers, which leaves space for better utilization of existing knowledge through self-alignment. Extensive experiments on Qwen2 ( Yang et al., 2024a ) and Baichuan ( Yang et al., 2023 ) show that only PKUE achieves consistent improvement on seven benchmarks covering factuality, helpfulness, and general skills in different forms and languages, presenting the best generalization. Notably, PKUE obtains 4 \times and 9 \times average improvement compared to existing methods ( Min et al., 2023 ; Lin et al., 2024a ; Zhang et al., 2024 ) . More ablation studies confirm the choices of self-generated data and DPO training. Our work proves that improving the knowledge utilization on solely precise and simple QA can promote generalized enhancement, spanning from generative tasks to diverse

Read the complete text