Reducing Hallucinations in Summarization via Reinforcement Learning with Entity Hallucination Index

Praveenkumar Katwe, Rakesh Chandra Balabantaray, Kali Prasad Vittala · 2025

Read Reducing Hallucinations in Summarization via Reinforcement Learning with Entity Hallucination Index by Praveenkumar Katwe, Rakesh Chandra Balabantaray, Kali Prasad Vittala (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-22744.txt.

Introduction

Praveenkumar Katwe and co-authors tackle a common flaw in machine-made summaries: names that were never in the source. Their 2025 paper builds a score that checks which names appear in a summary and whether each one is right and backed by the text. The team then trains a model to earn a high score.

First, the team made draft summaries of meeting records and scored each one. Next, they used that score as a reward to tune the model toward names that match the source. Scores rose across data sets, and checks by eye showed far fewer false names. Fluency and detail held up, and the team shares its full test setup for reuse.

Catalog record

CREATORPraveenkumar Katwe, Rakesh Chandra Balabantaray, Kali Prasad Vittala
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-22744

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Reducing Hallucinations in Summarization via Reinforcement Learning with Entity Hallucination Index Praveenkumar Katwe, Rakesh Chandra Balabantaray, Kali Prasad Vittala Abstract Reducing hallucinations in abstractive summarization remains a critical challenge for deploying language models (LMs) in real-world settings. In this work, we introduce a reward-driven fine-tuning framework that explicitly optimizes for Entity Hallucination Index (EHI), a metric designed to quantify the presence, correctness, and grounding of named entities in generated summaries. Given a corpus of meeting transcripts, we first generate baseline summaries using a pre-trained LM and compute EHI scores via automatic entity extraction and matching. We then apply reinforcement learning to fine-tune the model parameters, using EHI as a reward signal to bias generation toward entity-faithful outputs. Our approach does not rely on human-written factuality annotations, enabling scalable fine-tuning. Experiments demonstrate consistent improvements in EHI across datasets, with qualitative analysis revealing a significant reduction in entity-level hallucinations without degradation in fluency or informativeness. We release a reproducible Colab pipeline, facilitating further research on hallucination-aware model fine-tuning using lightweight, hallucintion metrics like EHI. 1 Introduction Abstractive summarization models, powered by large language models (LLMs), have achieved impressive results across various domains. However, a persistent challenge remains: hallucination , where generated summaries include incorrect or fabricated information not grounded in the source input ( Maynez et al. 2020 ; Durmus et al. 2020 ) . In high-stakes applications such as meeting summarization, medical reporting, or financial documentation, hallucinations, particularly involving named entities, can significantly undermine trustworthiness and utility. Existing work on hallucination detection primarily focuses on coarse-grained factuality metrics or relies on reference-based evaluations ( Falke et al. 2019 ; Pagnoni et al. 2021 ) . While such methods are valuable, they often fail to capture fine-grained entity-level hallucinations, and their dependence on ground-truth references limits scalability. Recent efforts have explored lightweight automatic metrics, yet integrating such evaluations directly into model training remains underexplored. In this work, we propose a novel fine-tuning approach that leverages the Entity Hallucination Index (EHI) as a reward signal to reduce hallucinations at the entity level. EHI quantifies the correctness and grounding of entities by comparing extracted entities from model outputs to those present in the input document, enabling efficient evaluation with reduced reference. We first generate baseline summaries using a pre-trained model, compute EHI scores through automatic entity extraction and matching, and fine-tune the model using reinforcement learning with EHI as the reward function. This method biases the model toward producing more entity-faithful summaries without requiring human factuality annotations. Our contributions are as follows: • We introduce an EHI-guided fine-tuning framework that improves entity faithfulness in abstractive summarization. • We present a scalable reinforcement learning pipeline that does not depend on human-labeled factuality datasets. • We demonstrate empirical improvements in EHI scores across meeting transcript datasets and conduct qualitative analysis to highlight reductions in hallucination. • We release a reproducible Colab-based implementation to facilitate further research on hallucination-aware summarization. Our findings suggest that optimizing language models with fine-grained entity-focused rewards like EHI provides an effective pathway for enhancing faithfulness and reliability in summarization tasks. 2 Related Work Hallucination in Summarization. Hallucination, or the generation of content that is unfaithful to the input, is a long-standing challenge in abstractive summarization ( Maynez et al. 2020 ) . Several studies have quantified hallucination prevalence across datasets and models ( Pagnoni et al. 2021 ) . Techniques such as constrained decoding ( Cao et al. 2020 ) and fact-checking-based reranking ( Goyal et al. 2022 ) have been proposed to mitigate hallucination during or after generation. Entity-Level Hallucination Detection. Prior work has recognized the importance of entity-level evaluation for factuality. Metrics like FEQA ( Durmus et al. 2020 ) and QuestEval ( Scialom et al. 2021 ) attempt to assess factual consistency using question answering techniques. However, these methods often require reference summaries or external QA systems, limiting scalability. Our work builds on this line by employing the Entity Hallucination Index (EHI) , a lightweight evaluation metric focusing specifically on named entity faithfulness. Reward-Driven Fine-Tuning. Reinforcement learning from human feedback (RLHF) has gained traction for aligning model outputs with desired qualities, such as factuality and helpfulness ( Ouyang et al. 2022 ) . In summarization, factuality-aware reward models have been used for fine-tuning ( Pasunuru and Bansal 2018 ) . Unlike previous approaches that rely on coarse factuality rewards or human-labeled datasets, we propose fine-tuning using EHI as a direct, automatic reward signal, thereby promoting entity-grounded summarization without additional supervision. 3 Methodology Our objective is to fine-tune a language model (LM) to generate summaries with reduced entity hallucination. We achieve this by designing a reward function based on the Entity Hallucination Index (EHI) and applying reinforcement learning to maximize it. Figure 1 presents an overview of our pipeline. Input Transcript Baseline Summarization Model EHI Scoring Reward-Based Fine-Tuning Update Figure 1: Overview of our fine-tuning pipeline using EHI rewards. 3.1 Problem Formulation Given an input document x , the goal is to generate a summary y such that entities mentioned in y are grounded in x . Traditional maximum likelihood training does not explicitly optimize for factual consistency. Instead, we introduce a reward-driven fine-tuning strategy where the reward is proportional to entity faithfulness, measured using EHI. 3.2 Datasets Preparations We use the official datasets provided in the AutoMin 2025 shared task 1 1 1 https://ufal.github.io/automin-2025/ , which includes the ELITR Minuting Corpus for summarization and the ELITR-Bench dataset for question answering 2 2 2 https://github.com/utter-project/ELITR-Bench/ . We followed the task protocols as outlined in the shared task instructions, including the official train/dev/test splits. For summarization (Task A), gold summaries are provided for reference-based evaluation. For question answering (Task B), answer spans are matched against gold annotations for exact match and F1 scoring. We evaluate our approach on given meeting transcript datasets, which consist of multi-turn conversational dialogues along with abstractive gold summaries. Each sample is formatted as a flattened transcript-summary pair. To simulate real-world entity usage, datasets include a diverse range of named entities spanning persons, organizations, locations, and events. We randomly partition each dataset into training (80%), validation (10%), and test (10%) splits. Entity extraction is performed on both source documents and summaries for computing the Entity Hallucination Index (EHI). During fine-tuning, we exclusively utilize the training split, while validation EHI scores are monitored to select the best checkpoint. Final evaluations are reported on the held-out test set. Figure 2: TASK B: Question-Answer Generation 3.3 Baseline Summarization We begin with a pre-trained encoder-decoder LM (e.g., Flan-T5-Large ) fine-tuned on generic summarization tasks. For each input document, the model generates an initial summary using standard greedy or beam search decoding strategies. 3.4 Entity Hallucination Index Computation To assess entity faithfulness, We formally define the Entity Hallucination Index (EHI) as: \text{EHI}=\frac{e^{\text{PH}}+e^{\text{EF}}}{e^{\text{PH}}+e^{\text{EF}}+e^{\text{NH}}+e^{\text{OF}}+e^{\text{LF}}} (1) where x_{i}\in\{\text{PH},\text{EF},\text{NH},\text{OF},\text{LF}\} represent scores corresponding to Positive Hallucination (PH), Extractiveness Factor (EF), Negative Hallucination (NH), Overfocused Relations (OF), and Lost Focus (LF), respectively. We define the following factors: • Positive Hallucination (PH): Measures newly introduced entities that are factually correct and beneficial. • Extractiveness Factor (EF): Measures entities accurately extracted from the input document into the summary. • Negative Hallucination (NH): Captures hallucinated entities that are incorrect or not grounded in the input. • Overfocused Relations (OF): Penalizes summaries

Read the complete text