AugAbEx : Way Forward for Extractive Case Summarization

Purnima Bindal, Vikas Kumar, Sagar Rathore, Vasudha Bhatnagar · 2025

Read AugAbEx : Way Forward for Extractive Case Summarization by Purnima Bindal, Vikas Kumar, Sagar Rathore, Vasudha Bhatnagar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-12290.txt.

Introduction

Purnima Bindal, Vikas Kumar and Vasudha Bhatnagar propose AugAbEx, a pipeline that strengthens legal case summarization by adding a second kind of summary to an existing dataset. Summarizing judgments is hard for lawyers because the language is complex, jargon depends on context, and documents are long. Abstractive systems also carry known risks in this domain.

AugAbEx takes a dataset with human-written gold summaries and augments it with extractive silver summaries that select key passages and reflect expert opinion. The authors check the added summaries for structure, wording and meaning within a legal framework. Tests across seven English legal datasets support the value of having both summary types available.

Catalog record

CREATORPurnima Bindal, Vikas Kumar, Sagar Rathore, Vasudha Bhatnagar
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-12290

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AugAbEx : Way Forward for Extractive Case Summarization Purnima Bindal, Vikas Kumar, Sagar Rathore, Vasudha Bhatnagar Abstract Summarization of legal judgments poses a heavy cognitive burden on law practitioners due to the complexity of the language, context-sensitive legal jargon, and the length of the document. Therefore, the automatic summarization of legal documents has attracted serious attention from natural language processing researchers. Since the abstractive summaries of legal documents generated by deep neural methods remain prone to the risk of misrepresenting nuanced legal jargon or overlooking key contextual details, we envisage a rising trend toward the use of extractive case summarizers. Given the high cost of human annotation for gold standard extractive summaries, we engineer a light and transparent pipeline that leverages existing abstractive gold standard summaries to create the corresponding extractive gold standard versions. The approach ensures that the experts‘ opinions ensconced in the original gold standard abstractive summaries are carried over to the transformed extractive summaries. We aim to augment seven existing case summarization datasets, which include abstractive summaries, by incorporating corresponding extractive summaries and create an enriched data resource for case summarization research community. To ensure the quality of the augmented extractive summaries, we perform an extensive comparative evaluation with the original abstractive gold standard summaries covering structural, lexical, and semantic dimensions. We also compare the domain-level information of the two summaries. We commit to release the augmented datasets in the public domain for use by the research community and believe that the resource will offer opportunities to advance the field of automatic summarization of legal documents. keywords Case Summarization, Abstractive Summary, Extractive Summary, Legal Entities, LegalBert, Data Augmentation 1 Introduction The importance and utility of case summaries for case preparation, judgment writing, or simply explaining the case to a layperson, have garnered huge research interest in legal case summarization research. Though recent advancements in generic automatic text summarization have favoured abstractive techniques for their ability to produce human-like summaries, but summarizing legal documents presents its own set of challenges, including the need for precision and preservation of legal terminology. Recently, 8 compare abstractive case summaries generated by eight LLMs from general and legal domains. The authors established that though some abstractive case summarization models score slightly higher on traditional evaluation metrics 1 1 1 ROUGE, BLEU, METEOR metrics are computed in the study for Indian court case judgments. , the inconsistencies and hallucinations in the AI-generated case summaries make them practically unusable without manual examination by a domain expert. Large carbon footprint, lack of understanding of the legal domain, and related issues are other arguments that discourage the use of current SOTA generative AI models for abstractive case summarization. Several studies indicate that extractive case summaries score better compared to their abstractive counterparts in terms of automatic evaluation metrics as they better preserve the accuracy and language of the original case documents ( 2 ; 20 ; 22 ) . Since extractive summaries comprise sentences directly picked from the judgment, there is lesser risk of misinterpreting the meaning of key legal phrases, statutes, or case laws, compared to abstractive summaries generated by transformer based models. An investigation of lawyers’ perception of legal documents reveals a clear preference for documents that they perceive as crafted by a human, over those believed to be generated by AI ( 11 ) . The law experts can pad the semantic gaps due to lack of coherence and cohesion in extractive summaries, as long as the sentences containing the core legal entities are present in the summary. Our collaborators from the legal domain opine that legal entities like statutes, provisions, and precedents in a case summary are most informative for the law expert for a quick revelation of the applicable areas of law. Hence, an extractive summary rich in legal entities is more valuable for a law expert than the corresponding fluent AI-generated summary, which may be counter-factual. Recently released CivilSum dataset ( 16 ) , consisting of clauses 2 2 2 See appendix 10 for a sample gold standard reference summary from the CivilSum dataset with legal entities in the judgment as gold standard summaries, vindicates our argument. We envisage that the law practitioners disfavor of abstractive summaries is the prime push behind the trend for developing extractive case summarization methods, and it will drive more efforts toward supervised and unsupervised extractive case summarization methods. Creating high-quality extractive summary datasets, ingrained with experts’ opinions of saliency, is imperative for progress in the field of extractive case summarization. Consistent and precise legal language in the gold standard summaries is essential for training high-fidelity case summarizers. However, crafting human-written summaries to train extractive case summarizers is a humongous task due to the sheer volume of cases, time constraints and cognitive burden on law professionals. Automatic evaluation of the quality of extractive algorithmic summaries is another challenge to be addressed. Most existing case summarization systems rely on lexical overlap (ROUGE metric) for quantitative evaluation of the summary quality ( 9 ; 10 ; 19 ; 15 ; 25 ; 3 ; 8 ) . In this scenario, existing case-summarization datasets with abstractive summaries fall short of the expectation. We argue that evaluating the case summaries along structural, lexical and semantic dimensions is highly desirable. Most importantly, the algorithmic summaries must be evaluated for the core legal content against the gold standard references. Law professionals, the ultimate consumers of the case summaries, rely on the technical legal terms, provisions, and statutes, precedents, etc. to draw inferences during legal research. Recently, 7 considered precision metric based on named and number entities present in case document to assess the quantum of legal information. 1.1 Motivation and Contributions The pace of development of extractive case summarizers must be paired with the growth of benchmark datasets with extractive summaries, for a headway and positive impact in the area of legal judgment summarization. A fair evaluation of the system-generated extractive summaries demands comparison with extractive gold standard summaries. However, manual curation of gold standard extractive summaries is infeasible due to the heavy cognitive burden and the consequent prohibitive cost of legal expertise worldwide. This is the prime motivating factor for the current endeavor. We aim to leverage the human-written abstractive gold standard summaries in the existing case summarization datasets and transform them into their corresponding extractive counterparts. The objective is achieved by engineering a transparent, cost-effective pipeline inspired by several existing works. The additional advantage our strategy begets is the strong alignment of the transformed extractive summaries with the expert’s opinion about the important aspects of the case. We ensure the quality of newly created summaries by performing a thorough comparative analysis of original abstractive summaries written by law experts and the transformed extractive summaries. We compare the structural, lexical and semantic features, and scrutinize the prevalence of legal entities in the two summaries. We reinforce our investigation by performing a pairwise comparison of the original abstractive gold reference summaries (OAG) and the transformed summaries using the Bradley-Terry model ( 18 ) for the four aspects. We refer to the extractive versions as Transformed Extractive Gold standard (TEG), as they can be used as gold standard summaries for evaluating extractive summarizers. Achieving our overarching objective to enhance abstractive case summarization

Read the complete text