Segment First, Retrieve Better: Realistic Legal Search via Rhetorical Role-Based Queries
Shubham Kumar Nigam, Tanmay Dubey, Noel Shallum, Arnab Bhattacharya · 2025
Read Segment First, Retrieve Better: Realistic Legal Search via Rhetorical Role-Based Queries by Shubham Kumar Nigam, Tanmay Dubey, Noel Shallum, Arnab Bhattacharya (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00679.txt.
Introduction
Shubham Nigam and three co-authors built a legal search tool that works the way lawyers do, from parts of a case, in a paper published in 2025. Past rulings guide new ones, but full texts are long and often only fragments are known at search time. Their tool, TraceRetriever, cuts rulings into role parts such as facts and grounds. Only the weighty parts form the query.
Three search kinds run in one pipeline: word match, vector match, and a fine re-ranking step. A rank fusion blends their lists before the final sort. The role labels come from a network trained on Indian rulings. Tests on two legal datasets show the pipeline holds up as case loads grow. The authors present it as a firm base for real-world case search.
Catalog record
| CREATOR | Shubham Kumar Nigam, Tanmay Dubey, Noel Shallum, Arnab Bhattacharya |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-00679
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Segment First, Retrieve Better: Realistic Legal Search via Rhetorical Role-Based Queries Shubham Kumar Nigam, Tanmay Dubey, Noel Shallum, Arnab Bhattacharya Abstract Legal precedent retrieval is a cornerstone of the common law system, governed by the principle of stare decisis, which demands consistency in judicial decisions. However, the growing complexity and volume of legal documents challenge traditional retrieval methods. TraceRetriever mirrors real-world legal search by operating with limited case information, extracting only rhetorically significant segments instead of requiring complete documents. Our pipeline integrates BM25, Vector Database, and Cross-Encoder models, combining initial results through Reciprocal Rank Fusion before final re-ranking. Rhetorical annotations are generated using a Hierarchical BiLSTM CRF classifier trained on Indian judgments. Evaluated on IL-PCR and COLIEE 2025 datasets, TraceRetriever addresses growing document volume challenges while aligning with practical search constraints, reliable and scalable foundation for precedent retrieval enhancing legal research when only partial case knowledge is available. 1 Introduction The common law system’s foundation rests upon the principle of stare decisis , mandating judicial adherence to precedents established in prior rulings when addressing analogous issues and facts within the same jurisdiction. As legal documentation grows in complexity and volume, sophisticated Natural Language Processing (NLP) techniques become indispensable for understanding, analyzing, and retrieving relevant precedents. TraceRetriever plays a crucial role in upholding stare decisis , facilitating the identification of past judgments with similar legal contexts to ensure consistent application of the law. The sheer volume of legal resources, including judgments, statutes, and regulations, poses a significant challenge for legal professionals seeking pertinent precedents, underscoring the urgent need for effective retrieval mechanisms. Figure 1: Illustration of rhetorical role segmentation in a legal document. The left side shows the original excerpt, while the right side displays the labeled segments. In our approach, only relevant segments such as Facts and Issue are retained to emulate real-world legal case retrieval scenarios, where complete information like Reasoning or Decision may not be available at query time ( Nigam et al. 2025 ) . A notable limitation in much of the existing work on automated precedent retrieval is its reliance on using entire prior case documents as queries. This approach deviates significantly from real-world legal practice, where lawyers typically formulate search queries based on specific factual details and legal issues extracted from the case at hand, often with limited initial information. To address this gap, this paper tackles the challenge of mimicking real-world legal search scenarios in TraceRetriever by proposing a novel heuristic approach. Our methodology strategically integrates the complementary strengths of a keyword-based model (BM25), a semantic Vector Database, and a fine-grained Cross-Encoder for re-ranking. A key innovation of our work lies in utilizing a trained Hierarchical Bidirectional LSTM (HierBiLSTM) model by ( Bhattacharya et al. 2019 ) to classify sentences within legal documents into distinct rhetorical roles. We then leverage the role segments, identified through this classification, as the query for our retrieval pipeline. This deliberate use of limited, rhetorically-informed query components directly mirrors the information scarcity often encountered in practical legal research. The core problem this paper addresses is therefore the development of a TraceRetriever system that effectively operates with limited, contextually relevant information, thereby more accurately reflecting real-world legal search processes. To evaluate the effectiveness of our proposed TraceRetriever pipeline, we conducted experiments on two established legal datasets: the Indian Legal Text Understanding and Reasoning (IL-PCR) dataset ( Joshi et al. 2023 ) and the Competition on Legal Information Extraction and Entailment (COLIEE) 2025 dataset. Our pipeline employs a heuristic approach that strategically integrates the strengths of three distinct retrieval models: a semantic Vector Database, the BM25 algorithm, and a more nuanced Cross-Encoder. To further refine the initial retrieval results from the Vector Database and BM25, we implemented Reciprocal Rank Fusion (RRF), a robust re-ranking technique. In our TraceRetriever pipeline, we established BM25 as a robust baseline, representing a traditional keyword-based approach to information retrieval. To enhance the relevance and accuracy of our results, we implemented a sophisticated re-ranking strategy that leverages both semantic understanding and fine-grained interaction. Specifically, we employed Cross-Encoders to re-rank the top-k documents initially retrieved by two distinct methods: the lexical matching of BM25 and the semantic similarity captured by our Vector Database (a bi-encoder-based approach). This multi-faceted strategy effectively integrates the strengths of three complementary retrieval paradigms: Our key contributions are: 1. A realistic legal retrieval strategy using rhetorical role-based queries reflecting limited-information scenarios. 2. Development of TraceRetriever: A hybrid pipeline integrating BM25, vector search, and cross-encoder re-ranking. For the sake of reproducibility, we have made our dataset, code, and RAG-based pipeline implementation via an github repository 1 1 1 https://github.com/ShubhamKumarNigam/Legal_IR . 2 Related Work Legal case retrieval has witnessed a rapid transformation with the advent of LLMs, RAG pipelines, and rhetorical role labeling. Traditionally, legal information retrieval relied heavily on lexical matching (e.g., BM25), which struggled to handle the semantic and structural nuances of legal texts. Recent innovations focus on improving retrieval accuracy by leveraging domain-specific embeddings, legal document structures, and rhetorical role understanding. Several systems have explored enhancing legal QA and retrieval using hybrid architectures. ( Wiratunga et al. 2024 ) integrates Case-Based Reasoning with RAG to improve contextual relevance and factual correctness in legal question-answering. Similarly, ( Panchal et al. 2025 ) utilizes FAISS and DeepSeek embeddings to make Indian legal knowledge accessible through a chatbot interface. Another significant trend is the use of rhetorical roles in structuring legal texts. ( Bhattacharya et al. 2019 ; Malik et al. 2022 ) pioneered rhetorical role classification in Indian legal judgments, showing that deep neural architectures such as BiLSTM-CRF and multi-task learning can outperform traditional methods. ( Marino et al. 2023 ) further advanced this by stacking transformers over LEGAL-BERT to capture inter-sentence dependencies for rhetorical role classification across multilingual legal datasets. These works collectively demonstrate the feasibility and utility of segmenting legal documents into roles such as Facts , Issues , and Reasoning categories that are highly valuable for information extraction and retrieval. In recent studies, ( Bhattacharya et al. 2019 ) proposed a CRF-BiLSTM model specifically for as signing rhetorical roles to sentences in Indian legal documents. In the context of document-to-document legal retrieval, methods like ( Althammer et al. 2022 ) , ( Ma et al. 2023 ) , and ( Li et al. 2023 ) aim to overcome the challenges of long input lengths and weak semantic relevance by employing paragraph aggregation, structure-aware pretraining, and custom contrastive loss functions. Meanwhile, ( Tang et al. 2023 ) and ( Tang et al. 2024 ) take a graph based approach, modeling the connectivity between cases via attributed case graphs or global semantic networks to achieve state-of-the-art performance. ( Nigam et al. 2022 ) presents a cascaded retrieval framework that integrates BM25 for lexical matching with Sentence BERT and Sent2Vec for semantic understanding. Interestingly, results show that BM25 alone often outperforms neural models, reaffirming the robustness and relevance of lexical approaches in legal case retrieval. Beyond traditional lexical and semantic methods, several recent studies have explored innovative architectures to enhance legal case retrieval by addressing challenges such as long document length, complex legal semantics, and noisy or sparse queries. ( Hu et al. 2022 ) proposes a retrieval method grounded in legal facts by combining