LegalSearchLM : Rethinking Legal Case Retrieval as Legal Elements Generation

Chaeeun Kim, Jinu Lee, Wonseok Hwang · 2025

Read LegalSearchLM : Rethinking Legal Case Retrieval as Legal Elements Generation by Chaeeun Kim, Jinu Lee, Wonseok Hwang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-23832.txt.

Introduction

Chaeeun Kim and co-authors rethink legal case retrieval, the search for past cases that match a new one. Older systems match word patterns or text embeddings, and they were tested on small case sets with narrow crime types. The team argues those tests poorly reflect real legal search in Korea.

They build a large Korean benchmark covering hundreds of crime types and over a million candidate cases. Their model, LegalSearchLM, first reasons out the legal elements of the query case. It then generates text built from those elements, grounded in real cases, to guide the match. Retrieval becomes a reasoning task, not just a word match.

Catalog record

CREATORChaeeun Kim, Jinu Lee, Wonseok Hwang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE129 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-23832

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LegalSearchLM : Rethinking Legal Case Retrieval as Legal Elements Generation Chaeeun Kim, Jinu Lee, Wonseok Hwang Abstract Legal Case Retrieval (LCR), which retrieves relevant cases from a query case, is a fundamental task for legal professionals in research and decision-making. However, existing studies on LCR face two major limitations. First, they are evaluated on relatively small-scale retrieval corpora (e.g., 100-55K cases) and use a narrow range of criminal query types, which cannot sufficiently reflect the complexity of real-world legal retrieval scenarios. Second, their reliance on embedding-based or lexical matching methods often results in limited representations and legally irrelevant matches. To address these issues, we present: (1) LEGAR BENCH , the first large-scale Korean LCR benchmark, covering 411 diverse crime types in queries over 1.2M candidate cases; and (2) LegalSearchLM , a retrieval model that performs legal element reasoning over the query case and directly generates content containing those elements, grounded in the target cases through constrained decoding. Experimental results show that LegalSearchLM outperforms baselines by 6 – 20% on LEGAR BENCH , achieving state-of-the-art performance. It also demonstrates strong generalization to out-of-domain cases, outperforming naive generative models trained on in-domain data by 15%. Language Crime types of query Query case Retrieval pool Target case per query COLIEE2024 English - 400 1,734 (per query) - LeCaRD Chinese - 107 100 (per query) 10.33 LeCaRDv2 Chinese - 800 55,192 (per query) 20.89 LEGAR BENCH {}_{\textit{Standard}} Korean 411 411*N 1,226,814 200 LEGAR BENCH {}_{\textit{Stricter}} Korean 160 ~15,777 169,230 14.69 Table 1: Scale comparison of LCR benchmarks. LEGAR BENCH expands the retrieval pool and query coverage. N indicates query set expandability via similar case groups. 1 Introduction Legal AI has increasingly gained attention from legal professionals to raise the productivity of their work. Among various legal applications, Legal Case Retrieval (LCR) Feng et al. (2024) ; Deng et al. (2024b) ; Su et al. (2024) ; Deng et al. (2024a) ; Li et al. (2023a) ; Xiao et al. (2021) ; Li et al. (2023b) ; Li et al. (2023c) ; Zhang et al. (2023) , which identifies relevant precedents for a given case, plays a particularly crucial role in maintaining judicial fairness and supporting the decision-making process of legal experts. However, previous work on the legal case retrieval task has clear limitations. (1) Most evaluations have been conducted in small-scale settings, where a predefined set of candidate documents is provided for each query and the queries do not sufficiently reflect the diverse criminal case types Ma et al. (2021) ; Li et al. (2023b) ; Li et al. (2023c) ; Xiao et al. (2018) ; Li et al. (2023d) . (2) The tested retrieval models are limited to embedding-based similarity and lexical matching approaches, with the former struggling to capture the rich semantics of legal literature as it compresses complex documents into fixed-size vectors, and the latter often leading to unfocused matches due to a lack of semantic understanding Wang et al. (2023) ; Magesh et al. (2024) ; Kim et al. (2024) . To address the first, we present LEGAR BENCH ( Lega l Case R etrieval Bench mark), the first large-scale Korean LCR benchmark, comprising two dataset versions tailored to different evaluation needs: (1) LEGAR BENCH {}_{\textit{Standard}} is designed for comprehensive assessment across a broad range of crime categories. The queries cover 411 distinct crime types and are evaluated over a retrieval pool of 1.2M cases. To achieve this, we systematically construct the benchmark using crime types based on statutory provisions, even further detailed than official charge titles used in courts (e.g., defamation by fact disclosure – Article 307(1), or false allegation – Article 307(2), both sharing the charge title defamation ) (Section 2.1 ). (2) LEGAR BENCH {}_{\textit{Stricter}} evaluates stricter relevance criteria than LEGAR BENCH {}_{\textit{Standard}} . It considers more factual details and legal issues within the same crime type that can affect the final judgment or sentence, which is crucial for legal practitioners. To enable this, we annotate the pool of 170K cases across 160 crime types using 102 crime-specific legal factors and 443 corresponding options (Section 2.2 ). To address the second, we shift our focus on the LCR task from sequence-to-sequence matching to generating the key legal elements that determine relevance by proposing LegalSearchLM . It directly generates important legal elements via constrained decoding and returns the documents that contain them ( Li et al., 2024 ; Kim et al., 2024 ; Li et al., 2023e ; Li et al., 2023f ; Bevilacqua et al., 2022 ) , mitigating the shortcomings of traditional retrievers like BM25 and dense embeddings. Given the specificity of the legal domain, we adopt diverse strategies including first token-aware generation and self-supervised fine-tuning (SSFT) (Section 3 ). We evaluate LegalSearchLM on LEGAR BENCH , comparing it against strong baselines including lexical matching and embedding-based methods from both general-purpose and legal-domain models. Our experimental results show that LegalSearchLM outperforms the best baseline by 6% in precision. Our training strategy leads to significantly better generalization, achieving a 15% improvement over generative retrieval models trained with naive identifiers on in-domain data (Section 6 ). In summary, our contributions are as follows. • We introduce the first Korean LCR benchmark, LEGAR BENCH , which has the largest and most diverse criminal cases. • We present LegalSearchLM that generates legal elements that should be included in target case via first-token-aware decoding. • Our LegalSearchLM achieves state-of-the-art performance on LEGAR BENCH, demonstrating remarkable generalization ability on unseen crime types. 2 LEGAR BENCH Categories #Crime types of queries Standard Stricter Traffic offenses 13 9 Fraud 21 8 Injury or Violence 31 19 Sexual crime 132 111 Defamation or Insult 8 6 Finance or Insurance 5 1 Drug 5 4 Murder 2 2 Theft or Robbery 38 - Obstruction of Business 13 - Destruction 5 - Threat 11 - Criminal trespass 15 - Embezzlement or Breach of trust 15 - Gambling 7 - Negligent homicide and injury 6 - Obstruction of right 5 - Child abuse or School violence 10 - Medical or Food drug 11 - Corporation 3 - Bribery 3 - Car 2 - Labor or Employment 11 - Industrial or Serious accidents 4 - Military duty or law 2 - Consumer or Fair trade 1 - Arrest or Detention 1 - Intellectual property 3 - IT or Privacy 2 - Misdemeanor 1 - Sexual norms 1 - Tax, Administ, Const law 14 - Other criminal offenses 10 - Total 411 160 Table 2: Statistics of Crime Typology and Corresponding Criminal Types. LEGAR BENCH {}_{\textit{Standard}} includes 411 query types across 33 crime categories, while LEGAR BENCH {}_{\textit{Stricter}} covers 160 query types across 8 categories. In this section, we describe relevance criteria and the construction process of LEGAR BENCH . LEGAR BENCH features the most comprehensive set of query and target cases (See Table 1 ), with relevance criteria rigorously defined by the lawyers involved in the construction process. It offers two dataset versions based on different evaluation needs, LEGAR BENCH {}_{\textit{Standard}} and LEGAR BENCH {}_{\textit{Stricter}} . Figure 1: Examples of Relevance Cases. (Query) is a query case on distributing false images/videos for profit. The Blue Highlight indicates profit, the Yellow Highlight represents the creation of false images/videos, and the

Read the complete text