LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning
Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han, Yike Guo · 2025
Read LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning by Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han, Yike Guo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07443.txt.
Introduction
Predicting court judgments with language models fails when the reasoning wanders, and Weijie Shi and co-authors build a system that checks its own steps. LegalReasoner first finds a case's dispute points, splitting a hard case into smaller questions. It then reasons step by step while a separate verifier grades each step for correctness, progress, and promise.
When the verifier spots an error, expert-designed strategies trace the fault and repair that step. To train the system, the team releases LegalHK, a dataset of 58,130 Hong Kong court cases annotated with dispute points, reasoning chains, and verification labels. On a large open model, agreement with real court decisions rose from about seventy-two to eighty percent.
Catalog record
| CREATOR | Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han, Yike Guo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-07443
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han, Yike Guo Abstract Legal judgment prediction (LJP) aims to function as a judge by making final rulings based on case claims and facts, which plays a vital role in the judicial domain for supporting court decision-making and improving judicial efficiency. However, existing methods often struggle with logical errors when conducting complex legal reasoning. We propose LegalReasoner, which enhances LJP reliability through step-wise verification and correction of the reasoning process. Specifically, it first identifies dispute points to decompose complex cases, and then conducts step-wise reasoning while employing a process verifier to validate each step’s logic from correctness, progressiveness, and potential perspectives. When errors are detected, expert-designed attribution and resolution strategies are applied for correction. To fine-tune LegalReasoner, we release the LegalHK dataset, containing 58,130 Hong Kong court cases with detailed annotations of dispute points, step-by-step reasoning chains, and process verification labels. Experiments demonstrate that LegalReasoner significantly improves concordance with court decisions from 72.37 to 80.27 on LLAMA-3.1-70B. The data is available at https://huggingface.co/datasets/weijiezz/LegalHK. 1 Introduction Legal judgment prediction (LJP) aims to function as a judge by predicting rulings based on facts of legal cases. It serves crucial roles in supporting legal decision-making processes, improving judicial efficiency and justice Cui et al. 2023 , and enabling citizens to grasp potential case outcomes without expensive legal consultation Feng et al. 2022a . However, when confronted with complex cases, LJP requires a highly specialized ability to capture key points of cases and conduct a thorough analysis to reach reliable and well-founded conclusions. Unfortunately, existing LJP methods overlook the importance of logical reasoning in judicial decision-making. While BERT-based models like LegalBERT Chalkidis et al. 2019 ; Chalkidis et al. 2020 and jurBERT Masala et al. 2024 have demonstrated promising results, they inherently lack logical reasoning capabilities. Although large language models (LLMs) have shown significant improvements in executing complex multi-step reasoning through carefully designed prompts Trautmann et al. 2022 , retrieval-augmented generation Santosh et al. 2024 , and fine-tuning Ammar et al. 2024 , even state-of-the-art LLMs still frequently exhibit logical errors. To remedy the reasoning vulnerabilities of LLMs in the LJP task, particularly when dealing with high-complexity cases, there is a critical need for both guided reasoning and robust verification mechanisms that can detect and rectify logical errors throughout the inference process. To enable effective legal reasoning, there are a series of challenges to maintain logical consistency and generate well-founded judgments. First, legal cases often involve implicit causal relations and contentions, making them more complex. Second, the lack of effective process supervisors in judicial reasoning makes it challenging to monitor and validate the logical flow of legal analysis, potentially leading to errors and inconsistent conclusions. Third, addressing reasoning errors requires problem attribution and solutions, with each type of error demanding a tailored remediation approach. For instance, when legal provisions are misunderstood, we need to review the relevant laws for clarity. Similarly, when reasoning doesn’t match case facts, we must check that the logic aligns with the evidence. Figure 1: An illustration of legal judgment prediction. LegalReasoner first breaks down the process into identifying disputes, and then subsequent reasoning steps are monitored by a verifier. If errors occur, they are traced back to predefined issues and corrected by tailored strategies, ensuring reliable judicial reasoning. In this paper, we propose LegalReasoner, which first identifies the disputes to decompose complex cases, and then step-wise verifies and corrects potential errors in the legal reasoning process as depicted in Figure 1 . To facilitate legal case decomposition and reasoning, we construct LegalHK, a comprehensive dataset containing 58,130 Hong Kong court cases, which are annotated with dispute points, evidence analysis, relevant statutory provisions, precedent citations, and step-by-step reasoning chains leading to final judgments. Upon this foundation, a process verifier is proposed to validate each legal reasoning step. When reasoning errors are detected, our correct component first attributes to specific error types designed by legal experts, such as legal principle misapplication, discrepancy between facts and reasoning, etc. Next, it activates targeted problem-solving mechanisms to address these issues through tailored strategies, like provision retrieval, fact tracking, enforcement pathway analysis, etc. Our key contributions can be summarized as follows: • To ensure the reliability of judicial reasoning, we propose a process verifier to detect errors at each step, complemented by expert-designed attribution and resolution strategies to correct errors. • To enhance case decomposition and structured reasoning in LLMs, we open-source the LegalHK dataset with detailed annotations. • To the best of our knowledge, we are the first to propose verification and correction mechanisms for the legal reasoning process, improving concordance with court decisions from 72.37 to 80.27 on LLAMA-3.1-70B. 2 Related Work 2.1 Legal Judgment Prediction Existing Legal judgment prediction (LJP) methods primarily fall into two categories: BERT-based and LLM-based approaches. BERT-based methods treat the LJP task as a multi-class classification to identify appropriate charges Huang et al. 2024 ; de Arriba-Pérez et al. 2022 . To enhance legal knowledge, many researchers Chalkidis et al. 2019 ; Chalkidis et al. 2020 focused on fine-tuning by legal corpora. Feng et al. 2022b and Deroy et al. 2024 supplied judgment information by constrained event extraction and document summarization, respectively. Masala et al. 2024 introduced Longformer and SLiding Encoder-Decoder Ivgi et al. 2023 to handle long legal documents, while Liu et al. 2022 incorporated case-based contrastive embedding enhancement. Nevertheless, these methods focus on picking up the token-level clues instead of thinking based on facts. Recently, with the rise of LLMs, carefully designed legal prompts have been employed for judicial decision-making Trautmann et al. 2022 . ChatLaw Cui et al. 2024 utilized multi-agent collaboration, while Santosh et al. 2024 incorporated precedents as judgment references to enhance reliability. These methods have been adapted for various regions and languages, including China Sun et al. 2024 , Europe Santosh et al. 2024 , Korea Hwang et al. 2022 , etc Millan-Arias et al. 2023 ; Masala et al. 2024 . Despite LLMs placing greater emphasis on reasoning, frequent logical errors lead to incorrect final predictions. 2.2 LLMs Reasoning and Verification Although LLMs have demonstrated excellent potential in handling complex tasks, the accumulation of reasoning errors remains a serious challenge. Lightman et al. 2023 showed that models trained using process rewards outperform those supervised by outcome rewards. To automatically annotate training datasets for process rewards, Math-Shepherd Wang et al. 2024 and ReST-MCTS Zhang et al. 2024 defined it based on their potential to deduce the correct final answer. To address reasoning errors, researchers have explored LLM self-correction based on output feedback. SELF-REFINE Madaan et al. 2024 introduced an iterative self-improvement strategy, while REFINER Paul et al. 2023 and Re^{3} Yang et al. 2022 employed dedicated critic models to provide step-by-step feedback. For code generation, CRITIC Gou et al. 2023 leveraged external verification tools and fed their critiques back to LLMs. However, these process supervision approaches have mainly been applied to mathematical and coding tasks, and cannot adapt to the LJP task. 3 Task Definition Given a legal case c consisting of a plaintiff’s claim c_{claim} and factual descriptions c_{fact} , we aim to perform judicial reasoning \{s_{i}\}_{i=1}^{n}