Debate-Feedback: A Multi-Agent Framework for Efficient Legal Judgment Prediction
Xi Chen, Mao Mao, Shuo Li, Haotian Shangguan · 2025
Read Debate-Feedback: A Multi-Agent Framework for Efficient Legal Judgment Prediction by Xi Chen, Mao Mao, Shuo Li, Haotian Shangguan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05358.txt.
Introduction
Chen, Mao, Li, and Shangguan built this legal prediction system in twenty twenty-five. AI tools that predict court judgments often lean on huge archives of past cases. They also underuse the reasoning power of modern language models. The authors borrow the debate stage of a real trial to structure the task.
Several model agents argue the case from opposing sides. A separate checker scores how reliable each round of debate is. Feedback from the checker steers later rounds. The setup needs far fewer past cases than older methods. In comparative tests it beat both general and legal-specific models. The authors present debate plus feedback as a light, dynamic route for legal AI.
Catalog record
| CREATOR | Xi Chen, Mao Mao, Shuo Li, Haotian Shangguan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-05358
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Debate-Feedback: A Multi-Agent Framework for Efficient Legal Judgment Prediction Xi Chen, Mao Mao, Shuo Li, Haotian Shangguan Abstract The use of AI in legal analysis and prediction (LegalAI) has gained widespread attention, with past research focusing on retrieval-based methods and fine-tuning large models. However, these approaches often require large datasets and underutilize the capabilities of modern large language models (LLMs). In this paper, inspired by the debate phase of real courtroom trials, we propose a novel legal judgment prediction model based on the Debate-Feedback architecture, which integrates LLM multi-agent debate and reliability evaluation models. Unlike traditional methods, our model achieves significant improvements in efficiency by minimizing the need for large historical datasets, thus offering a lightweight yet robust solution. Comparative experiments show that it outperforms several general-purpose and domain-specific legal models, offering a dynamic reasoning process and a promising direction for future LegalAI research. 1 Introduction Figure 1: A brief introduction of Debate-Feedback Structure LegalAI leverages artificial intelligence technologies such as natural language processing, machine learning, and deep learning to address various legal tasks ( Aletras et al., 2016 ; article ; zhong2020how ) , including legal document analysis and consultation. A key area of LegalAI is Legal Judgment Prediction (LJP) ( zhong-etal-2018-legal ; ma2021precedent ; 10255647 ) , which focuses on predicting court judgments. LJP tasks typically use historical legal case data, including background information, case descriptions, statements from both parties, precedents, and court verdicts. Predictions range from binary outcomes (e.g., plaintiff vs. defendant wins) to multi-class tasks (e.g., sentence prediction). NLP technologies, combined with advanced models like LegalBERT chalkidis2019legal and Lawformer xiao2021lawformerpretrainedlanguagemodel , have achieved strong results by learning from large datasets. The debate model is a system that integrates large language modeling (LLM) with argumentative reasoning techniques to simulate the process of debate or contention ( irving2018ai ; nie2020adversarialnlinewbenchmark ) , ultimately arriving at a decision or conclusion on a specific issue through the debate process. In a typical debate task, multiple LLM agents assume different roles and are deliberately guided to provide answers from various perspectives or positions. These generated arguments are then synthesized to assist the LLM in reaching a final conclusion zeng2022socraticmodelscomposingzeroshot . In this paper, we propose a Debate-Feedback model to explore an efficient and convenient method for predicting legal judgement. Fig[ 1 ] shows the general framework of the model in the task of predicting decision results. Specifically, Debate-Feedback can be divided into four steps. First, the collected historical legal cases L_{i} will be formatted into Case Background C_{i} , Plaintiff Claim P_{i} and Defendant Statement D_{i} . These information will be provided to the judge LLM for initial prediction. In the second step of the debate, multiple LLM agents will be guided to answer the prediction questions from different perspectives, and then exchange opinions and debate to generate their own comments E_{i} . In the verification phase, a pre-trained assistant model \mathcal{E} will conduct a reliability analysis on each LLM’s comments combined with case information. The results of the analysis will be provided to the judge LLM for reference together with each agent’s comments. The judge LLM will give the prediction O_{i} for this round based on the above information \mathcal{E}=E_{i}\oplus L_{i} . More details are illustrated in the Methodology section. In summary, we introduce a Debate-Feedback model that enhances legal judgment prediction by incorporating a multi-agent debate process and reliability evaluation, providing a more efficient and accurate solution with reduced reliance on large datasets. 2 Related Work Legal documents are characterized by lengthy texts and complex logic, which has led prior research to focus on two key approaches to address these challenges: training legal LLM and using retrieval augmentation. 2.1 Legal LLM In-context Learning(ICL) is a learning paradigm widely applied in large language models (LLMs) by using a set of context examples to guide predictions during reasoning ( dong2024surveyincontextlearning ; liu2021pretrainpromptpredictsystematic ; gutierrezpachas2022distributionmarkovchainreversetime ; min2022metaicllearninglearncontext ) . However, due to the often extensive length of legal texts, naive ICL methods are constrained by LLM input length limits. As a result, LegalAI solutions typically combine ICL with fine-tuning or pre-training of models to overcome these limitations. For instance, LegalBERT chalkidis2019legal fine-tunes BERT on legal datasets, achieving strong results in legal text classification and provision retrieval. Similarly, Lawformer xiao2021lawformerpretrainedlanguagemodel handles lengthy Chinese legal documents, while CaseLaw-BERT paul2023pretrainedlanguagemodelslegal , fine-tuned on case law datasets, enhances legal case retrieval and judgment prediction. Despite their success, these approaches rely heavily on large, domain-specific datasets, which can limit their applicability across different legal systems and languages. 2.2 Retrieval Augmentation Retrieving relevant legal precedents—court judgments or legal decisions from previous cases—is a mainstream approach to assist LLMs in making predictions, especially in overcoming the challenge of lengthy texts. By providing recommended samples, this method guides the LLM’s reasoning process more effectively ( zhong2020how ; 10.1145/3462757.3466066 ) . Ma et al. introduced a framework that deeply integrates legal precedents into judgment prediction wu-etal-2023-precedent , combining the reasoning capabilities of LLMs with domain-specific models to enable more accurate and context-aware predictions. Similarly, Caseformer su2024caseformerpretraininglegalcase employs a pre-training strategy that emphasizes distinctions between cases, enhancing case retrieval performance. Although retrieval augmentation improves the handling of long texts, it still relies on the availability of large datasets, and its reliance on specific legal systems and languages can limit broader applicability across different jurisdictions. 3 Methodology In this section, we first systematically introduce our Feedback-Debate model, followed by an analysis of the limitations of the general debate architecture in specific legal scenarios, along with proposed solutions to address these shortcomings. Overview Algorithm[ 1 ] presents the pseudo code for the debate-feedback framework in binary classification. The input is a preprocessed legal event text, labeled as S , and the main language model (LM) plays the role of the judge, predicting the probability of a legal judgment, LM:S\rightarrow[0,1] . Two agents, t_{ne} and t_{po} , debate from opposing perspectives, providing inputs to refine the judgment. Each debate round involves these agents exchanging and debating their positions, with n defining the number of iterations. The assistant model \mathcal{E} evaluates the reliability of the agents’ arguments and outputs a probability. If the reliability exceeds a threshold, the main LM adjusts its prediction by weighting the latest information, otherwise it defaults to the initial prediction. The final decision is smoothed over all rounds to produce a stable outcome. (Note that notation \oplus does not mean xor, but rather combination in a non-additive sense.) Algorithm 1 Debate-Feedback Input: LM, \mathcal{E} : \mathcal{S}\to[0,1] ; n , T \in N ; x\in S ; t_{ne} , t_{po} : \mathcal{S}\to\mathcal{S} ; Output: Final decision y\in(0,1) ; O_{0}\leftarrow LM(x) ; for i\leftarrow 1 to n do // Debate Step a:a_{ne},a_{po}\leftarrow t_{ne}(x),t_{po}(x) ; e:e_{ne},e_{po}\leftarrow t_{ne}(x\oplus a_{po}),t_{po}(x\oplus a_{ne}) ; // Verification Step v:v_{ne},v_{po}\leftarrow\mathcal{E}(e_{ne}),\mathcal{E}(e_{po}) ; sum=LM(a,e,v) ; if Threshold(v) then O_{i}=(1-T)*O_{i-1}+T*LM(x,sum) ; end if else O_{i}=LM(x) ; end if end for y\leftarrow O_{n} ; Reliability Analysis Through experiments, we observe that a simple debate model can sometimes lead to worse prediction results. This occurs because legal predictions differ from mathematical problems, as they often involve subjective tendencies. A straightforward example is when we guide multiple LLMs to debate from the perspectives of the plaintiff and defendant, it is challenging for them to reach a