Designing Human-AI System for Legal Research: A Case Study of Precedent Search in Chinese Law Conference: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan Extended Abstracts of the CHI Conference on Human Factors in Computing Syste
Jiarui Guan, Ruishi Zou, Jiajun Zhang, Kimpan Xin, Bingsu He, Zhuhe Zhang, Chen Ye · 2025
Read Designing Human-AI System for Legal Research: A Case Study of Precedent Search in Chinese Law Conference: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan Extended Abstracts of the CHI Conference on Human Factors in Computing Syste by Jiarui Guan, Ruishi Zou, Jiajun Zhang, Kimpan Xin, Bingsu He, Zhuhe Zhang, Chen Ye (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-08235.txt.
Introduction
Jiarui Guan and six co-authors study how to design a smart tool that helps lawyers search past cases in Chinese law. Their short paper, from two thousand twenty-five, starts from what lawyers say they need.
Lawyers have found most smart legal tools of limited use in daily work. A key flaw is that the user can set the input and see the output, but has no grip on the steps in between. That gap raises doubt on both quality and fair use. The team held a study with five legal workers on case search. They drew out needs that can guide a tool where the user stays in charge of the search path, not just its ends.
Catalog record
| CREATOR | Jiarui Guan, Ruishi Zou, Jiajun Zhang, Kimpan Xin, Bingsu He, Zhuhe Zhang, Chen Ye |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-08235
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Designing Human-AI System for Legal Research: A Case Study of Precedent Search in Chinese Law Conference: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’25), April 26-May 1, 2025, Yokohama, Japan DOI: 10.1145/3706599.3720167 ISBN: 979-8-4007-1395-8/2025/04 CCS: Human-centered computing Human computer interaction (HCI) CCS: Applied computing Law Jiarui Guan, Ruishi Zou, Jiajun Zhang, Kimpan Xin, Bingsu He, Zhuhe Zhang, Chen Ye Abstract . Recent advancements in AI technology have seen researchers and industry professionals actively exploring the application of AI tools in legal workflows. Despite this prevailing trend, legal practitioners found that AI tools had limited effectiveness in supporting everyday tasks, which can be partly attributed to their design. Typically, AI legal tools only offer end-to-end interaction: practitioners can only manipulate the input and output but have no control over the intermediate steps, raising concerns about AI tools’ performance and ethical use. To design an effective AI legal tool, as a first step, we explore users’ needs with one specific use case: precedent search. Through a qualitative study with five legal practitioners, we uncovered the precedent search workflow, the challenges they face using current systems, and their concerns and expectations regarding AI tools. We conclude our exploration with an initial prototype to reflect the design implications derived from our findings. 1. Introduction Recent advancements in artificial intelligence (AI) have attracted growing interest from academia and industry in using AI tools to support legal practices ( Chen et al. 2024 ) 1 1 1 In this paper, we use the term “AI legal tools” to refer to a broad spectrum of AI-infused systems ( Amershi et al. 2019 ) that are either designed specifically for legal application or generic tools (e.g., chatbots) that can be repurposed for legal tasks. . According to a 2024 survey, 76% of legal departments and 68% of law firms use AI legal tools at least once per week ( Team 2024 ) . Despite the high adoption figures, practitioners found those tools less than satisfactory. Legal practitioners are among the least benefited by AI tools: research has found that the legal domain is among the lowest compared with other domains in productivity gain under AI adoption ( Jaffe et al. 2024 ) . Among possible reasons for the contrast between the enthusiasm and the effectiveness of AI legal tools, one critical concern derives from the AI algorithms’ design. Today’s AIs are mostly unexplainable, raising concerns regarding fairness, trustworthiness, and ethics ( Mehrabi et al. 2021 ; Caton and Haas 2024 ) using AIs in the high-stake, “critical societal domain” ( Chen et al. 2024 ) of law. Specifically, emerging AI legal tools (e.g., ( MetaLaw Similar Case Search: Boosting your legal research efficiency 10-fold with one click 2024 ; Large Model for Legal Application - New AI product from PKULAW.COM(2024) GPT ) ) normally apply an end-to-end approach (i.e., users only control input/output) in assisting legal tasks. Such an approach left no space for users to intervene in the intermediate processes of AI algorithms, raising doubts from both ethical and practical grounds using AI-generated results. In extreme cases, the approach might even lure practitioners into overtrusting AI results, posing significant risks for themselves and their clients (e.g., ( Weiser 2023 ) ). One prevalent way to resolve the conflict between human control and AI is through designing systems following the human-AI collaboration paradigm ( Wang et al. 2020 ) . Through careful human-AI interaction design ( Horvitz 1999 ; Amershi et al. 2019 ) , users can maintain certain agencies using the system while benefiting from AI assistance. Applications in other critical societal domains (e.g., healthcare) have demonstrated that systems following the human-AI collaboration paradigm can reap benefits in real-world high-stakes scenarios (e.g., ( Zhang et al. 2024 ) ). Despite the potential of human-AI collaboration, works at the intersection of computation and law are nascent in exploring this paradigm. Although some claimed to have designed a human-AI system ( Han et al. 2024 ) , we need further work through the lens of legal experts to understand the legal workflow–a first step in better aligning AI legal tools with user needs. Informed by this gap, we raised our research question: How should we design a human-AI legal research tool that enables legal practitioners while mitigating the risks of end-to-end AI tools? As a first step, we focused on one use case: precedent search . We chose this use case because 1) precedent search is an essential task for legal practitioners across the world regardless of the legal system ( Alexander 1990 ) , 2) precedent search is an established ground for algorithmic application (i.e., with traditional search), presenting a more immediate ground for introducing AI support, and 3) precedent search is instrumental but tedious and labor-intensive, an ideal candidate scenario where AI legal tools could benefit legal practitioners. Additionally, we scope our study within the context of the Chinese legal system. While the weight of precedents varies across legal systems, we argue that the Chinese system, which blends civil and common law practices, presents an ideal arena for our case study. This hybrid nature allows us to derive insights potentially generalizable to civil and common law systems. With the precedent search use case and the Chinese law context, we aimed to answer: RQ1 What is the legal practitioner’s workflow in precedent search? RQ2 What are the challenges in today’s legal tools that impose friction in precedent search? RQ3 What are the concerns and expectations of legal practitioners in AI legal tools? We conducted a qualitative study with five Chinese legal practitioners to answer the research questions. We blended methods like think-aloud and interviews to understand legal practitioners’ needs through multiple lenses (Sec. 3 ). Analyzing the multimodal data we collected (Sec. 4 ), we identified a pattern of the precedent search workflow: precedent search can be seen as an iterative sensemaking process , the product of which impacts various stakeholders. Participants surfaced multiple challenges of today’s precedent search tools, including a lack of support in search term development, limited reading assistance, and unpleasant context switching between tools. Meanwhile, users unanimously acknowledged the potential of AI but implied AIs should assist instead of automating the current workflow. Leveraging the qualitative findings, we articulate a potential prototype for a human-AI precedent search system (Sec. 5 ). We envision such a tool to 1) provide step-by-step AI assistance corresponding to the precedent search workflow that guarantees full user control and 2) leverage sensemaking techniques such as semantic zoom ( Bederson and Hollan 1994 ) to provide a seamless interface for users to navigate the precedent information space. In summary, our study presents one of the first efforts at understanding how to design AI legal research tools through the lens of legal practitioners, using precedent search in China as the use case. We subsequently contribute a design vision derived from our qualitative study to introduce further discussion around this topic. 2. Background and Related Work We begin our discussion by elaborating on the legal underpinnings of precedents (Sec. 2.1 ). Then, we discuss AI legal tools’