How May U.S. Courts Scrutinize Their Recidivism Risk Assessment Tools? Contextualizing AI Fairness Criteria on a Judicial Scrutiny-based Framework
Tin Nguyen, Jiannan Xu, Phuong-Anh Nguyen-Le, Jonathan Lazar, Donald Braman, Hal Daumé III, Zubin Jelveh · 2025
Read How May U.S. Courts Scrutinize Their Recidivism Risk Assessment Tools? Contextualizing AI Fairness Criteria on a Judicial Scrutiny-based Framework by Tin Nguyen, Jiannan Xu, Phuong-Anh Nguyen-Le, Jonathan Lazar, Donald Braman, Hal Daumé III, Zubin Jelveh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-02749.txt.
Introduction
Tin Nguyen and six co-authors ask how courts might judge AI tools that predict repeat crime, in a 2025 paper. Tech and legal fields have built separate ideas of fairness. That split makes it hard to turn tech measures into lasting policy. The problem is sharpest for high-stakes tools used in courts.
The team searched legal sources for signs of tech fairness ideas. Main tech criteria could be linked to rights such as due process and equal protection. Laws, though, tend to adopt only some criteria and can clash with each other. The authors build a new frame that joins court scrutiny levels with tech criteria. They test the frame against rules in Europe, China, and India as well. The aim is shared ground for policy and design.
Catalog record
| CREATOR | Tin Nguyen, Jiannan Xu, Phuong-Anh Nguyen-Le, Jonathan Lazar, Donald Braman, Hal Daumé III, Zubin Jelveh |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-02749
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
How May U.S. Courts Scrutinize Their Recidivism Risk Assessment Tools? Contextualizing AI Fairness Criteria on a Judicial Scrutiny-based Framework Tin Nguyen, Jiannan Xu, Phuong-Anh Nguyen-Le, Jonathan Lazar, Donald Braman, Hal Daumé III, Zubin Jelveh Abstract The AI/HCI and legal communities have developed largely independent conceptualizations of fairness. This conceptual difference hinders the potential incorporation of technical fairness criteria (e.g., procedural, group, and individual fairness) into sustainable policies and designs, particularly for high-stakes applications like recidivism risk assessment. To foster common ground, we conduct legal research to identify if and how technical AI conceptualizations of fairness surface in primary legal sources. We find that while major technical fairness criteria can be linked to constitutional mandates such as “Due Process” and “Equal Protection” thanks to judicial interpretation, several challenges arise when operationalizing them into concrete statutes/regulations. These policies often adopt procedural and group fairness but ignore the major technical criterion of individual fairness. Regarding procedural fairness, judicial “scrutiny” categories are relevant but may not fully capture how courts scrutinize the use of demographic features in potentially discriminatory government tools like RRA. Furthermore, some policies contradict each other on whether to apply procedural fairness to certain demographic features. Thus, we propose a new framework, integrating U.S. demographics-related legal scrutiny concepts and technical fairness criteria, and contextualize it in three other major AI-adopting jurisdictions (EU, China, and India). 1 Department of Computer Science, University of Maryland, College Park, Maryland, USA 2 Robert H. Smith School of Business, University of Maryland, College Park, Maryland, USA 3 College of Information, University of Maryland, College Park, Maryland, USA 4 George Washington University Law School, Washington DC, USA 5 Department of Criminology and Criminal Justice, University of Maryland, College Park, Maryland, USA [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected] Introduction Recidivism risk assessment (RRA) tools, widely deployed models that predict an individual’s likelihood to re-offend following a criminal charge, raise ethical concerns surrounding fairness, transparency, and potential bias. These tools rely on quantifiable factors (e.g., criminal history, age at first offense) to produce a risk score that assists court officials in bail, sentencing, parole, and other correctional decisions. Outcomes of RRA systems have a direct impact on the lives of people accused or convicted of crime and an indirect impact on their families and communities: manifesting in the forms of emotional and economic hardship when incarcerated family members serve longer sentences or parole conditions, and in the forms of diminished social capital, disintegration, and cycles of crime and poverty for communities with high rates of incarceration. RRAs have traditionally been studied in several fields, namely Psychology ( Hanson and Morton-Bourgon 2009 ) , Criminology ( Caudy, Durso, and Taxman 2013 ) , Law ( Nishi 2019 ) , Statistics ( Imrey and Dawid 2015 ) , and other interdisciplinary communities ( Mann, Hanson, and Thornton 2010 ; Sreenivasan et al. 2000 ; Hamilton et al. 2016 ) . In this cross-disciplinary work, spanning Law, AI and Human-Computer Interaction (HCI), we contribute new legal insights for fairness audit of RRA tools. Research on the fairness of RRA tools has grown significantly across many disciplines since 2016, when an investigative article 1 1 1 https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing from a non-profit organization, ProPublica, drew public attention to racial bias in COMPAS, a proprietary AI-based RRA tool used in Florida and many other U.S. states ( Angwin et al. 2022 ) . Academic research on fairness gained much traction from multiple angles following that publicized project, sometimes by directly criticizing the use of AI for RRA on fairness grounds ( Dressel and Farid 2018 ; Green and Chen 2019 ) , other times by subtly pointing out nuanced limitations of fairness criteria (e.g., it is almost impossible to simultaneously optimize for several fairness criteria and/or accuracy ( Chouldechova 2017 ; Berk et al. 2021 ) ). The technical AI literature includes procedural fairness (“fairness through unawareness” or excluding a feature from model input ( Bart, Teodorescu, and Morse 2024 ) ) and two mathematical categories of outcome-based fairness: group and individual fairness. Aligned with the intuitive definition by Kleinberg, Mullainathan, and Raghavan (2017) , group fairness , or group parity, is achieved when a statistical metric of interest, e.g., positive outcome rate, is equalized across different groups with respect to a sensitive feature, e.g. race or sex/gender ( Pedreshi, Ruggieri, and Turini 2008 ) . Dwork et al. (2012) introduced individual fairness based on the intuition that similar individuals should get similar outcomes. Formally, for a pair of individuals, given an input-space distance metric to measure how differently situated they are and an output-space distance metric to measure how different the distributions of their possible outcomes are, individual fairness requires that their output-space distance should be upper-bound by their input-space distance. The more similarly situated two persons are (e.g., same demographics and criminal histories), the more likely that they receive similar outcomes (e.g., RRA scores). RRA tools have already been widely adopted across U.S. government bodies for decades. Motivated by the emerging need to identify a constitutional basis for establishing enforceable fairness standards on such AI tools, our main research questions (RQs) are: RQ1.a : Do technical AI fairness criteria for RRA have an enforceable legal basis under the U.S. Constitution? RQ1.b : Are there any legal challenges when such abstract fairness criteria are translated into concrete policies? RQ2 : How to develop a new legal framework to mitigate those challenges, informed by established legal theories? Two main contributions follow. First, we analyzed primary sources of law, including the U.S. Constitution, case law interpreting the Constitution, as well as statutes and regulations at the state and federal levels. The goal is to explore a legal basis for technical fairness criteria, as well as challenges when operationalizing legally relevant concepts of fairness. Second, to address the challenges, we propose an integrated framework that combines technical fairness criteria and the judicial “scrutiny” concept (typically applied when the government treats people differently based on their demographics). This “scrutiny” concept is particularly suited to the RRA use case (compared to other high-stakes AI-assisted decision-making use cases such as loan approval ( Goyal et al. 2024 ) or job application screening ( Rigotti and Fosch-Villaronga 2024 ) , which are typically undertaken by private-sector actors) because the U.S. judiciary has developed elaborate scrutiny standards to scrutinize discriminations by public-sector actors (e.g., state and federal government bodies) against private citizens. Our framework utilizes a common unit of analysis, scrutiny-worthiness, to help auditors decide on which fairness-related criteria and for which demographics they should prioritize their audit. Background: Conceptualization of Fairness HCI vs. AI For a high-level overview, Narayanan et al. (2024) conducted a comprehensive review of empirical research on the perceived fairness of AI-assisted decision-making in the organizational setting (e.g., employee performance review, hiring). Their work builds upon the theoretical framework of organizational justice by Colquitt (2001) , which includes four dimensions: procedural fairness, interpersonal fairness, informational fairness (i.e., explainability), and distributive (i.e., outcome-based) fairness. However, only two of their four fairness dimensions (distributive fairness, and to a lesser extent, procedural fairness) are frequently discussed within the fairness literature of the AI community, as comprehensively reviewed by Pessach and Shmueli (2022) . Furthermore, Ryan, Nadal, and Doherty (2023) interviewed roughly