Fairness Is More Than Algorithms: Racial Disparities in Time-to-Recidivism
Jessy Xinyi Han, Kristjan Greenewald, Devavrat Shah · 2025
Read Fairness Is More Than Algorithms: Racial Disparities in Time-to-Recidivism by Jessy Xinyi Han, Kristjan Greenewald, Devavrat Shah (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18629.txt.
Introduction
Jessy Han, Kristjan Greenewald, and Devavrat Shah ask, in a 2025 paper, whether risk scores tell the whole story of racial gaps in reoffending. Most fairness work treats reoffending as a yes-or-no outcome and audits the algorithm's predictions. These authors instead study time until reoffending, inside a causal framework that lets race, the score, and life context interact. They build a formal survival test for whether the score fully explains racial differences.
Run on the well-known COMPAS data, the test finds no racial gap inside risk groups over short horizons. In the low-risk group, though, a significant gap emerges after about nine months and survives finer score bands, score noise, and a competing-risks check. The authors read this as evidence that housing, work, and support, not the algorithm alone, shape who returns. Fairness, they argue, is more than algorithms.
Catalog record
| CREATOR | Jessy Xinyi Han, Kristjan Greenewald, Devavrat Shah |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-18629
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fairness Is More Than Algorithms: Racial Disparities in Time-to-Recidivism Jessy Xinyi Han, Kristjan Greenewald, Devavrat Shah Abstract . Racial disparities in recidivism remain a persistent challenge, and the growing adoption of risk assessment algorithms has intensified the scrutiny of their sources. Past works have primarily focused on disparities in the predictions of these algorithms, viewing recidivism as a binary outcome. While sociological and criminological research has long documented non-algorithmic factors in recidivism, it remains unclear whether the risk assessments that decision-makers act on fully account for racial disparities. This work presents a multi-stage causal framework for time-to-recidivism that captures the interactions between race, the risk assessment algorithm, and contextual factors. We introduce interventional racial parity and a formal survival analysis test, conducted with observational data, of whether the algorithmic risk assessment fully accounts for racial differences in recidivism. Applied to the COMPAS dataset, the test detects no racial disparity within risk groups at short follow-up horizons. A statistically significant disparity becomes detectable after roughly nine months in the low-risk group and persists under finer score stratification, risk score perturbation, and a competing-risks analysis. This suggests that factors beyond the algorithmic scores, possibly including structural disparities in housing, employment, and social support, may shape recidivism over time, underscoring the need for policy interventions beyond algorithmic improvements, particularly for low-risk defendants. 1. Introduction With millions of formerly incarcerated people returning to prison each year, the cycle of re-offending after release, known as recidivism , remains a pressing challenge worldwide. In the United States, recidivism is a particularly complex issue closely entwined with the stark racial, economic, and social inequalities that permeate the criminal justice system. The emerging literature suggests that minority groups face disparate treatment across various stages of the criminal justice process, from the initial 911 call for service ( Han et al., 2024 ) , through policing ( Fryer, 2019 ) , court sentencing ( Rehavi and Starr, 2014 ) , probation and parole decisions ( Kimchi, 2019 ; Huebner and Bynum, 2008 ) , and re-entry support ( Western and Sirois, 2019 ) . Therefore, a close-up examination of the pathways and extent of such racial disparities must precede any effective reforms for a fair and equitable criminal justice system. Amid these systemic challenges, the increasing deployment of algorithmic risk assessment tools, such as COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) ( Northpointe, 1996 ) , has added another layer of complexity to discussions of fairness and equity in the criminal justice system. These tools, originally designed to standardize decision-making, reduce human biases in bail, parole, and sentencing decisions, and enhance consistency and efficiency in judicial outcomes, have faced continual scrutiny and criticism since their adoption ( Berk et al., 2021 ; Barabas et al., 2018 ; Corbett-Davies et al., 2017 ) . The complexity surrounding these tools arises from three dimensions. First, it concerns the competing definitions of fairness. In particular, ProPublica’s influential news report on machine bias, which analyzed the impact of the COMPAS risk assessment tool, suggested the existence of racial disparities in terms of predictive accuracy: African-American individuals who did not recidivate within a two-year period were disproportionately labeled as higher risk compared to their Caucasian counterparts ( Angwin et al., 2016 ; Larson et al., 2016 ) . Yet this is only one of several competing definitions of fairness, which in general cannot all hold at once. Second, such an algorithm-centered analysis does not fully account for the broader structural context or potential non-algorithmic factors, such as the socioeconomic conditions substantially studied in sociological and criminological literature, that may also contribute to disparities in the outcomes of recidivism ( Kubrin and Stewart, 2006 ; Kirk, 2009 ; Kirk, 2015 ) . Third, it also leaves out the time trajectory of how potential bias flows through the pathways embedded in the criminal justice system over time. 1.1. Contributions The primary contribution of this work is to provide a systematic approach to answer the following question: does the algorithmic risk assessment fully account for the racial disparities in time-to-recidivism, and if not, when and how do the remaining disparities emerge? In answering this question, the key challenge lies in disentangling the interactions between algorithmic decisions, perceived race, and additional contextual factors over time. This work overcomes the limitations of prior work towards addressing this question through the contributions summarized next. We propose a multi-stage causal framework that captures the complete trajectory from arrest to potential re-offense or return to custody. This allows us to examine both direct and indirect pathways through which racial disparities can manifest over time. The framework rests on a single modeling assumption: given fixed contexts and algorithmic decisions, race itself does not make someone recidivate sooner or later. As such, whether the algorithmic assessment fully accounts for the observed disparities is then a testable hypothesis. Building on this framework, we introduce the notion of interventional racial (dis-)parity, a fairness criterion that examines whether individuals of different races who are otherwise identical exhibit equivalent time-to-recidivism patterns under the influence of the criminal justice system and contextual factors. We move beyond static measures of fairness that treat criminal justice outcomes as a simple True or False binary predictive question to consider the dynamic and context-dependent nature of recidivism shaped by structural inequality over time, through the lens of survival analysis. To determine whether this racial parity holds, we establish Theorem 3.2 , which yields a data-driven test comparing the recidivism curves across races within the same risk score group. The challenge is that the true time-to-recidivism is often masked by censoring: when an individual returns to custody for non-criminal reasons before any re-offense, the data reveal only the earlier of the two times. To address this, we employ the log-rank test from survival analysis, which can be properly conducted with observational censored data. The causal model itself supplies the statistical guarantee for this test: under the null hypothesis, the race labels within a risk score group are exchangeable, so the test can be calibrated by permuting them. We track the log-rank statistic across the entire follow-up period and reject when its largest deviation exceeds the corresponding permutation quantile. As such, the resulting test controls the family-wise error rate (FWER) across all follow-up horizons, that is, the probability of falsely declaring a disparity at one or more horizons, accounting for the multiple testing inherent in examining the whole trajectory (Proposition 3.5 ). As a byproduct, we obtain the earliest follow-up time at which a disparity can be declared significant, which we term the detectability onset . Rejection of the null hypothesis therefore means precisely that the algorithmic risk assessment alone does not account for the racial disparities in time-to-recidivism, so factors beyond it contribute. We utilize this framework to analyze the COMPAS dataset curated by ProPublica, which records the risk scores, criminal histories, and subsequent re-arrests of roughly 10,000 defendants in Broward County, Florida. Within a short-term follow-up period, we do not find sufficient evidence to support that recidivism patterns across racial groups are different within the same risk score group. However, disparities become significant once the follow-up period exceeds the detectability onset of approximately nine months, specifically for individuals categorized as low risk