FairML: A Julia Package for Fair Classification
Jan Pablo Burgard, João Vitor Pamplona · 2024
Read FairML: A Julia Package for Fair Classification by Jan Pablo Burgard, João Vitor Pamplona (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01585.txt.
Introduction
Fair classification in machine learning is the subject of a 2024 paper by Jan Pablo Burgard and João Vitor Pamplona. The authors present FairML, a package for the Julia language. The tool is built to cut unfairness in the final forecast, such as when one group is treated worse than another.
The work splits fair learning into three stages. First, the data are resampled to fix imbalance. Next, a classifier is trained, and fair methods can be built into that step. Last, the cut-off used to turn scores into labels is chosen with care. Tests with simulated data show how each stage works alone and how the stages work when they are combined.
Catalog record
| CREATOR | Jan Pablo Burgard, João Vitor Pamplona |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-01585
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FairML: A Julia Package for Fair Classification Jan Pablo Burgard, João Vitor Pamplona Abstract . In this paper, we propose FairML.jl , a Julia package providing a framework for fair classification in machine learning. In this framework, the fair learning process is divided into three stages. Each stage aims to reduce unfairness, such as disparate impact and disparate mistreatment, in the final prediction. For the preprocessing stage, we present a resampling method that addresses unfairness coming from data imbalances. The in-processing phase consist of a classification method. This can be either one coming from the MLJ.jl package, or a user defined one. For this phase, we incorporate fair ML methods that can handle unfairness to a certain degree through their optimization process. In the post-processing, we discuss the choice of the cut-off value for fair prediction. With simulations, we show the performance of the single phases and their combinations. 1. Introduction The increase of automated decision-making necessitates the development of fair algorithms. These algorithms must adhere to societal values, particularly those that promote non-discrimination [ 16 ] . While machine learning can offer precise classifications, depending on the data situation it can also inadvertently perpetuate classification biases in crucial domains like loan approvals [ 20 ] and criminal justice [ 23 ] . For instance, loan approval algorithms may unfairly disadvantage single applicants by considering marital status. Similarly, criminal justice algorithms that associate race with recidivism risk can lead to discriminatory sentencing, neglecting individual circumstances. This underscores the critical need for fair classification frameworks to guarantee equal opportunities and outcomes, especially when applied within artificial intelligence application. Driven by the growing concern of bias perpetuation by algorithms, the field of fair classification has seen a significant rise. Numerous research papers are now dedicated to exploring approaches that can mitigate bias across a wide range of algorithms. Notable examples include fair versions of logistic and linear regression [ 5 ] , support vector machine [ 33 ] , random forests [ 43 ] , decision trees [ 1 ] , and generalized linear models (GLMs) [ 21 ] . These methods are designed to promote fair and equitable outcomes for all individuals by reducing potential biases that may stem from historical data or algorithmic design choices. Moreover, in machine learning, training data for automated decision-making algorithms often originates from surveys. These surveys are usually designed using a sampling plan, which can deviate from the common assumption that each data point is sampled independently and with an equal probability of inclusion. Disregarding this can introduce additional bias. To mitigate this issue, approaches that handle mixed effects were proposed. Some examples can be seen in applications like Psychology [ 10 ] and Medicine [ 15 ] . For a detailed discussion on survey methods and sampling strategies, see [ 28 ] . Packages for fair classification are already part of the literature, with versions available for Python [ 27 ] and R [ 36 ] . There also exists the Fairness package [ 2 ] in Julia , aiming to equalize accuracies across sensitive groups. Although these packages present several techniques, none of them consider mixed effects. Moreover, the package developed for R considers the fairness metrics statistical parity, equality of opportunity and individual fairness. Our proposal focuses more on disparate impact, disparate mistreatment, false positive rate equality and false negative rate equality. We choose these metrics because they can be adapted as constraints in the model. Besides that, our package considers fairness as constraints, solving the constrained optimization problems via solver while the Python package uses other algorithms such as boosting tree [ 19 ] that penalizes unfairness. Additionally, our package handles mixed effects data. The Julia programming language has been growing increasingly, especially in the field of machine learning. One reason is the availability of robust tools for optimization problems [ 6 ] . For this reason, a package for fair classification in Julia that takes into account an optimization problem adds value to the academic community. This paper is organized as follows: In Section 2 , we establish the theoretical underpinnings of fair classification. In Section 3 , we present a novel resampling method for preprocessing data with the aim of reducing disparate impact. In Section 4 , we introduce optimization problems, previously proposed in the literature, that address unfairness metrics. There we also adapt the optimization methods for data with mixed effects. In Section 5 , we present cross-validation-based post-processing methods to determine an optimal cut-off value for the classification process. Finally, in Section 6 , we conduct a comprehensive evaluation of our proposed package’s effectiveness through various tests. Our key findings and potential future directions are presented in Section 7 . 2. Machine Learning for fair classification In machine learning, binary classification algorithms are used to estimate a specific classification \hat{y}\in\{-1,1\} for a new data point x based on a training set \mathcal{D}={(x^{\ell},y_{\ell})}^{n}_{\ell=1} , with n being the number of points. For the point x^{\ell} \in X=\left[x^{1},\dotsc,x^{n}\right] , if y_{\ell}=1 , we say that x^{\ell} is in the positive class and if y_{\ell}=-1 , x^{\ell} belongs to the negative class for each \ell\in[1,n]:=\{1,\cdots,n\} . Moreover, x^{\ell}\in\mathbb{R}^{p+1} , for each \ell\in[1,n] , due to the addition of an extra column with the value 1 as the data intercept. When aiming for fairness in binary classification, we balance achieving good accuracy ( AC ) with ensuring fairness for observations \ell based on their sensitive feature s_{\ell}\in\{0,1\} , this is a standard approach in fair classification as stated by [ 41 ] . In this work the set of sensitive features is represented by SF , being the name of the sensitive variables. While fairness in machine learning can be assessed through various metrics, in this paper we focus on disparate impact (DI) and disparate mistreatment (DM) that can be seen in [ 42 , 4 ] , respectively. The main reason is that they are already adapted to constraints within an optimization model, as demonstrated in [ 41 ] . Considering the true labels and the predicted classifications of a supervised machine learning approach, we can categorize the data into four groups. A point is classified as true positive ( TP ) or true negative ( TN ) if its predicted class (positive or negative, respectively) matches its true label. Conversely, points are classified as false positives ( FP ) or false negatives ( FN ) if their predicted class differs from the true label. Based on this classification scheme, we can calculate accuracy, a metric where higher values indicate better classification performance. The formula for accuracy is as follows: AC:=\dfrac{TP+TN}{TP+TN+FP+FN}\in[0,1]. Now, we present the fairness metrics. Disparate Impact Disparate impact refers to a situation where the probability under the prediction model (\mathbb{P}) is different conditional on the sensitive feature values. A classifier is considered fair with respect to disparate impact if the probability of the point being classified as positive is equal when conditioning on the sensitive feature s , i.e., \mathbb{P}(\hat{y}_{\ell}=1|s_{\ell}=0)=\mathbb{P}(\hat{y}_{\ell}=1|s_{\ell}=1). To compute the disparate impact of a specific sensitive feature s consider: \displaystyle\mathcal{S}_{1}=\{x^{\ell}:\ell\in[1,n],\text{ }s_{\ell}=1\}, \displaystyle\mathcal{P}=\{x^{\ell}:\ell\in[1,n],\text{ }y_{\ell}=1\}, \displaystyle\mathcal{D}^{\mathcal{P}}_{0}=\mathcal{S}_{0}\cap\mathcal{P}, \displaystyle\mathcal{D}^{\mathcal{P}}_{1}=\mathcal{S}_{1}\cap\mathcal{P}, (1) \displaystyle\begin{split}&\mathcal{S}_{0}=\{x^{\ell}:\ell\in[1,n],\text{ }s_{\ell}=0\},\\ &\mathcal{N}=\{x^{\ell}:\ell\in[1,n],\text{ }y_{\ell}=-1\},\\ &\mathcal{D}^{\mathcal{N}}_{0}=\mathcal{S}_{0}\cap\mathcal{N},\\ &\mathcal{D}^{\mathcal{N}}_{1}=\mathcal{S}_{1}\cap\mathcal{N}.\end{split} Let \mathcal{S}_{0} and \mathcal{S}_{1} be disjoint subsets of dataset X , where