FROC : Building Fair ROC from a Trained Classifier

Avyukta Manjunatha Vummintala, Shantanu Das, Sujit Gujar · 2024

Read FROC : Building Fair ROC from a Trained Classifier by Avyukta Manjunatha Vummintala, Shantanu Das, Sujit Gujar (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-14724.txt.

Introduction

Vummintala, Das, and Gujar address in twenty twenty-four how to make a trained scoring model fair to two protected groups at every cut-off point. Users pick a cut-off to trade off false alarms against missed cases. A model can look fair at one cut-off yet treat one group worse at another, which is a real risk in lending, hiring, and health tools.

They define fairness across the whole trade-off curve, not just at one point. They then give a fast method that adjusts a model's scores, with some randomness, to meet that standard. They prove the method loses as little accuracy as theory allows, under stated conditions. Tests on real datasets with many trained models support the claims.

Catalog record

CREATORAvyukta Manjunatha Vummintala, Shantanu Das, Sujit Gujar
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-14724

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FROC : Building Fair ROC from a Trained Classifier Avyukta Manjunatha Vummintala, Shantanu Das, Sujit Gujar Abstract This paper considers the problem of fair probabilistic binary classification with binary protected groups. The classifier assigns scores, and a practitioner predicts labels using a certain cut-off threshold based on the desired trade-off between false positives vs. false negatives. It derives these thresholds from the ROC of the classifier. The resultant classifier may be unfair to one of the two protected groups in the dataset. It is desirable that no matter what threshold the practitioner uses, the classifier should be fair to both the protected groups; that is, the ℒ p \mathcal{L}_{p} norm between FPRs and TPRs of both the protected groups should be at most ε \varepsilon . We call such fairness on ROCs of both the protected attributes ε p \varepsilon_{p} -Equalized ROC. Given a classifier not satisfying ε 1 \varepsilon_{1} -Equalized ROC, we aim to design a post-processing method to transform the given (potentially unfair) classifier’s output (score) to a suitable randomized yet fair classifier. That is, the resultant classifier must satisfy ε 1 \varepsilon_{1} -Equalized ROC. First, we introduce a threshold query model on the ROC curves for each protected group. The resulting classifier is bound to face a reduction in AUC. With the proposed query model, we provide a rigorous theoretical analysis of the minimal AUC loss to achieve ε 1 \varepsilon_{1} -Equalized ROC. To achieve this, we design a linear time algorithm, namely FROC , to transform a given classifier’s output to a probabilistic classifier that satisfies ε 1 \varepsilon_{1} -Equalized ROC. We prove that under certain theoretical conditions, FROC achieves the theoretical optimal guarantees. We also study the performance of our FROC on multiple real-world datasets with many trained classifiers. August 24, 2026 1 Introduction The use of Machine Learning based Models (MLM) in decision-making is prevalent today. Practitioners use MLMs’ predictions in college admissions, credit scores, recidivism, employment, recommender systems, etc. [ 1 , 2 ] . However, there have been several reports of such MLMs discriminating against individuals belonging to certain groups based on protected attribute such as gender, age, race, color, and religion. E.g., in [ 3 ] , predictive models are found to be biased against the black population, or the Amazon recruitment team has to stop using the AI tool for shortlisting candidates as it was biased against females [ 4 ] . [ 5 ] ; [ 2 ] ; [ 6 ] show that many of such predictive models are unfair to females. Such unfair instances have driven researchers toward building a fair MLM. An MLM that achieves fairness with the least possible compromise on traditional performance guarantees such as accuracy is desirable MLM. Building a desirable MLM involves two main steps: a) formalizing and quantifying a fairness measure and b) designing algorithms to train MLM for quantified fairness. Researchers proposed many fairness measures, majorly belonging to two categories: (i) individual fairness [ 7 ] – individuals with similar input features receive similar decision treatment irrespective of their protected attribute. (ii) Group fairness – a particular statistical property must be similar across each protected group, e.g., Disparate Impact (DI) , Equalized odds (EO) [ 8 ] . Building Fair MLM Fair machine learning models (MLMs) can be developed by targeting different stages of the model training cycle. Approaches include: (i) Pre-processing methods, which act on input data to eliminate bias [ feldman15 , 9 ] . (ii) In-processing algorithms, which intervene during training to incorporate fairness as a constraint or within the learning objective [ 10 ] . (iii) Post-processing methods, which adjust the outputs of trained MLMs to produce fair results, requiring access to sensitive attributes. In-processing and pre-processing methods are tailored to specific fairness criteria and models, necessitating retraining for each new fairness definition. Post-processing methods, in contrast, are model-agnostic and do not depend on the training process, making them suitable for domain experts with limited MLM knowledge [ 11 ] . These methods are especially favored when retraining is infeasible, such as in large-scale systems like recommender systems [ 12 ] . Given a potentially biased scoring function, this paper addresses the challenge of constructing a fair probabilistic binary classifier with a binary-protected attribute. The goal is to ensure fairness without retraining the MLM, minimizing performance loss. Fairness and Performance Trade-offs For classification, one of the desired characteristics of an MLM is calibration [ 13 ] . Suppose a classifier predicts that a given input is accepted ( Y=1 ) with probability p , then calibration demands that the fraction of the accepted population, with the same features, is p . [ 13 , 14 ] have shown that calibration and equalized odds cannot be satisfied simultaneously except for highly constrained cases. Hence, researchers have been focusing on building classifiers (MLMs) with an appropriate approximate version of fairness [ 8 ] . When it comes to practitioners, they focus on Receiver Operator Characteristics (ROC) for evaluating a classifier as it best describes the classifiers. ROC measures the relative scores of the positive versus negative instances. The area under ROC-curve (AUC) is an appropriate performance metric to measure the predictive quality of such classifiers and to segregate positive and negative samples through ranking ( [ 15 , 16 , 17 ] ). AUC is particularly beneficial when the classifier is expected to segregate positive and negative labels, and the predictions must be fair across all threshold scores. To make the practitioner’s job effortless, we introduce a novel fairness measure, namely \varepsilon_{p} -Equalized ROC – no matter what threshold it uses for classification, the classifier is approximately fair, i.e., for all possible thresholds, the distance between the corresponding points of the ROC curves for both the protected group should be withing \varepsilon distance in the \mathcal{L}_{p} norm. We aim to build a new probabilistic classifier that satisfies \varepsilon_{1} -Equalized ROC with the minimal loss in AUC w.r.t. to the scoring function s . Our Approach: We assume query access to the ROC of s . First, we make sufficiently large k queries to the ROC for the protected groups and make a piece-wise linear approximation of the ROC curves of both the protected groups. Next, we transport ROCs within \varepsilon distance of each other to minimize the loss in AUC of the resultant ROC. We can achieve such transportation by randomizing scores across certain feasible classifiers for the given ROC curve. We call the space of these classifiers as ROC Space of s . The resultant classifier from such randomization across the ROC Space is a convex combination of these classifiers. In a nutshell, we transform the given s to a fair scoring function by such ROC transport. We refer to this procedure of ROC transport as FROC . We then geometrically prove that under certain conditions, FROC is optimal . Our Contributions: • We introduce a novel group fairness notion \varepsilon_{p} -Equalized ROC, enforcing fairness over all thresholds in a score-based classification, which is extremely useful for practitioners. • Next, we model a post-processing problem as a problem of finding an optimal transformation \mathcal{H} on a given scoring function s to minimize the performance loss due to

Read the complete text