The Effect of Enforcing Fairness on Reshaping Explanations in Machine Learning Models
Joshua W. Anderson, MS 1, Shyam Visweswaran, MD, PhD 1,2 · 2025
Read The Effect of Enforcing Fairness on Reshaping Explanations in Machine Learning Models by Joshua W. Anderson, MS 1, Shyam Visweswaran, MD, PhD 1,2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-02265.txt.
Introduction
Clinical machine learning must be accurate, fair across patient groups, and explainable enough for doctors to trust, and improving one can disturb the others. Joshua Anderson and Shyam Visweswaran examine in 2025 a neglected link in that triangle: how enforcing fairness changes a model's explanations.
It was already known that fairness work can move predictive performance, but its effect on explainability was little studied. If explanations shift when fairness is enforced, clinicians may see different reasons for the same prediction, which affects trust. The study maps that interaction for healthcare models, where explanations are an essential ingredient of safe use.
Catalog record
| CREATOR | Joshua W. Anderson, MS 1, Shyam Visweswaran, MD, PhD 1,2 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-02265
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Effect of Enforcing Fairness on Reshaping Explanations in Machine Learning Models Joshua W. Anderson, MS 1, Shyam Visweswaran, MD, PhD 1,2 Abstract \institutes 1 Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA 2 Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA Abstract Trustworthy machine learning in healthcare requires strong predictive performance, fairness, and explanations. While it is known that improving fairness can affect predictive performance, little is known about how fairness improvements influence explainability, an essential ingredient for clinical trust. Clinicians may hesitate to rely on a model whose explanations shift after fairness constraints are applied. In this study, we examine how enhancing fairness through bias mitigation techniques reshapes Shapley-based feature rankings. We quantify changes in feature importance rankings after applying fairness constraints across three datasets: pediatric urinary tract infection risk, direct anticoagulant bleeding risk, and recidivism risk. We also evaluate multiple model classes on the stability of Shapley-based rankings. We find that increasing model fairness across racial subgroups can significantly alter feature importance rankings, sometimes in different ways across groups. These results highlight the need to jointly consider accuracy, fairness, and explainability in model assessment rather than in isolation. Introduction Machine learning (ML) models are increasingly used in clinical settings to support diagnosis, risk stratification, and treatment decision-making. Their ability to learn complex patterns from high-dimensional data has made them valuable tools for improving efficiency and potentially enhancing patient outcomes. However, as these systems become more deeply integrated into clinical workflows, concerns about their reliability, transparency, and fairness have become central to discussions about their safe and responsible deployment 1 . Given the ethical complexities and inherent uncertainty in clinical practice, clinicians need to understand and validate ML recommendations before acting on them 2 . One of the most common approaches to transparency is the use of model explanations, which aim to visualize how data influences a model’s reasoning, making predictions more understandable to clinicians, researchers, and other stakeholders 3 . Inherently interpretable, white-box models can be understood through the structure itself as the primary explanation. For example, the coefficients of a linear model can explain the association between features and the predicted outcome. In contrast, many complex models (e.g., ensemble methods, gradient-boosted trees, deep networks) function as black-box systems whose internal mechanics are too complex to interpret directly. In these cases, post-hoc explanation methods provide an interpretable approximation of the original model’s behavior 4 . Alongside the need for transparency, there is growing concern about model fairness in clinical ML 5 . Fairness and bias are two terms that refer to the same fundamental issue: disparities in how models perform across different groups. Fairness metrics are used to identify and quantify these disparities in model performance, while bias mitigation strategies aim to reduce them. Improving fairness, or reducing bias, in ML models, particularly across racial groups, is crucial to minimizing unfair outcomes that disproportionately affect marginalized or historically underserved populations 6 . The intersection of model explanations and model fairness has been previously studied. Prior work has shown that explanation methods can support fairness evaluation and can also be incorporated into bias mitigation procedures 7 ; 8 . Explanations have also been found to influence human judgments of fairness, highlighting their role in shaping perceptions of algorithmic systems 9 . Prior work has largely focused on explanations as tools for achieving fairness. In this study, we evaluate how fairness constraints reshape explanations. We measure this interaction across three axes: predictive performance, fairness performance, and model explanation variability. These three dimensions capture not only how fairness constraints affect model accuracy and group-level fairness, but also how they alter the explanations that end users rely on to interpret model reasoning. Background Explainability . SHAP offers several advantageous properties, such as local accuracy, missingness, and consistency, and provides a principled way to generate additive feature-importance scores for individual predictions 10 . Local explanations are provided by SHAP values, which quantify how much each feature shifts a single prediction above or below the model’s baseline or average output. They represent how much each feature’s value moved the model’s output away from the baseline or average prediction for that specific instance 11 . Global explanations are generated by summarizing SHAP values across multiple predictions to assess each feature’s overall influence. By examining how consistently a feature raises or lowers predictions and by how much on average, typically using the mean absolute SHAP value, one can assess its overall importance. Global explanations are obtained by aggregating SHAP values across predictions for many instances to determine each feature’s overall influence. By examining how consistently a feature raises or lowers predictions and by how much on average, typically using the mean absolute SHAP value, one can assess its overall importance. Figure 1 shows examples of both (a) local and (b) global explanations for a model predicting risk of urinary tract infection (UTI) in children. (a) Local Explanation (b) Global Explanation Figure 1 : Example local and global explanations for predicting urinary tract infection (UTI) in pediatric patients. (a) Waterfall plot for a patient in the UTI dataset, showing SHAP feature importance values corresponding to the patient’s predicted probability of having a UTI. (b) Bar plot showing the mean absolute SHAP values for each feature in the UTI dataset, demonstrating the ranking and relative influence of features on the model’s predicted probability of UTI in pediatric patients. The computational complexity of \phi(f,x) makes computing exact values impractical for large datasets, so several estimation methods have been developed to leverage model architectures for efficient computation. Lundberg and Lee proposed the most popular approximation methods, including KernelSHAP (model-agnostic), DeepSHAP (neural networks), and TreeSHAP (tree ensembles) 10 ; 12 . Fairness in Machine Learning . The legal field distinguishes two core forms of discrimination: disparate treatment and disparate impact. Disparate treatment, called direct discrimination, occurs when individuals are explicitly treated differently because they are part of a protected group. Disparate impact, called indirect discrimination, occurs when a seemingly neutral regulation disproportionately harms members of a protected group. ML models trained without protected attributes that directly indicate group membership are less prone to causing disparate treatment. However, they can still unintentionally produce disparate impact and should be evaluated accordingly 13 . Fairness is measured using protected attributes for which disparate performance or treatment may raise ethical or legal concerns, such as gender, age, socioeconomic status, and disability. There are three most common fairness metrics: • Demographic parity (DP; statistical parity) : A model satisfies demographic parity if the overall rate of positive predictions is the same across protected groups (e.g., Black vs. non-Black groups). • Equalized odds (EOD) : A model satisfies equalized odds if true positive rates and false positive rates are equal across protected groups. • Predictive parity (PP; predictive value parity) : A model satisfies predictive parity if the positive predictive value (PPV), the probability that a positive prediction is correct, is the same across protected groups. For most predictive models, whether in medicine or in domains like criminal justice, DP is rarely appropriate because of group-level differences in the predicted outcomes. For example, different racial groups have different recidivism rates, with Black individuals generally showing higher rates of re-offending compared to White individuals in the