Contextual Fairness-Aware Practices in ML: A Cost-Effective Empirical Evaluation
Alessandra Parziale 1 , Gianmario Voria 1 , Giammaria Giordano 1, Gemma Catolino 1 , Gregorio Robles 2 , Fabio Palomba 1 · 2025
Read Contextual Fairness-Aware Practices in ML: A Cost-Effective Empirical Evaluation by Alessandra Parziale 1 , Gianmario Voria 1 , Giammaria Giordano 1, Gemma Catolino 1 , Gregorio Robles 2 , Fabio Palomba 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-15622.txt.
Introduction
Alessandra Parziale and five co-authors test cheap ways to make machine learning fairer, in a 2025 study. Models that sort people — for loans, jobs, or care — can treat groups unevenly. Known fixes target bias head-on, but they can also drag down accuracy, and what counts as fair shifts with the setting.
The team instead checks ordinary good practice at each stage of building a model. Careful data work, sound training habits, and honest testing may lift fairness without special bias tools. They weigh the cost of each practice against the gain, since teams have limited time and money. The findings help builders pick steps that buy the most fairness for the least cost in each setting they face.
Catalog record
| CREATOR | Alessandra Parziale 1 , Gianmario Voria 1 , Giammaria Giordano 1, Gemma Catolino 1 , Gregorio Robles 2 , Fabio Palomba 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-15622
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Contextual Fairness-Aware Practices in ML: A Cost-Effective Empirical Evaluation Alessandra Parziale 1 , Gianmario Voria 1 , Giammaria Giordano 1, Gemma Catolino 1 , Gregorio Robles 2 , Fabio Palomba 1 Abstract As machine learning (ML) systems become central to critical decision-making, concerns over fairness and potential biases have increased. To address this, the software engineering (SE) field has introduced bias mitigation techniques aimed at enhancing fairness in ML models at various stages. Additionally, recent research suggests that standard ML engineering practices can also improve fairness; these practices, known as fairness-aware practices, have been cataloged across each stage of the ML development life cycle. However, fairness remains context-dependent, with different domains requiring customized solutions. Furthermore, existing specific bias mitigation methods may sometimes degrade model performance, raising ongoing discussions about the trade-offs involved. In this paper, we empirically investigate fairness-aware practices from two perspectives: contextual and cost-effectiveness. The contextual evaluation explores how these practices perform in various application domains, identifying areas where specific fairness adjustments are particularly effective. The cost-effectiveness evaluation considers the trade-off between fairness improvements and potential performance costs. Our findings provide insights into how context influences the effectiveness of fairness-aware practices. This research aims to guide SE practitioners in selecting practices that achieve fairness with minimal performance costs, supporting the development of ethical ML systems. I Introduction Machine Learning (ML) applications continue to spread in diverse contexts, becoming integral in business operations due to their impact on efficiency, decision-making, and innovation [ 1 , 2 , 3 ] . However, this trend has raised ethical concerns around fairness —the expectation that models make unbiased decisions [ 4 ] . Bias in training data can lead models to make unfair decisions, presenting ethical and legal risks [ 5 , 6 ] . In response to these challenges, the software engineering (SE) research community, particularly within SE for Artificial Intelligence , has proposed multiple bias mitigation techniques, i.e., methods designed to reduce or eliminate biases in machine learning models by operating on data or algorithms. These solutions have been categorized as pre-processing , in-processing , and post-processing [ 7 ] , based on the ML development stage in which they operate. Different research evaluated these solutions empirically [ 8 , 9 , 10 ] , demonstrating their efficacy in mitigating bias. As fairness is a context-dependent issue, i.e., different ethical concerns arise in different contexts [ 11 ] , most of the solutions proposed so far have been evaluated under specific settings, including different application domains. Nevertheless, fairness-specific adjustments have implications for the economic sustainability of businesses . This dimension refers to the financial sustainability of the software over time [ 12 ] . In this regard, the application of these bias mitigation solutions may impact operating costs in terms of customer satisfaction, particularly with decreased performances of the resulting models [ 4 , 13 ] . De Martino et al. [ 12 ] performed a benchmark study of the implications of applying bias mitigation solutions on other sustainability dimensions, such as the economic one, highlighting that applying these algorithms involves complex trade-offs, particularly between fairness and performance. Drifting apart from specific bias mitigation methods, recent research in SE highlighted how fairness in ML can be addressed by carefully selecting common engineering practices during the development life cycle [ 14 ] . These practices have been defined as fairness-aware practices —common practices that have a positive impact on the fairness level of an ML model—and they have been cataloged in the six stages of an engineered ML development life cycle [ 15 ] . These start from early stages, with practices like ‘Multi-objective Optimization’ or ‘Data Balancing’ in ‘Requirements Elicitation’ and ‘Data Preparation’ , to practices like ‘Model Outcomes Analysis’ in the final stage of ‘Model Maintenance & Evolution’ . Furthermore, these practices have been evaluated through a survey with expert ML developers [ 16 ] , assessing the extent to which these have a positive impact on fairness, how often they are applied, and the perceived effort to be implemented. Results show that the majority of these practices have a positive impact on fairness, according to practitioners. Still, only a few of them are frequently applied despite not requiring high effort to be implemented [ 16 ] . Additionally, these practices appear to offer a distinct advantage in managing the fairness-performance trade-off, as they can enhance both fairness and model performance without the potential performance costs sometimes associated with bias mitigation techniques [ 17 ] . Stemming from these considerations, this work presents an empirical evaluation of fairness-aware practices with two main focuses. First, we perform a contextual evaluation, in which we select multiple application domains to understand if specific areas require the application of specific fairness-aware practices. Second, we introduce a novel evaluation metric to perform the trade-off analysis targeting the cost-effectiveness [ 18 ] of these practices. Such an evaluation is inspired by previous work in SE [ 19 ] and aims at representing the trade-off between the benefits of implementing a fairness practice in relation to its cost in terms of performance loss. To conduct our study, we select common datasets presented and used in previous research [ 11 , 8 , 12 ] but spanning over different contexts of application, e.g., Finance with the German credit dataset [ 20 ] or Law with the COMPAS dataset [ 11 ] . Afterward, we select a set of fairness-aware practices based on insights by practitioners regarding their positive impact on fairness and frequency of application [ 16 ] . Finally, we perform extensive experimentation with datasets and practices evaluating fairness and performance metrics to establish the cost of mitigating bias fairness through these practices. Results show that fairness-aware practices in ML models vary in effectiveness and cost-effectiveness across domains and datasets. Domains like Finance showed significant fairness improvements, while Economic showed none. Practices like Iterative Imputer , Simple Imputer , and Oversampling balanced fairness and performance well, while Mutation Testing was less effective. These results highlight the importance of context in selecting and evaluating fairness practices. Paper Structure. Section II reviews the research literature relevant to our study, highlighting the key distinctions that enable our work to push the state of the art forward. Section III outlines the research questions and describes the method used to address them. In Section IV , we present and analyze the study’s findings and discuss the implications for both researchers and practitioners. Section V addresses the primary limitations of the study and the strategies we employed to mitigate them. Lastly, Section VI offers concluding remarks. II Background and Related Work ML fairness, defined as the absence of bias against protected groups in automated decision-making systems [ 4 ] , has rapidly gained importance in the Software Engineering field. This heightened focus is reflected in a diverse and growing body of research that explores fairness from multiple perspectives [ 6 , 21 , 8 , 7 ] . As ethical controversies surrounding ML applications continue to surface [ 22 , 23 ] , they underscore an urgent need to prioritize fairness in ML practices across the field. Background. Research has proposed various bias mitigation approaches throughout the ML pipeline, categorized as pre-,