Demographic-Agnostic Fairness without Harm
Zhongteng Cai, Mohammad Mahdi Khalili, Xueru Zhang · 2025
Read Demographic-Agnostic Fairness without Harm by Zhongteng Cai, Mohammad Mahdi Khalili, Xueru Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-24077.txt.
Introduction
Fairness rules for prediction tools often force equal outcomes across groups, which can drag accuracy down for all. Zhongteng Cai and 2 co-authors seek fairness that does no harm, and they drop a common assumption: that group labels are known at training time. Many real data sets lack those labels.
Their method learns a classifier that splits people into groups on its own. It then trains a separate predictor for each learned group. Math on sample needs suggests the method can beat older paths, even ones that use true group labels. Tests on fake and real data back the claim. The aim is top accuracy for each group, as the group itself would choose.
Catalog record
| CREATOR | Zhongteng Cai, Mohammad Mahdi Khalili, Xueru Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-24077
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Demographic-Agnostic Fairness without Harm Zhongteng Cai, Mohammad Mahdi Khalili, Xueru Zhang Abstract As machine learning (ML) algorithms are increasingly used in social domains to make predictions about humans, there is a growing concern that these algorithms may exhibit biases against certain social groups. Numerous notions of fairness have been proposed in the literature to measure the unfairness of ML. Among them, one class that receives the most attention is parity-based , i.e., achieving fairness by equalizing treatment or outcomes for different social groups. However, achieving parity-based fairness often comes at the cost of lowering model accuracy and is undesirable for many high-stakes domains like healthcare. To avoid inferior accuracy, a line of research focuses on preference-based fairness, under which any group of individuals would experience the highest accuracy and collectively prefer the ML outcomes assigned to them if they were given the choice between various sets of outcomes. However, these works assume individual demographic information is known and fully accessible during training. In this paper, we relax this requirement and propose a novel demographic-agnostic fairness without harm (DAFH) optimization algorithm, which jointly learns a group classifier that partitions the population into multiple groups and a set of decoupled classifiers associated with these groups. Theoretically, we conduct sample complexity analysis and show that our method can outperform the baselines when demographic information is known and used to train decoupled classifiers. Experiments on both synthetic and real data validate the proposed method. 1 Introduction Machine learning (ML) and automated decision-making systems trained with real-world data can have inherent bias and exhibit discrimination against certain social groups ( Barocas, Hardt, and Narayanan 2023 ; Zhang et al. 2019 ; Zhang, Khalili, and Liu 2020 ; Zhang et al. 2020 ; Zhang and Liu 2021 ) . One common approach to mitigating the issue is to impose certain fairness constraints while making ML predictions. Among all fairness notions proposed in the literature, a class that has received the most attention is based on parity (equality) in treatment or outcomes for different social groups, e.g., demographic parity ( Dwork et al. 2012 ) and equalized odds ( Hardt, Price, and Srebro 2016 ) that require (true/false) positive rates to be equalized across different groups. However, enforcing these parity-based fairness notions often comes at the cost of lowering model accuracy, resulting in an inherent trade-off between fairness and accuracy ( Menon and Williamson 2018 ; Pang et al. 2024 ; Khalili, Zhang, and Abroshan 2021 ; Khalili, Zhang, and Abroshan 2023 ; Zuo, Khalili, and Zhang 2023 ; Pham, Zhang, and Zhang 2023 ; Zuo et al. 2024 ; Zuo, Zhu, and Khalili 2025 ; Abroshan, Elliott, and Khalili 2024 ) . In safety-critical domains such as healthcare, sacrificing predictive accuracy in exchange for fairness is undesirable as it violates both beneficence (i.e., do what is best for patients) and non-maleficence (i.e., do no harm) principles required in healthcare ethics ( Beauchamp and Childress 1994 ) . Related Works Preference-based No harm Without sensitive attributes Intersectionality Zafar et al. 2017 ✓ ✓ ✗ ✗ Ustun, Liu, and Parkes 2019 ✓ ✓ ✗ ✓ Lahoti et al. 2020 ✗ ✗ ✓ ✓ Islam, Chen, and Cai 2024 ✗ ✗ ✓ ✗ Pang et al. 2024 ✗ ✓ ✓ ✗ DAFH (Ours) ✓ ✓ ✓ ✓ Table 1: Comparison with related works: Zafar et al. 2017 and Ustun, Liu, and Parkes 2019 consider preference-based fairness to avoid harm on specific groups, but both require knowledge of sensitive attributes to generate group partition. Islam, Chen, and Cai 2024 ; Lahoti et al. 2020 consider parity-based fairness without accessing sensitive attributes, but they violate no harm constraint. Pang et al. 2024 introduces fairness-without-harm when sensitive attributes are inaccessible, but based on parity-based fairness metric. In contrast, our work aims to avoid harm without accessing sensitive attributes, and also deals with the intersectionality caused by the existence of multiple sensitive attributes. To avoid harming model accuracy, a few studies focus on preference-based fairness ( Zafar et al. 2017 ; Ustun, Liu, and Parkes 2019 ) . The idea is to train decoupled ML models for different social groups so that each group of individuals would collectively prefer the model assigned to them to other models. In other words, if individuals were allowed to choose among a set of ML models, a majority of them would still stick to the assigned model. Unlike parity-based fairness, preference-based fairness aims to achieve fairness without harm and avoids inferior accuracy for each group. In this paper, we focus on a preference-based fairness notion similar to the one proposed in ( Ustun, Liu, and Parkes 2019 ) , which requires that the decoupled ML models trained for different social groups satisfy two properties: rationality and envy-freeness . The former ensures that the accuracy of the decoupled model evaluated on the associated group is higher than that of the pooled model trained with data from all groups, while the latter ensures each group’s model has better accuracy than the models assigned to other groups. In addition to ( Ustun, Liu, and Parkes 2019 ) , some subsequent works proposed other variants of preference-based fairness and developed algorithms to train models satisfying the proposed fairness, which we discuss more in Section 2 . However, existing methods for achieving preference-based fairness typically assume that individual demographic information (i.e., sensitive attributes such as race and gender) is available during training or can be inferred from side information ( Zhu et al. 2023 ) , so that the decoupled models can be trained on the groups partitioned based on the sensitive attributes ( Ustun, Liu, and Parkes 2019 ) . Unfortunately, individuals’ sensitive attributes are not always accessible in practice, either because regulations mandate parity of treatment, prohibiting the use of sensitive attributes in decision-making ( Barocas and Selbst 2016 ) , or due to user privacy concerns that prevent the disclosure of such attributes. For example, when ML models are trained on patient data from multiple hospitals, protected information must be de-identified to comply with regulations such as the Health Insurance Portability and Accountability Act (HIPAA) ( U.S. Congress 1996 ) . Similarly, in applications like speech or face recognition, sensitive attributes such as race and gender may be unknown, yet it remains desirable to achieve high accuracy across all demographic groups. Even when sensitive attributes are available, partitioning groups based on them may not lead to more accurate decoupled models. Because individuals are often associated with multiple sensitive attributes such as gender, race, and age, models that are fair with respect to one sensitive attribute can easily violate fairness when assessed on more fine-grained intersectional groups ( Kearns et al. 2018 ) . Training decoupled classifiers for all intersectional groups is also unrealistic because each intersectional can have limited data samples and the resulting decoupled classifier can easily overfit. More importantly, sensitive attributes alone cannot adequately capture the full heterogeneity of the population. This motivates a central question: Can we identify a better group partition under which decoupled models satisfy preference-based fairness, without accessing sensitive attributes? To achieve this, we propose a demographic-agnostic fairness without harm (DAFH) optimization algorithm that jointly learns (i) a group classifier that finds the