Fairness under Competition
Ronen Gradwohl, Eilam Shapira, Moshe Tenneholtz · 2025
Read Fairness under Competition by Ronen Gradwohl, Eilam Shapira, Moshe Tenneholtz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16291.txt.
Introduction
Force one hiring algorithm to be fair and the market may still end unfair, because several fair algorithms competing can interact badly. Ronen Gradwohl, Eilam Shapira, and Moshe Tennenholtz open that ecosystem question in 2025, modeling firms whose classifiers each satisfy a standard fairness rule.
Even when every classifier is individually fair, the combined outcome can tilt, and the tilt scales with how alike the classifiers are and how much their data overlap. Worse, tuning a biased algorithm toward fairness can drag the whole ecosystem's fairness down. Experiments back the theory. The authors call for fairness to be judged at the level of the market, not just the model, a shift with regulatory teeth.
Catalog record
| CREATOR | Ronen Gradwohl, Eilam Shapira, Moshe Tenneholtz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-16291
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fairness under Competition Ronen Gradwohl, Eilam Shapira, Moshe Tenneholtz Abstract Algorithmic fairness has emerged as a central issue in ML, and it has become standard practice to adjust ML algorithms so that they will satisfy fairness requirements such as Equal Opportunity. In this paper we consider the effects of adopting such fair classifiers on the overall level of ecosystem fairness . Specifically, we introduce the study of fairness with competing firms, and demonstrate the failure of fair classifiers in yielding fair ecosystems. Our results quantify the loss of fairness in systems, under a variety of conditions, based on classifiers’ correlation and the level of their data overlap. We show that even if competing classifiers are individually fair, the ecosystem’s outcome may be unfair; and that adjusting biased algorithms to improve their individual fairness may lead to an overall decline in ecosystem fairness. In addition to these theoretical results, we also provide supporting experimental evidence. Together, our model and results provide a novel and essential call for action. 1 Introduction Algorithmic decision-making systems are not immune to human prejudices. This has been demonstrated by ample empirical evidence: For example, the use of algorithmic decision-making in determining loan approval and interest rates has led to minority applicants facing higher loan rejection rates and higher interest rates than non-minority applicants ( Bartlett et al., 2022 ; Fuster et al., 2022 ) , and credit card issuers offering lower credit limits for women than men ( Gupta, 2019 ; Telford, 2019 ) . Additional contexts in which such algorithmic bias has been documented are as diverse as job recruitment ( Doleac and Hansen, 2016 ) , insurance premiums and payouts ( Angwin et al., 2017 ) , college admissions ( Baker and Hawn, 2022 ; Gándara et al., 2024 ) , and more ( O’Neil, 2016 ) . To contextualize, consider a bank or other financial institution that issues loans and is subject to regulatory oversight. The bank employs historical data to train a classifier—a decision rule—to determine loan eligibility, with the aim of maximizing profit through successful loan repayments. However, such classifiers frequently exhibit biases against protected groups, such as ethnic minorities. In response, regulators can mandate fairness restrictions to mitigate these disparities. Bias is not only normatively problematic, but also has tangible adverse effects on the utility of affected individuals and, more broadly, on the welfare of the disadvantaged group. The regulator’s goal in imposing fairness adjustments to classifiers is thus to enhance the welfare of historically disadvantaged populations. One principled approach to fairness adjustment involves specifying a utility function, and requiring the classifier to ensure equal welfare across groups. Prominent fairness criteria such as Demographic Parity (DP) ( Agarwal et al., 2018 ; Dwork et al., 2012 ) , as well as Equal Opportunity (EO) and Equalized Odds (ED) ( Hardt et al., 2016 ) can be interpreted as special cases within this welfare-based framework. For instance, DP corresponds to a utility specification where each approved applicant gains a utility of one and zero otherwise. EO and ED, correcting for some criticisms about DP, does the same but only for the restricted set of deserving applicants. Fairness interventions are intended to promote equity. While even widely adopted constraints like DP and EO may lead to suboptimal long-term outcomes ( Liu et al., 2018 ) , common wisdom holds that proper fairness adjustments of ML algorithms is an essential requirement. But is the adoption of fair algorithms by firms sufficient to guarantee a fair AI ecosystem , in which multiple firms interact? Recent work in AI has recognized that AI systems should be evaluated in the context of multi-agent system, in which several stakeholders are active ( Kurland and Tennenholtz, 2022 ; Boutilier et al., 2024 ) . In lending, multiple financial institutions underwrite loans and issue credit cards; in employment, many firms vie for the same pool of job applicants; in insurance, numerous insurance companies offer policies; and in education, a large number of colleges compete over the same cohorts of students. In such multi-firm environments, equity is not about whether a single firm’s decisions are fair, but whether there is overall discrimination in the number and quality of offers issued and opportunities afforded to loan, job, insurance, and college applicants. In the world of algorithmic fairness, this translates into a novel question. Suppose there are several competing firms, each of which adopts a fair classifier. Does the adoption of fair classifiers induce a fair ecosystem, in which there is no overall discrimination? In this paper, we introduce the study of fairness with competing firms . We provide fundamental definitions, and demonstrate the failure of fair classifiers in obtaining fair ecosystems. In particular, we show that even if competing classifiers are individually fair, the ecosystem’s outcome may be unfair; and that adjusting biased algorithms to improve their individual fairness may lead to an overall decline in ecosystem fairness. Contributions We begin in Section 2 by developing a model of ecosystem fairness. We suppose there are multiple lenders who make loan offers to borrowers; however, the model applies analogously to job candidates and employers, insurance buyers and insurers, or student applicants and colleges. For this model we define our main notion of fairness under competition, a competition analogue of EO that we call Equal Opportunity under Competition (EOC). The level of EOC here is a measure of how far the ecosystem is from satisfying EOC, where a lower level implies higher ecosystem fairness. We also define a second, welfare-based version of EOC. An ecosystem satisfies this second notion if the welfare of different groups of borrowers is equalized when the lenders use their classifiers to make loan offers. In Section 3 we turn to study EOC. Our main results are that, even when the individual classifiers satisfy EO, the ecosystem may be far from EOC. We quantify how far, based on model primitives, and provide worst-case bounds. In particular, we identify two distinct forces under which EO classifiers do not satisfy EOC. The first force, analyzed in Section 3.1 , arises from differences in the correlations between the classifiers on the protected groups. The second force, analyzed in Section 3.2 , arises when the lenders’ pools of potential borrowers are overlapping but not identical. We quantify the extent to which each force decreases ecosystem fairness. To illustrate the first force, consider the following example: Example 1 There are two lenders and two groups of borrowers. Each lender has vast, distinct data on group 1, and trains a classifier to predict loan repayment. Neither lender has sufficient data on group 2, so both outsource to a third-party, who provides each lender with an (identical) classifier. Each lender uses her own and the third-party classifiers to offer loans to individuals predicted to repay the loan. In this example, if the lenders’ and third-party’s classifiers have the same accuracies, then each lender’s decisions satisfy EO. Note, however, that the lenders’ predictions on group 2 are identical, as they are using the same classifier, and so the Pearson correlation between them is high. On the other hand, since the lenders use different classifiers with different training data on group 1, their respective predictions on this group