Aggregating Concepts of Fairness and Accuracy in Prediction Algorithms
David Kinney · 2025
Read Aggregating Concepts of Fairness and Accuracy in Prediction Algorithms by David Kinney (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08829.txt.
Introduction
David Kinney presents a 2025 paper on how to judge prediction algorithms that must be both accurate and fair. Accuracy and fairness pull against each other, and each can be measured many ways. Designers lack shared rules for trading them off or for combining the many measures into one verdict.
Kinney argues for a simple answer: score an algorithm by a linear blend of its accuracy and fairness metrics. The argument leans on a classic result by Harsanyi in preference aggregation, which shows when individual rankings combine into a weighted sum. Under the conditions Kinney lays out, anyone who cares about both values should accept the blend as the all-things-considered measure. The paper is a work of philosophy of technology, not an experiment.
Catalog record
| CREATOR | David Kinney |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-08829
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Aggregating Concepts of Fairness and Accuracy in Prediction Algorithms David Kinney Abstract . An algorithm that outputs predictions about the state of the world will almost always be designed with the implicit or explicit goal of outputting accurate predictions (i.e., predictions that are likely to be true). In addition, the rise of increasingly powerful predictive algorithms brought about by the recent revolution in artificial intelligence has led to an emphasis on building predictive algorithms that are fair, in the sense that their predictions do not systematically evince bias or bring about harm to certain individuals or groups. This state of affairs presents two conceptual challenges. First, the goals of accuracy and fairness can sometimes be in tension, and there are no obvious normative guidelines for managing the trade-offs between these two desiderata when they arise. Second, there are many distinct ways of measuring both the accuracy and fairness of a predictive algorithm; here too, there are no obvious guidelines on how to aggregate our preferences for predictive algorithms that satisfy disparate measures of fairness and accuracy to various extents. The goal of this paper is to address these challenges by arguing that there are good reasons for using a linear combination of accuracy and fairness metrics to measure the all-things-considered value of a predictive algorithm for agents who care about both accuracy and fairness. My argument depends crucially on a classic result in the preference aggregation literature due to Harsanyi ( Harsanyi, 1955 ) . After making this formal argument, I apply my result to an analysis of accuracy-fairness trade-offs using the COMPAS dataset compiled by Angwin et al. ( ( Angwin et al., 2016 ) ). Conference: The 2025 ACM Conference on Fairness, Accountability, and Transparency; June 23–26, 2025; Athens, Greece The 2025 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’25), June 23–26, 2025, Athens, Greece DOI: 10.1145/3715275.3732031 ISBN: 979-8-4007-1482-5/2025/06 CCS: Theory of computation Design and analysis of algorithms † † cc-license: by 1. Introduction Consider an algorithm that recommends podcasts to users of a digital media service based on data collected, with consent, from those users. Suppose that across all users, there is a strong correlation between a user’s gender and their podcast preference: in particular, women are much more likely than men to listen to true crime podcasts ( Beattie et al., 2023 ) . On a mainstream view, there is some degree of unfairness attached to this algorithm using these correlation data to recommend a true crime podcast to a woman solely because she is a woman, especially if she is a new user with no history of listening to true crime podcasts ( Yang and Dobbie, 2020 ) . More serious violations of fairness in predictive systems can occur when a person is harmed in some way (e.g., given a harsher prison sentence) in virtue of an algorithmic prediction based primarily on their possession of a protected characteristic such as race or gender ( Angwin et al., 2016 ) . At the same time, predictive algorithms should strive for accuracy. In the case described above, when an algorithm has little user data to base its predictions on other than basic demographic data, it may be that the best recommendation, from the perspective of expected accuracy, is one that suggests true crime podcasts to women as a blanket policy. Thus, it seems that we are at least sometimes forced to balance a desire for fairness against a desire for accuracy when building predictive systems. There is now a large literature on the need to balance accuracy and fairness in predictive machine learning, with debates over whether the trade-off between accuracy and fairness is inherent or contingent (e.g., ( Wick et al., 2019 ; Cooper et al., 2021 ; Menon and Williamson, 2018 ; Chen et al., 2018 ; Zhao and Gordon, 2022 ; Liu and Vicente, 2022 ; Barlas et al., 2021 ; Žliobaitė, 2015 ; Friedler et al., 2021 ) ). See ( Li and Li, 2025 ) for a comprehensive review of this literature. While the balance between accuracy and fairness is, in itself, a difficult balance to strike, the trade-off is made even more difficult by the fact that both accuracy and fairness are highly contested, multi-faceted concepts that can be measured in a multitude of ways, with each measure differing non-trivially from the other. Indeed, an ontology of fairness metrics compiled by Franklin et al. contains eighteen different fairness metrics that measure thirty-nine different notions of fairness ( Franklin et al., 2022 ) . This state of variegation is even more pronounced for measures of accuracy. One desideratum for an accuracy measure that is fairly non-controversial is that it be the expected value of a strictly proper scoring rule . A scoring rule is a function s that provides a measure s(p,y) of the accuracy of a probability distribution p over some set of possible outcomes Y when the actual outcome is y\in Y . Such a scoring rule is strictly proper just in case the expected value \sum_{y\in Y}q(y)s(p,y) is maximized if and only if p=q . While some scoring rules have become standard in the literature (e.g., the Brier, logarithmic, and spherical scoring rules), there are an infinite number of strictly proper scoring rules from which one can choose ( ( Merkle and Steyvers, 2013 ) , p. 292). Thus, evaluators of predictive algorithms must not only balance fairness against accuracy, but must also balance different concepts of fairness and accuracy that are operationalized by different measures against one another. This paper uses a theoretical result to offer practical guidance for simultaneously managing these trade-offs. Specifically, I show that a classic result due to Harsanyi ( ( Harsanyi, 1955 ) ) provides a compelling reason for an agent who cares about both accuracy and fairness to calculate a linear combination of a set of real-valued utility functions, each of which is defined on a measure of accuracy or fairness. To my knowledge, this kind of first-principles argument for the linear aggregation of measures of fairness and accuracy for predictive algorithms has not previously been put forward. While my conclusion does not settle the question of exactly how we should balance preferences for various notions of fairness and accuracy when evaluating utility functions, it does tell us the precise mathematical form that such a solution might take, while also clarifying which parts of this balancing act have technical solutions and which parts must be solved through normative argumentation. The remainder of this paper proceeds as follows. In Section 2 , I provide the necessary formal preliminaries for the main result. In Section 3 , I argue that measures of accuracy and fairness should be aggregated via linear combination, as per Harsanyi’s theorem. In Section 4 , I discuss what I take to be the significant implications of this result. In Section 5 , I use the data set produced by Angwin et al. in their audit of the COMPAS algorithm for predictive criminal recidivism ( ( Angwin et al., 2016 ) ) to illustrate this significance. I conclude in Section 6 . 2. Formal Preliminaries 2.1. Prediction Algorithms Let a prediction algorithm be a function \mathcal{A}:\mathbf{X}\rightarrow\Delta(Y) , where \mathbf{X} is