Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Set
Gordon Dai *1, Pavan Ravishankar *1, Rachel Yuan 1, Daniel B. Neill †1 2 3, Emily Black †4 5 · 2025
Read Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Set by Gordon Dai *1, Pavan Ravishankar *1, Rachel Yuan 1, Daniel B. Neill †1 2 3, Emily Black †4 5 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-15634.txt.
Introduction
The authors study fairness among equally accurate models in a 2025 paper. For many tasks, a whole set of models performs almost equally well, a Rashomon set. Choosing among them at random may leave fairness gains on the table, or hide arbitrary harm to individuals.
The paper offers methods to sample models from that set efficiently and to find the fairest ones by measures such as statistical parity. It derives the chance that a person prediction flips within the set, and expressions for the set size and fairness range. The message is to choose deliberately, not arbitrarily.
Catalog record
| CREATOR | Gordon Dai *1, Pavan Ravishankar *1, Rachel Yuan 1, Daniel B. Neill †1 2 3, Emily Black †4 5 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 144 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-15634
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Set Thanks: * Co-first authors (equal contribution) Thanks: † Co-senior authors (equal contribution) Thanks: 1 Machine Learning for Good Laboratory, Department of Computer Science, NYU Courant Institute of Mathematical Sciences, New York University Thanks: 2 Robert F. Wagner Graduate School of Public Service, New York University Thanks: 3 Center for Urban Science and Progress, Tandon School of Engineering, New York University Thanks: 4 Center for Data Science, New York University Thanks: 5 Department of Computer Science and Engineering, Tandon School of Engineering, New York University Gordon Dai *1, Pavan Ravishankar *1, Rachel Yuan 1, Daniel B. Neill †1 2 3, Emily Black †4 5 Abstract . When selecting a model from a set of equally performant models, how much unfairness can you really reduce? Is it important to be intentional about fairness when choosing among this set, or is arbitrarily choosing among the set of “good” models good enough? Recent work has highlighted that the phenomenon of model multiplicity—where multiple models with nearly identical predictive accuracy exist for the same task—has both positive and negative implications for fairness, from strengthening the enforcement of civil rights law in AI systems to showcasing arbitrariness in AI decision-making. Despite the enormous implications of model multiplicity, there is little work that explores the properties of sets of equally accurate models, or Rashomon sets, in general. In this paper, we present five main theoretical and methodological contributions which help us to understand the relatively unexplored properties of the Rashomon set, in particular with regards to fairness. Our contributions include methods for efficiently sampling models from this set and techniques for identifying the fairest models according to key fairness metrics such as statistical parity. We also derive the probability that an individual’s prediction will be flipped within the Rashomon set, as well as expressions for the set’s size and the distribution of error tolerance used across models. These results lead to policy-relevant takeaways, such as the importance of intentionally looking for fair models within the Rashomon set, and understanding which individuals or groups may be more susceptible to arbitrary decisions. Conference: ACM Conference on Fairness, Accountability, and Transparency; ; Submitted for publication DOI: xx.xxxx/xxxx ISBN: 979-8-4007-0450-5/24/06 1. Introduction Recent work has drawn renewed attention to the fact that there are often many (approximately) equally accurate models available for the same prediction task ( 22 , 5 , 7 ) . This phenomenon—often called the Rashomon effect ( 7 ) , predictive multiplicity ( 22 ) , or model multiplicity ( 5 ) —has wide-ranging implications for both understanding and improving fairness, as these equally accurate models often differ substantially in other properties such as fairness ( 28 , 21 ) or model simplicity ( 30 , 31 , 29 ) . As prior work has pointed out, this multiplicity of models can be viewed as both a fairness opportunity and a concern ( 5 , 10 ) . On the positive side, legal scholarship has pointed to the fact that model multiplicity is relevant to how to interpret and enforce U.S. anti-discrimination law, and specifically, can strengthen the disparate impact doctrine to more effectively combat algorithmic discrimination ( 3 ) . In a recent paper, Black et al. ( 3 ) suggest that the phenomenon of model multiplicity could support a reading of the disparate impact doctrine that requires companies to proactively search the set of equally accurate models for less discriminatory alternatives that have equivalent accuracy to a base model deemed acceptable for deployment from a model performance perspective. On the negative side, several scholars have pointed out that facially similar models, with equivalent accuracy but differences in their individual predictions, can suggest that some model decisions are arbitrary since they seem to be made on the basis of model choice that does not impact performance (e.g., a <1% change in a model’s training set accuracy) ( 22 , 2 , 17 ) . This arbitrariness can impact model explanations and recourse as well: individuals with decisions that are unstable across small model changes may not receive reliable explanations for their model outcome, or ways to change it ( 25 , 6 , 4 ) . Further, if there is a group-based asymmetry of arbitrariness–e.g., if female loan applicants have more arbitrariness in their decisions than male loan applicants— this could lead to a group-based equity concern in and of itself. Understanding the extent of the benefits and risks of model multiplicity relies upon an understanding of the properties of the Rashomon set , or the set of approximately equally accurate models for a given prediction task, i.e., equally accurate up to some error tolerance \epsilon . While models in the Rashomon set are considered equivalent from a performance perspective, they may differ substantially in other properties—for the purposes of our paper, we focus on fairness. In order to understand the utility of searching for fairer models within the Rashomon set as suggested by recent legal literature, or the extent of the dangers of arbitrariness surfaced by the algorithmic fairness community, we need to understand more about Rashomon sets themselves. For example, whether companies should be required to search for less discriminatory models ( 3 ) rests on the question of how much of the disparity can be reduced by optimizing over the Rashomon set, as compared to choosing an arbitrary model without regard to fairness. In other words, how much do we gain by being intentional about fairness when selecting models within the Rashomon set? Similarly, concerns about arbitrariness relate to rates and distributions of the chance that an individual will have their prediction changed— is this arbitrariness harmful if only predictions that are very uncertain get flipped, or if all demographic groups have an equal chance of flipping? We can shed light on these important questions by understanding even basic facts about the Rashomon set, such as: what does a randomly sampled model from the Rashomon set look like? What is the average fairness for various metrics on the Rashomon set? How might one search through the Rashomon set? Can we find the fairest model within the Rashomon set? What is the chance that any one individual might experience a change in prediction in the Rashomon set? Or even, how large is the Rashomon set? Despite the enormous implications of model multiplicity, there is little work that explores the properties of Rashomon sets in general. In this paper, we present five main theoretical and methodological contributions that answer the above questions and more—furthering our understanding of the relatively unexplored properties of the Rashomon set, in particular with regards to fairness: • First, we define the largest possible Rashomon set R_{N}(\epsilon) , for N records drawn from a given data distribution, assuming an allowable error tolerance of \epsilon . We then develop an efficient method for sampling models uniformly at random from within the Rashomon set. • Second, we present two computationally efficient methods to find the fairest model within the Rashomon set, for statistical parity and error rate balance respectively. • Third, we derive the asymptotic probability that any individual will have their prediction flipped within the models of the Rashomon set