Investigating Data Interventions for Subgroup Fairness: An ICU Case Study Erin Tan, Judy Hanwen Shen, Irene Y. Chen Abstract In high-stakes settings where machine learning models are used to automate decision-making about individuals, the presence of algorithmic bias can exacerbate systemic harm to certain subgroups of people. These biases often stem from the underlying training data. In practice, interventions to “fix the data” depend on the actual additional data sources available—where many are less than ideal. In these cases, the effects of data scaling on subgroup performance become volatile, as the improvements from increased sample size are counteracted by the introduction of distribution shifts in the training set. In this paper, we investigate the limitations of combining data sources to improve subgroup performance within the context of healthcare. Clinical models are commonly trained on datasets comprised of patient electronic health record (EHR) data from different hospitals or admission departments. Across two such datasets, the eICU Collaborative Research Database and the MIMIC-IV dataset, we find that data addition can both help and hurt model fairness and performance, and many intuitive strategies for data selection are unreliable. We compare model-based post-hoc calibration and data-centric addition strategies to find that the combination of both is important to improve subgroup performance. Our work questions the traditional dogma of “better data” for overcoming fairness challenges by comparing and combining data- and model-based approaches. 1 Introduction When machine learning algorithms are used to guide real-world decision making, they are at risk of reproducing disparities found in the training data ( 8 ; 11 ; 2 ) . The majority of technical advancement has focused on model interventions that remedy the learned disparities of the model ( 17 ; 1 ) . Many have also called for sociotechnical approaches such as abstaining from using AI ( 37 ; 14 ; 5 ) or system approaches such as collecting more data ( 8 ; 15 ; 9 ) . “Improving the data,” in particular, has been frequently suggested as a vague, catch-all solution to the problem ( 4 ; 18 ) . However, there is a gap between broad suggestions of data remedies and what is effective in practice. There are three main challenges that practitioners face when translating between the promise of better data and practical methods of data addition. First, the best method of addition is task-dependent; the success of an approach in one method does not guarantee success on another dataset ( 32 ) . Second, we are restricted by the quality of the available data. The limitations of the dataset do not exist because practitioners choose to use suboptimal data, but because better data is simply unavailable. Third, the effect of data changes on subgroups as well as the general population must be considered. Due to the first challenge—that data limitations are task dependent—we narrow our focus on the case study of intensive care unit (ICU) data in particular. Bias in healthcare is particularly consequential, as disparities may translate into unequal treatment quality ( 26 ) , delayed or missed diagnosis ( 34 ; 35 ) , or increased costs of care ( 27 ) . Moreover, we find clinical EHR datasets to be well-representative of the second and third challenges described above. To combat the lack of quality data, EHR datasets are often aggregated across multiple sources (e.g. multiple hospitals or admission departments); this strategy allows for larger dataset sizes, yet effective methods for achieving better data quality remain unclear. Finally, EHR datasets contain heterogeneous patient subgroups, so the notion of better data may depend on the subgroup in question. Grounding our work in this domain allows us to better isolate the impact of specific data interventions, although we emphasize that our methods can be applied to any multi-source dataset. Our work focuses on the specific goal of improving subgroup performance through data interventions. Specifically, our contributions are: • We review the landscape of data interventions for fairness, identifying a gap between recommended data solutions and available algorithms. • We evaluate and compare existing data interventions for subgroup performance and find that they are not effective across two datasets—the eICU Collaborative Research Database and MIMIC-IV. We identify mean discrepancy as a key barrier to success for data addition. • We compare data-centric approaches with post-hoc model calibration and identify a hybrid approach best improves subgroup performance. 2 Related Work Data Perspectives on Fairness Pre-processing fairness interventions address biases that exist within the training data ( 28 ; 43 ) . These biases may arise as a result of systematic faults in data collection, where collection methods inherently under-represent certain groups or promote differences in data quality, labeling procedure, or missingness. When biases exist in the data, they are near-impossible to correct using in-process or post-processing interventions alone ( 8 ; 31 ) . “Adding more data” does not account for practical realities—in most cases, we can assume that in-distribution data has already been exhausted. Additionally, new data may be drawn from a biased distribution, such that adding more samples from the same distribution can amplify those disparities ( 43 ) . Adding data from external data sources yields unpredictable outcomes, as the counteracting effects from increased sample size and introduction of distribution shift work against one another ( 36 ; 40 ) . Fairness under distribution shifts is generally evaluated by quantifying the worst-case performance within a set of potential shifts ( 10 ; 40 ) . Distributionally-robust optimization (DRO) uses this notion by directly optimizing worst-case subgroup performance under distribution shift during training ( 30 ) . Parallel work in invariant learning aims to discover more generalizable patterns in data which remain stable across environments, but these are in-process methods ( 3 ; 33 ) . Data Interventions Data interventions for subgroup fairness are most commonly evaluated in the single-source setting. In general, the goal of all these data interventions is to mitigate the problem of imbalanced class distributions, a common cause of sample size bias and under-representation bias ( 43 ) . These include sampling-based techniques, such as over-sampling from minority classes ( 19 ; 7 ) or under-sampling from majority classes ( 24 ) . Standard over-sampling selects samples from the minority group to duplicate, which helps in adding more data, but does not offer any new information and is prone to overfitting the minority distribution. Newer methods use generative models, such as variational autoencoders, to add synthetic data, which work better on high-dimensional data than sampling ( 39 ) . Another approach to balancing subgroups is reweighing, which assigns importance weights to training instances, such that minority examples are given greater influence over the loss function to equalize the contributions of all subgroups ( 6 ; 22 ) . Calibration Calibration is one way to combat algorithmic bias—for all individuals receiving a predicted probability of p , ensure that a p fraction of them are positive samples in actuality. When we satisfy this constraint for all subgroups across some sensitive attribute (e.g. race or gender), we ensure that the model predictions give meaningful estimates of uncertainty for all individuals. Generally, model calibration is performed in post-processing by learning a 1 -dimensional transformation function \phi:[0,1]\rightarrow[0,1] mapping the uncalibrated model predictions f(x) to their calibrated form \phi(f(x)) . For example, Isotonic Regression ( 41 ) is a non-parametric calibration technique which learns a monotonically non-decreasing \phi . We use Isotonic Regression in our experiments to compare pre-processing and post-processing interventions. 3 Problem Setup Our goal is to improve the performance of a target subgroup using data interventions. In a real-world setting, our primary options for increasing data are to either 1) generate synthetic samples or 2) add out-of-distribution samples from external sources. In this work, we focus on the latter. In this section, we establish a formal framework for examining the impact of multi-source data addition on model fairness across subgroups. Figure 1: Change in Overall and subgroup-level accuracy after Whole-Source data addition (Logistic Regression). The change in Overall performance (a) does not reflect equally upon changes in performance across subgroups. For example, we observe that adding data from any source to Target Hospital 458 improves overall accuracy. While this change is reflected across the White (b) and Black (c) subgroups, the Other (d) subgroup experiences near-uniform decreases in test performance as a result of data addition. Scaling data directly from the Target Hospital (diagonal of each plot) generally yields improvements in overall and subgroup accuracy, but is typically not the best -performing data addition choice. We begin with a model f_{D_{\text{train}}}(D_{\text{test}}):X\mapsto\{0,1\} that is trained on dataset D_{\text{train}}=\{(x_{i},y_{i})\}_{i=1}^{n_{\text{train}}} and evaluated on D_{\text{test}}=\{(x_{i},y_{i})\}_{i=1}^{n_{\text{test}}} , where we consider a binary target variable: y\in\{0,1\} . We also have some discrete sensitive attribute A within our feature set X (e.g. race) which we can use to stratify our datasets into subgroups (e.g. by race: Asian, Black, White). We identify some target subgroup a^{\prime}\in A whose performance we would specifically like to improve, and use D_{\text{test}}^{a^{\prime}} to denote the subset of D_{\text{test}} where A=a^{\prime} . Let 0f(x_{i})] (3) 4.3 Data Addition Experiments To test our data addition heuristics, we performed three types of experiments, detailed in this section. • Baseline : For each source and model class, we train on 1000 samples and test on 400 samples. Change in performance is measured with respect to the Baseline . • Whole-Source : For each data source, we train a model on the combined training set of the 1000 samples from the test data source, as well as 1000 samples from the additional data source. As a control, we also train a model on 2000 samples for each test source. • Subgroup-Level : To the existing test data source of 1000 data points, we add only a subset of a source D_{i}^{a} without adding all other subgroups in D_{i} . Additional samples are capped at 1000 , although the vast majority of subgroups are smaller. 5 Limitations of Existing Data Addition Methods We motivate the need for informed data selection heuristics by exploring the complex realities of fairness under data addition. Figure 1 shows the change in overall and subgroup accuracies after Whole-Source data addition compared to the Baseline . Our key observation is that the effects of data addition vary across subgroups. For example, for Target Hospital 458 , addition of any data source improves the Overall, White, and Black accuracies but worsens outcomes for the Other subgroup. Similarly, for Target Hospital 420 , predictor accuracy for Overall and White subgroups improved by data addition with the Black and Other subgroups sometimes suffered significantly worsened accuracies. This is even the case where more data from the same source is added. Figure 2: Change in subgroup ratio vs. Change in subgroup test accuracy after Whole-Source data addition on the eICU Dataset. These findings show that it is 1) necessary to view the effects of data interventions (e.g. addition) at a subgroup level and 2) that it is not always beneficial to add all available data into your training set when subgroup performance matters. We need to make careful data decisions by considering the effects on different subgroups, but it remains unclear how to improve subgroup performance in this setting. In the following subsections, we investigate several common data selection strategies, showing that they are not effective in improving subgroup performance in our settings. Data Balancing The first set of informed data selection strategies we evaluated seeks to improve the presence of the target subgroup within the training dataset ( 12 ) . Traditionally, subgroup balancing is performed either by oversampling from minority classes ( 7 ) or undersampling majority classes ( 24 ) . The intuition for data balancing via selection is that, for some minority group a^{\prime} , we select the dataset D_{i_{*}} that maximizes the representation of a^{\prime} . Let |D_{i}| denote the total number of samples available from source \mathcal{S}_{i} . We investigate two different selection criteria to perform data balancing via addition: 1. Adding sources with the highest proportion of samples from the target subgroup i_{*}=\arg\max_{i}\frac{|D_{i}^{a^{\prime}}|}{|D_{i}|} 2. Adding sources with the greatest number of samples from the target subgroup i_{*}=\arg\max_{i}|D_{i}^{a^{\prime}}| In Figure 2 , we plot the changes in subgroup ratio after Whole-Source data addition against the change in subgroup performance. The ratio of a subgroup within a dataset is computed as \text{Ratio}(D,a)=\frac{|D^{a}|}{|D|} , so that we define the change in subgroup ratio as \Delta\text{Ratio}(D_{\text{train}},D_{\text{added}},a)=\text{Ratio}(D_{\text{train}\cup\text{added}},a)-\text{Ratio}(D_{\text{train}},a) . We did not observe a statistically significant relationship in 2 across any subgroup; increasing the ratio of a subgroup does not reliably improve subgroup performance in isolation. To test the second heuristic—choosing the source with the most available samples for the target subgroup—we examine the change in performance after Subgroup-Level data addition. The results are presented in Figure 3 , where we find that the quantity of available subgroup data within a source is not meaningful for determining how data addition will steer subgroup performance. Through our first two data selection methods—selecting the sources with the greatest number and highest ratio of samples from the test subgroup, respectively—we found that simply increasing the representation of samples from the test subgroup does not necessarily yield positive results if the data itself is uninformative about the target. Figure 3: Change in subgroup accuracy as a function of samples added in Subgroup-Level data addition (see Section 4 for details). Across nearly all combinations of subgroups and Test Hospitals, we find that adding more samples does not necessarily lead to larger performance gains. These visualizations lead to the conclusion that naive subgroup balancing is an uninformative data selection heuristic. Distribution Matching One critical observation from Figure 3 is that it suggests it is more important to understand the test distribution rather than the just the added source distribution, as some test sources generally benefit from any type of data scaling (e.g. Hospital 443, 252), whereas others exhibit minimal or negative effects from any out-of-distribution scaling (e.g. Hospital 199, 73). Thus, the next data selection method we evaluate chooses the sources which are most similar to the test distribution. Intuitively, this should yield desirable outcomes because we minimize the amount of distribution shift while still scaling the number of training samples. We employ the subgroup similarity score proposed in 36 as a heuristic to quantify the distance between two source distributions. Given that D_{\text{test}}^{a}\sim\mathcal{S}_{\text{target}}^{a}=P and D_{i}\sim\mathcal{S}_{i}=Q for any external source i , we derive a binary predictor s_{PQ}:X\rightarrow[0,1] where datapoints belonging to distribution P are assigned label 1 and points in Q are assigned label 0 . Intuitively, if P and Q are similarly distributed, s should output probabilities around 0.5 (random guessing) on an empirical test set containing points from P . If P and Q are very different, s_{PQ} should output probabilities closer to 1 . Thus, we can use as a heuristic for distributional similarity the expected value of the predictor probability on P while assuming a uniform probability for the empirical samples from P : \textsc{Score XY}=\mathbb{E}_{(x,y)\in P}[s_{PQ}(x,y)] (4) Applying our similarity score function to the subgroup case, we define the subgroup similarity score for a shared subgroup a between two sources to be: \textsc{Score XY}_{a}=\mathbb{E}_{(x,y)\in P}[s_{PQ}(x,y)|(x,y)\in a] (5) We again consider the results from Subgroup-Level data addition, this time from a distributional perspective. Using the subgroup similarity score introduced in Equation 5 , we find no statistically significant correlation between the distributional similarity of an added source with the test source and the added source’s effect on subgroup test accuracy. This relationship, or lack thereof, is presented in Figure 4 . Figure 4: Subgroup similarity score (using features and labels) vs. Change in subgroup test accuracy for the White (left) and Black (right) subgroups (eICU). The scores are computed using only the patients from the target subgroup in each source. These performance changes result from Subgroup-Level data addition. We do not observe statistically significant correlations in any Test Hospital for either subgroup. 6 Mean Consistency: A Barrier to Effective Data Intervention Following the findings in Section 5 , we shift our focus away from demographic differences between datasets, and instead look at the base rates themselves. One drawback of enforcing existing notions of fairness is that they lead to theoretically-bound tradeoffs with the overall performance of the model ( 13 ; 25 ) . The incompatibility arises due to differing base rates between subgroups ( 23 ; 42 ; 44 ) . This line of work implies that base rates should be examined when understanding the disparities induced by data. In our setting, we have the opportunity to favorably adjust subgroup base rates through data addition. More formally, we look at mean consistency, defined as the difference between a model’s predictive mean and the true mean of the test set. Concretely, we define the mean discrepancy between a learned predictor and a test dataset as follows: \textsc{Disc}(f,D)=\frac{1}{|D|}\bigg|\sum_{(x,y)\in D}\mathbf{1}[f(x)\geq t]-y\bigg| (6) This expresses the difference between the average output label of a model f over a dataset D and the true average value of the labels in D . Using this measure, we hypothesize that large mean discrepancies between subgroups are one source of unfairness in our datasets of interest. This can be understood as a situation where one subgroup exhibits starkly different outcome rates compared to others, leading to an overall training dataset and subsequent model mean which are skewed from the subgroup label mean. The formal relationship between mean discrepancy (Equation 6 ) and accuracy (Equation 1 ) can be derived using the triangle inequality: \displaystyle\textsc{Disc}(f,D) \displaystyle\leq\frac{1}{|D|}\sum_{(x,y\in D)}\bigg|\mathbf{1}[f(x)\geq t]-y\bigg| \displaystyle\leq 1-\text{Acc}(f,D) The mean discrepancy bounds the test accuracy, such that the greater the mean discrepancy of a model is, the lower its performance ceiling will be. This guarantee is empirically confirmed in Figure 5 (a), where we exhibit a clean boundary line y=1-x . We find that this correlation transfers to the data addition scenario—in Figure 5 (b), we plot a statistically significant correlation between the change in subgroup mean discrepancy after adding a data source with the resultant change in subgroup accuracy. This trend is seen across all ethnic subgroups. This finding provides strong evidence that mean discrepancy plays a key role in determining the effect of data addition on subgroup performance. The resultant data selection strategy is thus to add sources that correct the subgroup mean discrepancy exhibited over the base model. This leads to the following selection criteria: i_{*}=\arg\min_{i}\textsc{Disc}(f_{D_{\text{train}+i}},D_{\text{test}}^{a})-\textsc{Disc}(f_{D_{\text{train}}},D_{\text{test}}^{a}) Figure 5: Mean consistency and subgroup performance: (a) Subgroup Mean Discrepancy (Eq. 6 ) vs. Subgroup Test Accuracy (Eq. 1 ). (b) Change in Subgroup Mean Discrepancy vs. Change in Subgroup Test Accuracy. Strong negative correlations are observed across all subgroups in (a), and in all minority subgroups in (b). Figure 6: Difference in best-case subgroup performance (without calibration) and worst-case subgroup performance (with calibration) after Whole-Source data addition. The overwhelming majority of subgroups show positive differences, supporting the idea that making informed choices for data addition is more important than calibrating post-hoc. Data-Centric vs. Post-Processing Interventions Our empirical results complement existing techniques that use calibration to achieve better subgroup performance. Next, we apply post-processing calibration via Isotonic Regression on models trained using Whole-Source data addition. In these experiments, we additionally withhold a validation set of 200 samples drawn from the same distribution (or mixture of distributions) as the training set. We use this data to perform calibration after training. Black Other White W/o Cal. With Cal. W/o Cal. With Cal. W/o Cal. With Cal. Best 4.2% \pm 1.6 3.4% \pm 0.9 6.1% \pm 2.2 8.7% \pm 1.3 2.3% \pm 0.3 2.8% \pm 0.1 Median 0.6% \pm 2.6 1.5% \pm 1.1 1.2% \pm 3.2 0.1% \pm 1.7 0.7% \pm 0.9 2.2% \pm 0.1 Worst -4.1% \pm 2.8 -5.9% \pm 4.3 -3.4% \pm 2.8 -13.0% \pm 4.4 -3.4% \pm 2.7 1.2% \pm 0.4 Table 1: Average change in subgroup accuracy for the Best, Median, and Worst Whole-Source data additions across all sources, with and without calibration. In Figure 6 , we compare the change in subgroup accuracy after optimal data selection against calibration on sub-optimal data selection. We find that post-processing calibration is outperformed simply by choosing a better (or the best) available data source. However, we find that the combination of both data selection and calibration is generally most effective at improving subgroup performance. Table 1 summarizes the average change in subgroup accuracy for the best, median, and worst available data sources with and without calibration. We find that the combination of data addition with calibration, on average, yields the highest best-case outcomes, along with better worst-case outcomes. Overall, we find that without intentional dataset selection that takes into account desired fairness outcomes, calibration alone does not guarantee improvement. Under this problem set-up where we have fine-grained control over dataset composition, we find it necessary to first make informed data addition choices in order to maximize the performance boost from post-processing calibration. 7 Discussion In this work, we investigate the landscape of data interventions for improving subgroup performance where multiple additional sources of data are available. Through comprehensive experiments across the eICU and MIMIC-IV datasets, we demonstrate that naive strategies for data addition, such as increasing subgroup representation or minimizing distribution shift, fail to reliably improve subgroup performance. We identify mean discrepancy as a key bottleneck of subgroup performance, and formalize mean consistency as a necessary criterion for successful data interventions. We compare data interventions with post-processing calibration (model interventions), and find that “fixing the data” is more important for improving subgroup performance than direct post-hoc calibration. When we combine both data selection and calibration, we observe the best possible performance across most subgroups. 7.1 Limitations & Future Work Our experiments are purposefully narrowed to focus on ICU data using race as the sensitive attribute. This focus helps better isolate the the effects of data interventions on subgroup performance for a commonly task in clinical machine learning. Future works should investigate more tasks in the healthcare domain. Our ultimate hope is for a task-agnostic framework for the selection of data addition heuristics. Our work presents promising findings for deeper investigations into task-specific data selection heuristics. Making informed choices about training data is one of the most consequential steps of the machine learning pipeline. We hope our findings motivate deeper investigations into the practical data decision-making process, and promote methods which are conscious of the unequal effects they can have on under-served and under-represented populations. Appendix A Datasets The demographic data for both datasets used in this study—eICU and MIMIC-IV— are presented in this section. For both datasets, the examined sensitive attribute was patient self-reported race. We also provide information on the data sources used to stratify each dataset. A.1 eICU Collaborative Research Database Table 2 summarizes the subset of the eICU Collaborative Research Database used in this study. We treat different hospitals, denoted by a unique Hospital ID, as independent data sources. The full database includes 200,859 patient unit encounters in 208 hospitals across the US. For this study, we consider only the hospitals with over 2000 recorded patient encounters, a total of 12 data sources. Table 2: Subgroup information across hospitals with 2000 + patient encounters in the eICU Dataset Count (Rate) Hos ID Total Asian Black Other White 73 4320 61 (0.01) 622 (0.15) 347 (0.08) 3221 (0.76) 167 2107 29 (0.01) 154 (0.07) 421 (0.20) 1503 (0.71) 188 2299 29 (0.01) 517 (0.22) 64 (0.03) 1689 (0.73) 199 2529 3 (0.001) 42 (0.02) 48 (0.02) 2434 (0.96) 243 2812 24 (0.009) 873 (0.31) 83 (0.03) 1831 (0.65) 252 2210 7 (0.003) 152 (0.07) 50 (0.02) 1993 (0.91) 264 3745 31 (0.009) 263 (0.07) 64 (0.02) 3299 (0.90) 300 2370 19 (0.008) 267 (0.11) 84 (0.04) 2000 (0.84) 338 2762 5 (0.002) 41 (0.01) 143 (0.05) 2568 (0.93) 420 3425 52 (0.02) 157 (0.05) 276 (0.08) 2940 (0.86) 443 2580 12 (0.005) 1352 (0.53) 83 (0.03) 1119 (0.44) 458 2456 34 (0.01) 747 (0.30) 132 (0.05) 1542 (0.63) A.2 MIMIC-IV The Medical Information Mart for Intensive Care (MIMIC)-IV contains data for over 65,000 patients admitted to an ICU and over 200,000 patients admitted to the emergency department between 2008-2022 at the Beth Israel Deaconess Medical Center in Boston, MA. In this dataset, we treat the patient’s admission department as their data source. All patient encounters are categorized into three admission types: Emergency, Elective, and Urgent. Patient circumstances are vastly different between the three departments, introducing sufficient distributional changes to justify their treatment as independent sources. Table 3 summarizes the demographic rates across sources in the dataset. Table 3: Subgroup information across admission types in the MIMIC-IV Dataset Count (Rate) Adm Type Total Asian Black Hispanic Other White Elective 3464 69 (0.02) 155 (0.045) 75 (0.021) 478 (0.138) 2687 (0.775) Emergency 21669 502 (0.023) 2155 (0.099) 741 (0.034) 2965 (0.137) 15306 (0.706) Urgent 746 12 (0.016) 17 (0.022) 16 (0.021) 318 (0.42) 383 (0.51) Appendix B Results from the MIMIC-IV Dataset B.1 Experiment Details To confirm our findings in the eICU dataset, we conducted additional experiments on MIMIC-IV ( 21 ) , a freely accessible electronic health record dataset. We predict binary in-hospital mortality using diagnosis codes, procedures, and lab measurements extracted from the patient record. In the database, there are 9 possible admission types, including urgent care, surgical same-day, and emergency ambulatory services. We group the admission types into three broad categories to ensure sufficient sample sizes in each—Urgent, Elective, and Emergency care. We treat each of these admission categories as an independent data “source”. Subgroups are patient self-reported race. B.2 Results and Analysis The results of our data addition experiments are shown in Table 4 . Performance changes larger than 0.5\% in either direction are denoted with a colored arrow. For models trained on combined datasets, we find large inconsistencies in the performance changes across subgroups. For example, we find that the overall accuracy in Urgent care is notably improved when adding data from Elective departments, primarily as a result of significant improvements among the Asian, Other, and White populations. However, this improvement is also at the expense of the Black and Hispanic populations, who face 0.1\% and 1.1\% decreases in test accuracy, respectively. The decline in Hispanic performance is larger than improvements to any subgroup, begging the question of whether this addition would be desirable. Table 4: Change in overall & subgroup performance after Whole-Source data addition. Sources Change in ACC ( \Delta ) Test Added Asian Black Hispanic Other White Overall Elective Elective 0.001 0.006 \uparrow 0.007 \uparrow 0.006 \uparrow -0.001 0.000 Elective Emergency 0.001 -0.001 0.010 \uparrow -0.003 0.001 0.001 Elective Urgent -0.001 0.000 0.003 -0.003 0.002 -0.001 Emergency Elective -0.001 -0.005 \downarrow -0.003 0.010 \uparrow 0.000 -0.001 Emergency Emergency 0.002 -0.010 \downarrow -0.004 0.020 \uparrow 0.008 \uparrow 0.001 Emergency Urgent -0.001 -0.021 \downarrow -0.001 -0.003 0.001 -0.001 Urgent Elective 0.007 \uparrow -0.001 -0.011 \downarrow 0.007 \uparrow 0.006 \uparrow 0.009 \uparrow Urgent Emergency 0.005 \uparrow -0.042 \downarrow -0.018 \downarrow 0.007 \uparrow 0.016 \uparrow -0.001 Urgent Urgent 0.002 -0.012 \downarrow 0.018 \uparrow 0.000 0.001 0.004 Evaluation of Existing Data Addition Methods Tables 5 and 6 look at existing data addition strategies under the lens of fairness. They corroborate the results seen on the eICU dataset in Figures 2 and 3 , respectively, showing that these data balancing strategies are ineffective at explaining the performance gaps between subgroups. Table 5: Change in subgroup ratio and resulting change in subgroup accuracy after Whole-Source data addition on the MIMIC-IV dataset. Ethnicity Admission Type Added Source Rate Change Acc. Diff Correlation Asian Elective Elective 0.000 0.006 r=-0.28 , p=0.47 Elective Emergency 0.002 -0.001 Elective Urgent -0.002 0.000 Emergency Elective -0.002 -0.005 Emergency Emergency 0.000 -0.010 Emergency Urgent -0.004 -0.021 Urgent Elective 0.002 -0.001 Urgent Emergency 0.004 -0.042 Urgent Urgent 0.000 -0.012 Black Elective Elective 0.000 0.007 r=-0.20 , p=0.61 Elective Emergency 0.027 0.010 Elective Urgent -0.011 0.003 Emergency Elective -0.027 -0.003 Emergency Emergency 0.000 -0.004 Emergency Urgent -0.038 -0.001 Urgent Elective 0.011 -0.011 Urgent Emergency 0.038 -0.018 Urgent Urgent 0.000 0.018 Hispanic Elective Elective 0.000 0.006 r=-0.05 , p=0.89 Elective Emergency 0.006 -0.003 Elective Urgent -0.000 -0.003 Emergency Elective -0.006 0.010 Emergency Emergency 0.000 0.020 Emergency Urgent -0.006 -0.003 Urgent Elective 0.000 0.007 Urgent Emergency 0.006 0.007 Urgent Urgent 0.000 0.000 Other Elective Elective 0.000 -0.001 r=-0.62 , p=0.08 Elective Emergency -0.001 0.001 Elective Urgent 0.144 0.002 Emergency Elective 0.001 0.000 Emergency Emergency 0.000 0.008 Emergency Urgent 0.145 0.001 Urgent Elective -0.144 0.006 Urgent Emergency -0.145 0.016 Urgent Urgent 0.000 0.001 White Elective Elective 0.000 0.000 r=0.58 , p=0.10 Elective Emergency -0.035 0.001 Elective Urgent -0.131 -0.001 Emergency Elective 0.035 -0.001 Emergency Emergency 0.000 0.001 Emergency Urgent -0.096 -0.001 Urgent Elective 0.131 0.009 Urgent Emergency 0.096 -0.001 Urgent Urgent 0.000 0.004 Table 6: Subgroup samples added and resulting change in subgroup accuracy after Subgroup-Only data addition on the MIMIC-IV dataset. Ethnicity Admission Type Added Source Samples Added Acc. Diff Correlation Asian Elective Elective 19 0.006 r=-0.24 , p=0.54 Elective Emergency 23 -0.001 Elective Urgent 16 0.000 Emergency Elective 19 -0.005 Emergency Emergency 23 -0.010 Emergency Urgent 16 -0.021 Urgent Elective 19 -0.001 Urgent Emergency 23 -0.042 Urgent Urgent 16 -0.012 Black Elective Elective 44 0.007 r=-0.37 , p=0.33 Elective Emergency 99 0.010 Elective Urgent 22 0.003 Emergency Elective 44 -0.003 Emergency Emergency 99 -0.004 Emergency Urgent 22 -0.001 Urgent Elective 44 -0.011 Urgent Emergency 99 -0.018 Urgent Urgent 22 0.018 Hispanic Elective Elective 21 0.006 r=0.33 , p=0.38 Elective Emergency 34 -0.003 Elective Urgent 21 -0.003 Emergency Elective 21 0.010 Emergency Emergency 34 0.020 Emergency Urgent 21 -0.003 Urgent Elective 21 0.007 Urgent Emergency 34 0.007 Urgent Urgent 21 0.000 Other Elective Elective 137 -0.001 r=-0.36 , p=0.34 Elective Emergency 136 0.001 Elective Urgent 426 0.002 Emergency Elective 137 0.000 Emergency Emergency 136 0.008 Emergency Urgent 426 0.001 Urgent Elective 137 0.006 Urgent Emergency 136 0.016 Urgent Urgent 426 0.001 White Elective Elective 775 0.000 r=0.23 , p=0.55 Elective Emergency 706 0.001 Elective Urgent 513 -0.001 Emergency Elective 775 -0.001 Emergency Emergency 706 0.001 Emergency Urgent 513 -0.001 Urgent Elective 775 0.009 Urgent Emergency 706 -0.001 Urgent Urgent 513 0.004 Appendix C Supplemental Figures In this section, we provide supplemental figures and results that were not included in the main paper. C.1 Full Results of Whole-Source Data Addition Figure 1 shows the full set of results measuring the change in accuracy after Whole-Source data addition in the eICU dataset. The results are presented for overall as well as subgroup-level test accuracy. Figure 7: Change in Overall and subgroup-level accuracy after Whole-Source data addition. All results from abridged Figure 1 . C.2 Pareto Frontier We want to evaluate whether our data-centric methods for fairness are able to bypass the commonly-studied tradeoff between subgroup performance and overall performance. In these figures, we plot the Pareto Frontier of Whole-Source data addition in both the eICU and MIMIC-IV datasets. We do not observe any considerable tradeoff between overall and subgroup performance, a promising result. These results are visualized in 8 . Figure 8: Change in overall test accuracy vs change in subgroup test accuracy. The Pareto Frontier is plotted on each subfigure to visualize the tradeoff between fairness and performance. Appendix D AUC Figures We strengthen the findings our eICU experiments by extending evaluation to additional performance metrics. This section reports the findings of data addition on overall and subgroup AUC, defined in 3 . When sample sizes from minority groups are especially small, 43 recommend using metrics which consider the tradeoff between sensitivity and specificity, such as AUC. D.1 Results and Analysis Figure 24 shows the change in AUC after data addition compared to the base results. Observing the plots, we see a similar phenomenon as Figure 1 where the performance changes due to data addition are unequally distributed across subgroups. Most notably, Test Hospital 167 is sees improvements in Overall AUC in all addition cases, whereas the Black subpopulation exhibits significant declines in performance in all cases. In order to compute a meaningful AUC, both positive and negative samples must be included in the validation set. For splits where a subgroup did not have both classes represented, we leave the AUC values empty. In Figures 9 , 10 , and 11 , we find analogous results to Figures 2 , 3 , and 4 , respectively. In these experiments, we evaluate existing data interventions for fairness and find that they are unreliable in the scenario of OOD data addition. We do not find statistically significant relationships between the change in subgroup AUC with the change in subgroup samples, subgroup rate, or with the similarity of the added data source. Figure 9: Change in subgroup ratio vs. Change in subgroup test AUC on the eICU dataset. Same experiment conditions as were used to produce 2 . Figure 10: Number of subgroup samples added vs. Change in subgroup test AUC on the eICU dataset. Same experiment conditions as were used to produce 3 . Figure 11: Subgroup similarity score vs. Change in subgroup test AUC on the eICU dataset. Same experiment conditions as were used to produce 4 . Lastly, we reproduce Figures 6 and 1 looking at subgroup AUC, the results of which are shown in 12 and 13 , respectively. Although many samples are null, due to lack of positive samples in the validation set used for calibration, we still observe that performance gaps are generally positive. However, because the definition of calibration inherently includes accuracy, we do not observe as large of an effect on AUC as we did with accuracy. Figure 12: Difference in best-case subgroup AUC (without calibration) and worst-case subgroup AUC (with calibration) on the eICU dataset. Same experiment conditions as were used to produce 6 . Figure 13: Difference in best-case subgroup AUC (with calibration) and worst-case subgroup AUC (with calibration) on the eICU dataset. Same experiment conditions as were used to produce 1 . Appendix E Extended Model Classes This section shares the results and visualizations for all experiments conducted using Light Gradient Boosting Machine (LGBM) and Long Short-Term Memory (LSTM) models. The figures in the main paper are all showing the results using a Logistic Regression (LR) model. Each experiment was replicated on each model class. In general, it is observed that all model classes exhibit similar behavior under data addition. E.1 Light Gradient Boosting Machine (LGBM) Results The change in performance after Whole-Source data addition is shown in Figure 14 . We visualize the data addition combinations for all 12 hospitals used in the eICU Dataset. Results are shown for both accuracy and AUC at both the overall and subgroup levels. Figure 14: Change in performance after Whole-Source data addition using the Light Gradient Boosting Machine (LGBM) classifier on the eICU Dataset. We also analyze the results of naive data selection strategies—maximizing subgroup rate, samples added, and similarity score—in Figures 15 , 16 , and 17 , respectively. These are evaluated on the eICU dataset. We find that these existing data selection strategies are similarly ineffective when training an LGBM classifier. Figure 15: Change in Subgroup Rate vs. Change in Subgroup Accuracy after Whole-Source data addition, trained using an LGBM model. Figure 16: Number of subgroup samples added vs. Change in Subgroup Accuracy after Subgroup-Level data addition, trained using an LGBM model. Figure 17: Subgroup Similarity Score between the test subgroup and added subgroup vs. Change in Subgroup Accuracy after Subgroup-Level data addition, trained using an LGBM model. We compare data addition and calibration in 18 , finding that, in all cases, informed data selection outperforms calibration as a fairness intervention strategy. Finally, in 19 , we compare the combination of data selection and calibration on an LGBM classifier. Figure 18: Difference in best-case subgroup performance (without calibration) and worst-case subgroup performance (with calibration) after Whole-Source data addition. Performance is evaluated on the eICU dataset using the subgroup accuracy metric and trained using an LGBM classifier. Figure 19: Difference in best-case subgroup performance (with calibration) and worst-case subgroup performance (with calibration) after Whole-Source data addition. Performance is evaluated on the eICU dataset using the subgroup accuracy metric and trained using an LGBM classifier. E.2 LSTM Results The full set of results for Whole-Source data addition using the Long Short-Term Memory (LSTM) model are presented in 20 . In general, we find that the Overall accuracy is harmed by any out-of-distribution data addition, more so than is seen with the LR or LGBM models. This is not a necessarily unexpected result, as LSTMs rely more heavily on learned sequential dependencies and higher-order feature representations. Regardless, we still observe that the effects on Overall accuracy are not equally reflected across all subgroups. For example, adding data to Test Hospitals 338 and 199 are greatly helpful for improving Black subgroup accuracy despite minimal or negative changes to Overall accuracy. Similarly, we observe improvements to the Other subgroup of Hospital 443 under data addition, but minimal to negative differences among all other groups. Figure 20: Change in Overall and Subgroup Accuracy after Whole-Source data addition (LSTM). We applied the same three naive data addition methods as in Section 5 . The results are shown in Figures 21 , 22 , and 23 . Interestingly, we observe that all three naive methods—increasing subgroup rate, sample size, and selecting the most similar sources—were informative for improving subgroup accuracy on the Black subgroup only. In each, we find a statistically significant relationship for the Black subpopulation. However, as these results were not seen consistently across all subgroups and model classes, we do not find that these methods overall are reliable strategies for improving subgroup performance on EHR datasets. Figure 21: Change in Subgroup Rate vs. Change in Subgroup Accuracy after Whole-Source data addition, trained using an LSTM model. Figure 22: Number of subgroup samples added vs. Change in Subgroup Accuracy after Subgroup-Level data addition, trained using an LSTM model. Figure 23: Subgroup Similarity Score between the test subgroup and added subgroup vs. Change in Subgroup Accuracy after Subgroup-Level data addition, trained using an LSTM model. Figure 24: Change in Overall and Subgroup AUC by ethnicity group after Whole-Source data addition on the eICU dataset. Experiments are identical to the ones used to produce 1 .