Detecting dataset bias in medical AI using a generalized and modality agnostic auditing approach Authors: Nathan Drenkow, Mitchell Pavlak, Keith Harrigian, Ayah Zirikly, Adarsh Subbaswamy, Mohammad Mehdi Farhangi, Nicholas Petrick, Mathias Unberath Published: 2026 ABSTRACT AbstractDespite many success stories along the path of Artificial Intelligence’s (AI) rise in healthcare, there are comparably many reports of significant shortcomings and unexpected behavior of AI in deployment. A major risk is AI’s reliance on association-based learning, where non-representative machine learning datasets can amplify latent bias during training and hide it during testing. To unlock new tools capable of detecting such AI bias issues, we present Generalized Attribute Utility and Detectability-Induced bias Testing (G-AUDIT). G-AUDIT is a data modality-agnostic dataset auditing approach that automatically quantifies shortcut learning risks by examining the interplay between task-level annotations, sensor-level measurements, and patient, environmental, and acquisition characteristics. We demonstrate the broad applicability of this method by analyzing a range of medical datasets across three distinct modalities (images, text, and tabular data) and machine learning tasks, successfully identifying potential shortcuts commonly overlooked by traditional qualitative methods.Subject terms: Computer science, Health care Introduction The use of artificial intelligence (AI) and machine learning (ML) in healthcare has created numerous opportunities to improve quality, safety, efficiency, efficacy, affordability, and accessibility of healthcare through applications including disease detection1, forecasting2,3, opportunistic screening4, and quantitative medicine5. However, AI/ML models have repeatedly exhibited deterioration in performance and implicit biases when deployed6–12. Bias in this paper refers to dataset bias that contributes to the inability of an AI model to fully generalize due to misalignment between properties of the training dataset and the deployment population. Due to the data-driven nature of AI/ML methods, the brittleness and bias of these models is inherently tied to the data on which they are trained and evaluated prior to deployment. Bias in medical datasets may originate from a number of sources13,14 and the growing size of these datasets15–17 makes human assessment of the dataset as a whole intractable thus increasing the risk that latent biases may go undetected during data collection and subsequently exploited by downstream AI/ML models. The influence of dataset biases may manifest in a number of ways including shortcut learning10,18–23, bias and fairness issues in AI predictions24–28, and lack of generalization or robustness12. All of these issues are problematic, as they may result in harmful and unpredictable performance disparities of the model within and across patient populations in deployment. Dataset shortcuts 10,18–23 pose a particularly serious risk for AI models given that they are emergent properties of the dataset often arising from seemingly innocuous attributes in the data. In particular, these shortcuts manifest as features that are not causally-relevant to the task but may be exploited by AI/ML models to achieve low training error. At test time, changes in the occurrence of these features may result in incorrect predictions by the AI/ML model with potentially harmful outcomes to patients. For instance18, showed that the association between clinician markings and malignant skin lesions in dermascopic image datasets led to AI/ML models producing false positive rates that were 40% higher in the presence of markings at test time. Similarly22, found that AI/ML models were capable of exploiting laterality markers in chest X-ray images in order to reduce training error in detecting COVID-19. In these examples (and others), harmless features occurring naturally in the data (e.g., clinician markings, laterality indicators) become harmful when examined in the dataset as a whole. The best opportunity to address these data-driven AI/ML bias risks starts with exposing dataset-level risks prior to any model training or evaluation. Therefore, dataset audits, the focus of this work, may be important for identifying and quantifying model bias as early as possible before the training and evaluation phases. Complementary to our work, a large body of literature has directly examined the challenge of bias mitigation in AI/ML models. Popular methods29–31 aim to learn task-relevant representations while ignoring or forgetting features of the data that pertain to task-irrelevant, sensitive, or protected attributes. These methods were later applied in medical settings32–35 demonstrating the need for model bias mitigation and the challenges in successfully achieving it. While this remains an important area of research, we focus our attention on dataset audits. These audits aim to produce targeted hypotheses about how dataset composition may impact model bias in order to more directly inform model audits and guide bias mitigation strategies. Despite the importance of quantifying bias risk at the dataset-level, techniques for performing dataset audits remain fragmented and incomplete13. On the one hand, some methods may examine disparities in medical dataset metadata23,28,36 but they do not explicitly link these to the primary data with which AI/ML models are trained. In contrast, other methods show that patient attributes may be predictable from sensor-level measurements24,37 but may not fully identify whether the attribute-level features represent direct shortcuts for achieving accurate prediction. Some methods more directly examine the relation between data attributes and task model accuracy21,25,26,38 yet these do not enable assessment of the dataset-level risks independent of task-level modeling assumptions. Lastly, while most methods have focused on addressing bias with respect to conventional patient attributes (such as age, sex), some methods have begun to explore biases that may relate to other aspects of the data collection pipeline such as image acquisition or clinician-level markings10,39. In short, current dataset auditing methods lack practical utility because they do not holistically address both the risk of relevant attributes leaking information about the task and the risk of whether information about those attributes can be directly exploited during model training. Furthermore, existing auditing methods have been tailored to specific modalities or clinical tasks leading to a paucity of modality-agnostic methods for quantifying dataset-level bias risks. New automated dataset auditing techniques may be able to identify these primary risks and generate hypotheses to guide downstream model development, auditing, and mitigation. To address the limitations in quantitative dataset audits, we present a novel technique that represents a significant advance in enabling independent, dataset-level auditing and which we refer to as Generalized Attribute Utility and Detectability-Induced bias Testing (G-AUDIT) for datasets. The method presents a holistic approach to shortcut auditing by considering the interplay between attribute-level composition, sensor-level measurements, and task labels. G-AUDIT is generalizable and not tied to a particular data modality or ML task. The procedure (Fig. 1) utilizes the tools of information theory and causal inference to identify the presence and strength of association between attributes and task labels (utility) as well as the ability to infer the values of those attributes directly from the data itself (detectability). We then rank attributes according to their measured utility and detectability, thus providing AI/ML developers and clinicians alike a quantitative yet interpretable means of assessing dataset-level bias risk. The method is not limited to only patient-level attributes and allows for datasets to be audited with respect to any variables of relevance to the data collection process, patient population, and machine learning task. Overview of generalized attribute utility and detectability-induced bias testing (G-AUDIT). Given a dataset consisting of input with associated labels and metadata, we perform the Generalized Attribute Utility and Detectability-Induced bias Testing (G-AUDIT) to generate quantitative hypotheses on the relative risk of attributes in the form of a Detectability vs Utility scatter plot (right). We apply the G-AUDIT method to a range of clinical tasks covering image, text, and tabular data modalities. In each dataset, audits uncover shortcut risks (confirmed by domain experts) linked to non-patient attributes representing aspects of the data collection pipeline not typically considered as candidates for bias. We also provide an optional calibration procedure which allows for estimating a form of worst case performance degradation given the observed values of utility for attributes in the dataset. For the potential shortcuts flagged by the auditing method, we find that the estimated worst case degradation is substantial (often > 0.2 Area Under the Curve difference) indicating that models exploiting these shortcuts may stand to exhibit significant drops in performance in deployment. By applying G-AUDIT across clinical tasks and data modalities, the results from this study underscore and address the need for new methods to quantitatively assess dataset bias risk across all aspects of the data collection process. Results To demonstrate its broad effectiveness, the G-AUDIT procedure is applied to datasets from three different modalities, namely, image, text, and tabular data. For each dataset, we estimate the utility and detectability (see Methods) of each attribute relative to the underlying learning task. We do this to generate hypotheses that identify and rank potential shortcut attributes according to the risk that downstream models may exploit them. We also include results from an optional calibration procedure in which an approximate upper bound on the drop in performance-related metrics can be calculated for each potential shortcut using a synthetic attribute in precisely controlled conditions (see Bounding Performance Risk). The bound provides a means of estimating a form of worst-case downstream model performance risks for specific attributes in more familiar metrics. Skin lesion classification We first consider shortcut risks in vision-based tasks. The high dimensionality of image data creates many opportunities for shortcuts to exist without directly impacting the machine learning task. For instance, hospital-specific tokens placed in the chest X-ray field of view (e.g., ref. 40) may impact only a small number of pixels in the image but may constitute a shortcut when associated primarily with a specific disease condition in the training dataset. In cases like these, deep neural networks (DNNs) trained on the data may exploit salient features like tokens/watermarks (or other statistical regularities not task-related) to achieve low training error. We focus here on skin lesion classification where the construction of machine learning datasets may not be able to adequately balance across patient characteristics, clinical sites, and dermascopic imaging sensors/settings and where such features of the dataset and collection process may manifest as shortcuts. The ISIC 2019 skin lesion dataset41–43 was analyzed for bias with respect to the included metadata and patient attributes. The dataset consists of 25,331 training images and attributes age, race, sex, anatomical location, and skin color on the Fitzpatrick scale. Image metadata includes height, width, and year of collection. While the original task labels included seven diagnostic categories, we reduce the classification task to malignant vs. benign conditions (e.g., refs. 44–46) The data auditing procedure (see Methods) was applied to the entire training dataset relative to the binary malignancy classification task. Images with any missing attribute values were excluded from the analysis. The main auditing results are found in Fig. 2 and Table 1 where the height, width, and year attributes exhibit the highest combination of utility and detectability, indicating these attributes are more likely sources of bias within the dataset. While these particular attributes may seem unrelated to the task labels, this image metadata can act as a proxy for the camera type and/or clinical site. Furthermore, while all images are resized and cropped to a fixed resolution prior to running the auditing procedure, the high detectability scores indicate that some of this information is still retained in the images themselves, either directly or via proxy. G-AUDIT results and approximate bound on worst-case performance for ISIC 2019. a Attribute Detectability vs. Utility for the ISIC 2019 dataset. Confidence intervals calculated via basic bootstrap 77. b Approximate worst-case performance risk for a synthetic attribute with varying utility (see Bounding Performance Risk for more detail). Confidence intervals are derived from a t-distribution based on model performance across three folds; vertical lines indicate observed values of utility and indicate a worse-case drop in AUC of ≈ 0.2. Table 1G-AUDIT and baselines applied to ISIC 2019 datasetG-AUDIT (ours)BaselinesAttributeUtilityDetectabilityDetection F1SPLIT F1Year0.0520.8860.9660.981Image Height0.0500.8780.8900.948Image Width0.0480.8660.5210.583Age0.0350.1160.2940.334Anatomical Location0.0120.2970.5670.624Sex0.0030.1610.7310.768Skin Color (Fitzpatrick)0.0000.4620.5280.632Comparison between Utility and Detectability across all available dataset attributes. Also included are macro-averaged F1 scores from the detection model and conducting the SPLIT procedure24,37 on trained task models (see Sec. 1). Note that only the relative rankings of attributes are comparable between utility, detectability, and F1 scores. Only the F1 scores of the baselines are comparable by their respective values. G-AUDIT and baselines applied to ISIC 2019 dataset Comparison between Utility and Detectability across all available dataset attributes. Also included are macro-averaged F1 scores from the detection model and conducting the SPLIT procedure24,37 on trained task models (see Sec. 1). Note that only the relative rankings of attributes are comparable between utility, detectability, and F1 scores. Only the F1 scores of the baselines are comparable by their respective values. To test the effect of making incorrect assumptions about the direction of dependency between images X and labels Y, we run G-AUDIT under the (incorrect) assumption that X → Y. In this causal case, we do not condition on the label Y in the process of estimating detectability. The results of this experiment are seen in Fig. 3. While the choice to condition on Y in the anti-causal case (or not in the causal scenario) is theoretically sound, we see that making the incorrect causal assumption has little empirical effect in this ISIC audit. This result gives additional confidence in using utility and detectability to assess risks associated with each attribute in the dataset. G-AUDIT sensitivity to causal assumptions for ISIC. The audit procedure for estimating detectability was run on the ISIC data under both the causal and anti-causal scenario. Differences in detectability are attributed to the effect of conditioning on Y in the anti-causal scenario but not in the causal case. Since choices of architecture for the surrogate attribute prediction models could have an effect on the estimate of detectability, we ran an additional sensitivity analysis of this modeling choice. In addition to the ResNet-18 architecture used in the prior results, we also estimated detectability using ResNet-34, ResNet-50, Swin-T, and Swin-B models as shown in Fig. 4. While the results indicate differences in the estimates of detectability, the differences are minor and the relative ranking of attributes by detectability remains stable. These results provide clear evidence that shortcut risks identified by G-AUDIT on this ISIC dataset are relatively insensitive to modeling choices. G-AUDIT sensitivity to surrogate model architecture. Comparison of utility vs. detectability when changing the model architecture used for attribute prediction in estimating detectability. Differences in detectability values are observed, but relative rankings by detectability are preserved across architectures. Stigmatizing language in electronic health records We next consider the text domain where the dimensionality of the data remains high and the variability of natural language create unique opportunities for shortcuts to exist. Similarly to the image domain, DNNs are often used to learn compact representations of natural language text for various tasks and may exploit shortcuts in the data that are not relevant to the clinical task yet allow them to achieve low training error. We apply the G-AUDIT procedure to the electronic medical record dataset introduced by ref. 47 for the purpose of characterizing stigmatizing language usage by physicians. The dataset contains 5201 annotated instances across 3 tasks, with each task focusing on a different thematic group of stigmatizing language — Credibility & Obstinacy, Compliance, and Descriptions of Appearance/Demeanor. Models are provided a window of text centered around a keyword or phrase which has been identified by domain experts as being a potential indicator of unconscious bias towards a patient. The models are asked to characterize the implication of the input instance (e.g., “the patient claims to brush their teeth 2x daily”; “unable to track down insurance claims”). Each instance is associated with auxiliary attributes which indicate a patient’s race and sex, and the clinical setting from which the statement was drawn (e.g., OB-Gyn, Surgery). The stigmatizing language dataset serves as an interesting case study for several reasons. First, the authors of ref. 47 included an analysis which examined how well each attribute could be predicted based on the last embedding layer of the primary stigmatizing language task models. Their results provide a direct reference for G-AUDIT. Second, as is often true in practice, the stigmatizing language dataset has an ambiguous causal structure. This allows us to evaluate the robustness of G-AUDIT to potential errors in misspecification of the direction of dependency between attributes and labels. Finally, there are documented disparities in the prevalence of stigmatizing language between demographic groups which we expect to show up directly within the utility measure. To gather an unbiased estimate of the prevalence of stigmatizing language in the population, models should not make use of demographics as a predictive shortcut. As shown in Fig. 5, clinical specialty had a higher utility than both patient race and sex for the compliance and appearance/demeanor tasks. Prior work shows that downstream models for these tasks likely did not encode sex and race characteristics beyond what could be explained by a reliance on clinical specialty alone47. This is consistent with the observation that clinical specialty had a higher utility than race and sex, implying a stronger association with the task label. Importantly, this does not preclude performance disparities based on these sensitive attributes—clinical specialties in the JHM dataset include OB-GYN with an all-female patient population and Pediatrics with an approximately 95% Black patient population47. Instead, the results suggest that downstream models are more likely to exploit shortcuts related to the identification of different clinical domains than to directly encode race or sex to improve performance. G-AUDIT results for stigmatizing language in EHR data. Utility vs. detectability for three identical attributes across three detecting stigmatizing language in EHR (a compliance, b appearance and demeanor, c credibility and obstinacy) when calculating detectability for the (unconditioned) X → Y and (conditioned) Y → X cases. 95% confidence intervals calculated via basic bootstrap77. In terms of detectability, we find differences between conditioned and unconditioned measures to be fairly small. The differences under both the causal and anti-causal assumptions indicate the relative shortcut risks across attributes are stable. While knowledge of the correct causal assumption may permit a single theoretically correct approach to estimating detectability, we find empirically that the differences between the causal and anti-causal assumptions are sufficiently small in the EHR data to allow us to still draw conclusions about attribute risks in the dataset. While we see similarity in results between causal and anti-causal assumptions here, we note that this is not always the case. Other instances in healthcare settings48–51 have found that estimates of risk may become skewed when incorrect assumptions are made. We raise this point to emphasize the importance of making correct causal assumptions and testing both hypotheses (as done here) when the direction of causality is not certain. Finally, within the Credibility and Obstinacy task, while all attributes had relatively low utility, the detectability of sex was higher than that of any other EHR task and attribute evaluated. This presents a potential explanation for the findings of ref. 47, which identified sex within the Credibility and Obstinacy task as the only demographic attribute recoverable above baseline levels across all three tasks. Mortality prediction with intensive care unit tabular data Lastly, we consider the dataset auditing problem in the context of tabular data. While not as feature rich or high dimensional as images and text, tabular data presents its own unique challenges from an auditing perspective. While the features learned by task models in the image and text domains often lack interpretability (e.g., DNN embeddings), tabular data provides a direct mapping between attributes and their measured values. In these cases, attributes used as inputs to a task model can still act as shortcuts when they exhibit undesirable associations with the task labels that could arise due to sample bias or incorrect usage of the attribute itself. For instance, if clinical sites differ in their test-ordering protocols, associations between disease conditions and clinical site may be reflected in the test orders/results provided to machine learning task models52. Furthermore, the gap in task performance between DNNs and more traditional ML models (e.g., SVMs, ensemble methods, etc.) is much smaller in the tabular case and this provides an opportunity to measure both detectability and task model performance over a wider class of machine learning models. Here, we evaluated the G-AUDIT methodology on tabular data extracted from the publicly available Medical Information Mart for Intensive Care (MIMIC) III Dataset17. Specifically, we extracted a dataset for predicting mortality in Intensive Care Unit (ICU) patients using features involved in the Simplified Acute Physiology II (SAPS II) score53,54, a score used to measure disease severity in patients after their first 24 h in the ICU. These features include patient age as well as summary statistics of heart rate, systolic blood pressure, temperature, labwork indicators/results, ventilator usage, and Glasgow Coma Scale (GCS) during the first 24 hours of the patient’s ICU stay. For continuous variables (e.g., age), we discretize the data using the Freedman-Diaconis rule55 to select the bin width that best matches the resulting distribution to the theoretical data distribution. Our task is to predict patient mortality based on the tabularized data of a given patient. The final processed dataset consists of 34,386 patient records and 40 features, detailed in full in Supplement S5.3. Features are one-hot encoded and include medication details, missing variables (e.g., GCS or lab test results), and patient demographics such as race and insurance coverage. We select an FT-Transformer 56 with default hyperparameters for use as a task model in all experiments. A benefit of this selection is the ability to apply the SPLIT approach24,37 to empirically test how well the G-AUDIT detectability measure corresponds with other baseline approaches for measuring detectability24,37. As shown in Fig. 6, we find that, across model classes, G-AUDIT-based detectability correlates strongly with SPLIT-measured recoverability, with Spearman’s ρ values of 0.92, 0.92, 0.76, 0.84, 0.75, and 0.86 for decision tree, random forest, logistic regression, FT-Transformer, XGBoost, and naive Bayes task models, respectively (all p < . 0001). Recoverability of attributes via SPLIT for ICU mortality prediction data. We order each attribute from least to greatest recoverability in a downstream FT-Transformer task model as measured by SPLIT24,37. By doing this, we see that attributes that are more recoverable tend to have higher detectability values, though there is variance model to model. Additionally, we find that a large proportion of the most detectable potential shortcuts consist of missing values. Importantly, both obvious shortcuts such as ’temperature missing’—which is easily detectable because it is explicitly represented in the input data as a negative placeholder value for temperature—as well as less obvious instances like dopamine, norepinephrine, vasopressin, ventilator and non-invasive ventilator usage are highly detectable. We do note some variability in performance that does not always seem correlated with model strength as measured in Fig. 7. This suggests using a small ensemble of models of varying complexity may provide a more holistic view of detectability; though, early results do indicate this variability can be reduced by avoiding over-binning in estimation steps (see Supplement S2). Task model performance for ICU mortality prediction data. We first calibrate expectations by exploring the unmodified performance of various models on the task of predicting mortality. By running our synthetic calibration process ((Bounding Performance Risk), we identify that some of the attributes in our set may cause worst case performance drops as great as 0.2 AUC for an average task model assuming full detectability (see Fig. 8). Approximate bound on worst-case performance for ICU mortality prediction data. Following the calibration procedure, we can explore the worst-case impact a completely detectable shortcut has on downstream models. Dotted vertical lines correspond to the utility values of real attributes in the dataset. Blue lines correspond to test sets where the shortcut’s relationship to the label holds true; red correspond to the worst-case counterfactual set. Finally, review of a detectability vs utility plot in Fig. 9 makes clear that intervention-related attributes tend to have the highest utility while missing data entries have highest observed detectability. Interestingly, Medicare and private insurance attributes had non-zero utility and non-negligible detectability. Full, attribute-specific data is available in Supplement S5.3. G-Audit results for ICU Mortality Prediction data. Attribute Detectability vs. Utility for the tabularized MIMIC III dataset17. Detectability measured using decision tree surrogate models. Confidence intervals calculated via basic bootstrap77. Discussion The G-AUDIT provides a procedural means to identify and quantify potential sources of dataset bias and machine learning shortcuts. The technique identifies potentially harmful relationships between attributes and task labels that may be exploited via the input data. G-AUDIT estimates the presence and strength of these relationships through attribute utility and detectability. Attributes with both high-utility and high-detectability are indicative of dataset bias and act as primary targets for downstream algorithmic auditing. G-AUDIT provides a new approach to fully quantitative measurement of dataset bias that expands beyond auditing typical clinical factors to also include non-clinical attributes (e.g., imaging factors). G-AUDIT also contrasts from model-centric approaches that train task models on the data and then examine disparities in performance across subgroups. While these other methods may uncover some degree of shortcut risk, they are unable to decouple the relationship between the model and data, so we cannot be sure of the source of the performance disparity. Prior to this method, the closest points of comparison for dataset auditing consider only patient attributes (such as age or sex) and overlook other aspects of the dataset which may be a stronger source of bias (such as the collection site or model of imaging device). In fact, the analysis from this study finds that non-patient attributes are often of greatest concern which is consistent with previous studies that focused on specific shortcut learning scenarios such as the use of digital watermarks in x-ray images22 or presence of chest drains for detecting pneumothorax23. While these cases were found through manual inspection by researchers, the G-AUDIT procedure automatically and more efficiently enables detection of these possible sources of bias prior to model training. A limitation of G-AUDIT is its constraint to auditing only the metadata attributes available in the training/validation dataset. This may be a limiting factor for some research and public datasets where the available metadata is often limited because of patient privacy concerns. However, in the development of AI/ML models for use in critical healthcare settings, in-depth knowledge of the underlying datasets is likely necessary for understanding the accuracy and usefulness of a AI-enabled device validation57. Data management will likely include rich metadata associated with patient characteristics, environmental factors, and data acquisition settings. In those cases, G-AUDIT is primed to enable comprehensive, streamlined auditing of the relevant datasets. Since G-AUDIT is implicitly tied to measurable and interpretable attributes of the data, clinicians and AI developers are likely better equipped to interpret the results of the audit and identify relevant courses of action (e.g., bias mitigation and/or model auditing strategies). In this way, G-AUDIT is complementary to on-going model auditing and debiasing research, including slice discovery methods such as Domino and FACTS which can be used to discover unlabeled attributes for our approach to audit58,59. While this manuscript remained focused on demonstrating the efficacy of quantitative dataset audits for uncovering shortcut risks, this work opens up new avenues for future research to translate these detected risks into robust AI models. We show the generality of G-AUDIT by applying it across multiple machine learning tasks and data modalities. G-AUDIT is shown to be effective in both causal and anti-causal scenarios where the relationship between the labels and data may be reversed. G-AUDIT runs easily for both scenarios, additional knowledge or a priori assumptions about the directionality may be leveraged to improve the overall efficiency. Future work will consider augmenting G-AUDIT to also include automated detection of label-data directionality. While G-AUDIT is guaranteed to examine the metadata from every sample in the dataset, as the size of the audited datasets or the number of attributes increases, additional considerations will be necessary to ensure that G-AUDIT remains computationally viable. Nonetheless, unlike algorithmic audits where the procedure needs to be run for each new task model, G-AUDIT can be run once per dataset-task combination and adding new tasks only translates into re-calculating utility and detectability given the original attribute predictions and new task labels. In this work, we provide a generalized, quantitative technique for generating hypotheses related to the non-representativeness of a dataset to inform downstream model training and auditing. As adoption of data-driven machine learning methods for safety- and cost-critical medical applications continues to grow, principled quantitative methods for analyzing the underlying training and evaluation data may be important tools for highlighting potential limitations of the data. G-AUDIT provides the first such approach and results demonstrate that commonly overlooked dataset attributes may induce biased AI outputs that negatively impact the diagnosis and treatment of underlying patient conditions. This method provides a positive step towards identifying and mitigating these risks. Methods Utility and detectability The core objective of this data auditing procedure is to identify and quantify the degree to which each attribute of the data represents a potential learning shortcut. For this study, datasets consist of samples (X), associated metadata/attributes (A), and task labels (Y). We establish the existence (not direction) of a relationship between A ↔ Y through a measure we call utility. The utility of an attribute refers to our ability to infer the value of the task label Y simply by observing A. In the extreme case, if A is perfectly correlated with Y, then machine learning models simply need to detect A in order to correctly solve the task. However, a large value for an attribute’s utility is typically not sufficient to consider it a shortcut risk. For this, knowing the attribute’s detectability is critical. Since machine learning models will typically only take X (but not A) as input, detectability measures the extent to which the value of A can be inferred from X. Attributes with high-utility and high-detectability represent the greatest risk for biasing downstream task models. However, while we use utility and detectability as a proxy for risk, features that are causally relevant to the task and have high-utility and -detectability are useful, rather than representing possible shortcuts. Even in the case of clinically-relevant attributes in the dataset, these attributes can become shortcut risks when their relative association with the task labels and raw data becomes stronger than in common practice. For instance, it is understood that anatomical location is often associated with the malignancy of skin lesions43. However, in an extreme case where malignant lesions are only in images collected from specific anatomical locations, the use of location information becomes a shortcut. In tabular sepsis prediction tasks, administration of an intervention could provide useful signal for mortality but negatively affect the clinical usefulness of a naively implemented risk score60. This illustrates that the task of dataset auditing to assess shortcut risks is more nuanced than predicting whether a given attribute is a spurious or invariant feature. Therefore, G-AUDIT provides a useful ranking of attribute risks so that downstream domain expertise may then determine whether high-risk attributes are reasonable features or unanticipated dataset flaws. Measuring utility and detectability We use information theory and principles of causal inference to measure the utility and detectability of dataset attributes. Utility is measured as the mutual information, MI(A; Y) = H(Y) − H(Y∣A). It represents the reduction in uncertainty about the value of Y by observing A while accounting for the entropy of the underlying distributions and adjusting for chance based on the number of categories (as in refs. 61,62) We rely on the faithfulness assumption which implies that if there is a relationship between A, Y, then MI(A; Y) > 0. For detectability, the objective is to determine the ability to infer the value A from the data and we accomplish this by first training a surrogate model f: A → X to predict A from X. We next estimate the utility and detectability between data and labels for both possible directions of dependency between data and labels. In many cases, clinician and domain expertise may allow us to eliminate certain directions of dependency thus improving the interpretability of the results and reducing overall computation. Our ultimate goal is to identify whether there is a relationship A ↔ X and if so measure its strength. We consider the direction of dependence between data and labels in order to control for the potential of information leaking through Y and this process relies on the assumed/known relationship between Y, X. We will show that each case permits a theoretically correct solution regarding how to account for Y in estimating detectability. Assuming an incorrect direction can theoretically result in estimates of detectability that are misaligned with reality (although our empirical results show the difference can be small). We provide a simple example in Supplement S1 to further illustrate the risks of making incorrect assumptions and the benefit of testing both hypotheses for the directional relationship between X, Y. In severe cases, these deviations may lead researchers to draw incorrect conclusions about the relative risks of attribute-induced shortcuts (potentially missing high-risk instances). Nonetheless, in cases where the direction of dependence between X, Y is not known and cannot be assumed with high confidence, G-AUDIT can be run once for each hypothesis for the direction of dependence. The results of these two runs will allow researchers to compare how estimates of detectability change as well as the impact these changes might have on estimates of overall attribute risks. In anti-causal scenarios (i.e., Y → X), we know it could be the case that A → Y → X and as a result information could be leaked about the attribute into X not by a shortcut, but by solving the task itself. To control for this, we condition on Y during the training process of our surrogate attribute prediction model f:X→A^. Specifically, we assume either Y is already discrete or define YD as a sufficiently granular discretization of Y. Then, we partition X, YD into disjoint training subsets S such that for a given subset Si, all yDj∈Si have the same value. For each subset Si ∈ S, we train separate surrogates fi(X)=A^. By training separate surrogates in this manner, we ensure that differences in task labels Y that could be used by fi to predict a given attribute are minimized. Based on test set predictions obtained using cross-validation with all fi, we obtain a full set of predictions (A^) for the entire dataset and we similarly measure MI(A;A^) to understand how recoverable A is from X while reducing the impact of task relevant information. In the causal case (i.e., X → Y), we are able to directly estimate the relationship without conditioning. In fact, we cannot condition on Y since it represents a collider (i.e., Y may be dependent on both X and A). In that case, measuring the strength of relationship between X ↔ A given Y could give falsely inflated results since conditioning on a collider Y creates an otherwise non-existent association between its parents (X, A). For example, if two conditions both increase mortality rate through different pathways, then given that a patient has died, knowledge of either attribute provides information about the other (e.g., given A is the presence or absence of trauma wounds and X contains information relating to disease, a patient who died but did not have trauma wounds is more likely to have had disease: A ↔ X∣Y even though A⊥X). As a result, we do not condition for cases where we expect X → Y and instead directly estimate MI(A;A^). Depending on the data types of the label and variables to be audited, different estimators of mutual information can be chosen. Given the low dimensional nature of the problem, we prefer simple binning approaches as compared by Chaves et al. (2024). Importantly, for fair comparison across variables with potentially many categories, this adjustment for random chance is critical62. We provide a detailed comparison of estimators and preferences in Supplement S2. Data audit procedure A key aspect of the data auditing procedure is that every sample in the dataset contributes to the calculation of attribute utility and detectability. For determining utility, only the labels and metadata are used and can be estimated without any model training. However, detectability is reliant on attribute prediction, so we ensure that every sample in the dataset contributes to the detectability estimate using a cross-validation procedure that ensures unbiased predictions of A^. We start by partitioning the full dataset into K disjoint folds. In the anti-causal case (see previous section), we partition the data by label first before subsequently partitioning into the K disjoint folds (per label). For each fold, we hold out the data of that fold for testing and use the data from the K − 1 remaining folds to train or finetune a sufficiently expressive machine learning model to predict A^ from X. Given that trained model, we predict A^ on the held-out fold’s data. After repeating the train/test procedure for each fold, and each value in YD (as necessary in the anti-causal scenario due to the interdependency between attribute, data, and label), we aggregate all predictions together to compute detectability measures. Bounding performance risk While utility and detectability measures provide a quantitative measure of bias, how to interpret them in the context of potential model performance risks is less clear. The magnitudes of these measures are not easily comparable across datasets so we cannot rely on thresholds or conventions to directly assess risk. We could instead look to the relative rankings of attributes with respect to utility/detectability to understand attribute risk ordering, but cannot directly translate values to drops in task-relevant metrics like AUC. To address this concern, we provide a supplementary method for generating an approximate upper bound on performance risk with which attribute utility may be interpreted. To construct the bound, we first create a synthetic attribute which we can insert with 100% detectability (e.g., a visible watermark in a fixed image position, a single token added to a text input, an added column to tabular data). We then vary the utility of this synthetic attribute and train a task model for each case. We evaluate the task models on synthetic and counterfactual data distributions and measure the resulting performance drop between the two distributions. The counterfactual data distribution is constructed in such a way as to create the worst case scenario where the synthetic attribute is anti-correlated with the true label and any shortcut exploited during training will yield worst-case behavior at test time. Formally, let X be the feature data, Y be the true binary task labels (of length NY), and A be the synthetic artifact values. For simplicity, we assume A is also binary and initialize the values of A to have the same values as Y in the dataset (i.e., the normalized utility is 1.0 when A = Y). To vary the utility, we randomly select an index set I for N