DPA: A one-stop metric to measure bias amplification in classification datasets

Bhanu Tokas*, Rahul Nair*, Hannah Kerner · 2024

Read DPA: A one-stop metric to measure bias amplification in classification datasets by Bhanu Tokas*, Rahul Nair*, Hannah Kerner (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-11060.txt.

Introduction

Topolnicki, Skibiński, and Golak present a 2024 method for sizing the error bars on nuclear theory results. Few-nucleon calculations rest on potentials with many knobs, and tracking how each knob's error flows to the answer is costly. Their trick borrows backpropagation, the gradient engine of neural networks, which shines when inputs are many and outputs few.

They test the method on the deuteron, the simplest nucleus, and on nucleon scattering. Gradients from a standard Python library carry parameter errors to binding energy and phase shifts. A brute-force sampling check gives near the same answers, which validates the shortcut. The tools let nuclear theorists trace which parts of a potential drive the total uncertainty.

Catalog record

CREATORBhanu Tokas*, Rahul Nair*, Hannah Kerner
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-11060

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DPA: A one-stop metric to measure bias amplification in classification datasets Bhanu Tokas*, Rahul Nair*, Hannah Kerner Abstract Most ML datasets today contain biases. When we train models on these datasets, they often not only learn these biases but can worsen them — a phenomenon known as bias amplification. Several co-occurrence-based metrics have been proposed to measure bias amplification in classification datasets. They measure bias amplification between a protected attribute (e.g., gender) and a task (e.g., cooking). These metrics also support fine-grained bias analysis by identifying the direction in which a model amplifies biases. However, co-occurrence-based metrics have limitations — some fail to measure bias amplification in balanced datasets, while others fail to measure negative bias amplification. To solve these issues, recent work proposed a predictability-based metric called leakage amplification (LA). However, LA cannot identify the direction in which a model amplifies biases. We propose Directional Predictability Amplification (DPA), a predictability-based metric that is (1) directional, (2) works with balanced and unbalanced datasets, and (3) correctly identifies positive and negative bias amplification. DPA eliminates the need to evaluate models on multiple metrics to verify these three aspects. DPA also improves over prior predictability-based metrics like LA: it is less sensitive to the choice of attacker function (a hyperparameter in predictability-based metrics), reports scores within a bounded range, and accounts for dataset bias by measuring relative changes in predictability. Our experiments on well-known datasets like COMPAS (a tabular dataset), COCO, and ImSitu (image datasets) show that DPA is the most reliable metric to measure bias amplification in classification problems. 1 Introduction Machine learning models should perform fairly across demographics, genders, and other groups. However, ensuring fairness is challenging when training datasets are biased, as is the case with many datasets. For instance, in the ImSitu dataset ( Yatskar et al., 2016 ) , 67\% of the images labeled “cooking” feature females, indicating a gender bias that women are more likely to be associated with cooking than men Zhao et al. (2017) . Given a biased training set, it is not surprising for a model to learn these dataset biases. Surprisingly, models not only learn dataset biases but can also amplify them Zhao et al. (2017) ; Wang and Russakovsky (2021) ; Zhao et al. (2023) ; Subramonian et al. (2024) ; Wang et al. (2024) . In the example from ImSitu, where females and cooking co-occurred 67\% of the time, bias amplification occurs when >67\% of the images predicted as cooking feature females. Several co-occurrence-based metrics ( BA_{\rightarrow} , Multi_{\rightarrow} , BA_{MALS} ) have been proposed to measure bias amplification in classification datasets. They measure the bias amplification between a protected attribute (e.g., gender), denoted as A , and a task (e.g., cooking), denoted as T Zhao et al. (2017) ; Wang and Russakovsky (2021) ; Zhao et al. (2023) . If A and T co-occur more than random in the training dataset, these metrics measure how much more they co-occur in the model’s predictions. For instance, if the co-occurrence between females ( A ) and cooking ( T ) is 67\% in the training dataset and 90\% at test time, the bias amplification value is 23\% . Co-occurrence-based metrics help with a fine-grained bias analysis as they can identify the direction in which a model amplified biases. In the cooking example from ImSitu, metrics like BA_{\rightarrow} and Multi_{\rightarrow} can measure how much a model amplified the bias towards predicting women as cooking ( A\rightarrow T ) and towards predicting cooks as women ( T\rightarrow A ). Such disentanglement of bias amplification is important, as it helps practitioners come up with targeted bias intervention efforts. However, co-occurrence-based metrics have a limitation: they cannot measure bias amplification if A is balanced with T . Metrics like BA_{\rightarrow} and BA_{MALS} assume that if attributes and tasks are balanced in the training dataset, there are no dataset biases to amplify. Previous works have shown that simply balancing attributes and tasks does not ensure an unbiased dataset. Biases may emerge from parts of the dataset that are not annotated Wang et al. (2019) . Suppose we balance imSitu such that 50\% of the images labeled “cooking” feature females. Now, assume that cooking objects in ImSitu, like hairnets, are not annotated. If most of the cooking images with females have hairnets, while most of the cooking images with males do not, the model may learn a spurious correlation between hairnets, cooking, and females. Hence, the model may more often predict the presence of a female when cooking images have hairnets in the test set, leading to bias amplification between females and cooking. However, since gender appears balanced with respect to the cooking labels, BA_{\rightarrow} and BA_{MALS} would report 0 bias amplification. While other co-occurrence metrics like Multi_{\rightarrow} may report non-zero bias amplification for balanced datasets, these values are often misleading, as Multi_{\rightarrow} cannot capture negative bias amplification scenarios. To correctly measure bias amplification in balanced datasets (by accounting for biases from unannotated elements), Wang et al. Wang et al. (2019) proposed a predictability-based metric called leakage. They defined dataset leakage ( \lambda^{\text{D}} ), which refers to how well attribute A can be predicted from true task labels T , and model leakage ( \lambda^{\text{M}} ), which refers to how well A can be predicted from a model’s task predictions \hat{T} . They defined bias amplification as \lambda^{\text{M}} - \lambda^{\text{D}} . \lambda^{\text{D}} and \lambda^{\text{M}} are measured using a separate attacker function trained to predict A . In this work, we refer to Wang et al.’s method of calculating bias amplification as leakage amplification ( LA ). Since LA focuses on predictability, it can measure bias amplification in balanced and unbalanced datasets. However, it cannot identify the direction in which a model amplifies biases ( A\rightarrow T and T\rightarrow A disentanglement is not possible with LA ). We do not have one metric that can ( 1 ) correctly report positive and negative bias amplification scenarios, ( 2 ) accurately measure bias amplification for both balanced and unbalanced datasets, and ( 3 ) identify the direction in which a model amplifies biases. Practitioners need to evaluate their models on multiple bias amplification metrics to assess these properties, making the process tedious. We propose Directional Predictability Amplification ( DPA ), a one-stop predictability-based metric that addresses all these properties of bias amplification. In addition to being a one-stop metric, DPA addresses several issues found in earlier predictability-based metrics like LA , including issues such as reporting unbounded bias amplification values, measuring absolute change in predictability instead of relative change, and being highly sensitive to attacker function. Our key contributions: (1) DPA is the only directional metric that can accurately identify positive and negative bias amplification in both balanced and unbalanced classification datasets. (2) DPA reports bias amplification values within a bounded range of [-1,1] , enabling easy comparison across different models (3) DPA accounts for the original bias in the dataset as it measures relative change in predictability (instead of absolute change). (4) DPA is minimally sensitive to the choice of attacker function. This eliminates the pain of choosing a suitable attacker to measure bias amplification. 2 Related Work Co-occurrence for Bias Amplification Men Also Like Shopping ( BA_{MALS}

Read the complete text