Identifying Bias in Deep Neural Networks Using Image Transforms

Sai Teja Erukude, Akhil Joshi, Lior Shamir · 2024

Read Identifying Bias in Deep Neural Networks Using Image Transforms by Sai Teja Erukude, Akhil Joshi, Lior Shamir (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-13079.txt.

Introduction

Sai Teja Erukude and co-authors show how to unmask hidden bias in image datasets with simple picture transforms in a 2024 paper. Image networks act as black boxes, and slanted test sets can flatter their scores. A classic tell is background leakage, where models learn scenery rather than the object itself.

When no clean background crop exists, the team applies Fourier and wavelet transforms and median filters to the full image. These views recover the background signal a network may secretly ride on. Checked across common benchmark sets, the method flags biases that plain accuracy hides. The authors urge bias hunts before any score is believed.

Catalog record

CREATORSai Teja Erukude, Akhil Joshi, Lior Shamir
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-13079

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Identifying Bias in Deep Neural Networks Using Image Transforms Sai Teja Erukude, Akhil Joshi, Lior Shamir Abstract CNNs have become one of the most commonly used computational tool in the past two decades. One of the primary downsides of CNNs is that they work as a “black box”, where the user cannot necessarily know how the image data are analyzed, and therefore needs to rely on empirical evaluation to test the efficacy of a trained CNN. This can lead to hidden biases that affect the performance evaluation of neural networks, but are difficult to identify. Here we discuss examples of such hidden biases in common and widely used benchmark datasets, and propose techniques for identifying dataset biases that can affect the standard performance evaluation metrics. One effective approach to identify dataset bias is to perform image classification by using merely blank background parts of the original images. However, in some situations a blank background in the images is not available, making it more difficult to separate foreground or contextual information from the bias. To overcome this, we propose a method to identify dataset bias without the need to crop background information from the images. That method is based on applying several image transforms to the original images, including Fourier transform, wavelet transforms, median filter, and their combinations. These transforms were applied to recover background bias information that CNNs use to classify images. This transformations affect the contextual visual information in a different manner than it affects the systemic background bias. Therefore, the method can distinguish between contextual information and the bias, and alert on the presence of background bias even without the need to separate sub-images parts from the blank background of the original images. Code used in the experiments is publicly available. 1 Introduction In the past decade, CNNs have revolutionized the field of computer vision due to their unprecedented performance capabilities with regard to image analysis, combined with ease-of-use due to the availability of libraries. These networks use automatic extraction and selection of data-driven abstract features, which means a substantial reduction in the need for manual feature engineering. CNNs are currently widely used in a variety of applications, including object detection, recognition, medical imaging, and many more. With the substantial advantages, CNNs also have notable limitations. For example, training CNNs is computationally expensive, normally involving the need for advanced GPUs or computing clusters for execution. Training CNNs often requires a large number of samples ( Uchida et al., 2016 ) , and overfitting is a common situation when using CNNs ( Santos and Papa, 2022 ; Thanapol et al., 2020 ; Gavrilov et al., 2018 ; Sikha and Benitez, 2024 ) . The need for large training sets leads to the common use of data augmentation, which can in itself affect the bias in a controlled or uncontrolled manner ( Pastaltzidis et al., 2022 ; Babul et al., 2024 ; Raja Babu et al., 2024 ) . Changing the size of the input images can also change the performance of a CNN ( Richter et al., 2021 ) . In addition to these limitations, one of the primary downsides of CNNs is their “black box” nature. While CNNs are trained with data samples, the process of updating their weights through the training remains opaque. The rules that define how classifications are made are convoluted and unintuitive, and it has been proven difficult to interpret the reason behind the decision made by a deep neural network ( Ball, 2023 ) . The definition of what a CNN really “learns” from the data thus becomes equally difficult to make. In other words, lack of interpretability simply means that CNNs should be used with caution as their classification processes are not necessarily clear to the user ( Wang et al., 2020 ; Buhrmester et al., 2021 ) . Ideally, a CNN that can achieve high accuracy in classification on benchmark datasets should perform well also in real-world settings. On the other hand, empirical studies ( Ball, 2023 ; Tommasi et al., 2015 ; Dhar and Shamir, 2021 ) show that many popular datasets were biased, and possibly not representative of a model’s performance concerning real-world image recognition. That shows that CNNs trained on benchmark datasets might be driven by information that is not limited to the information related to the task they aim at solving. Tools that are used to identify biases or irrelevant information that drives the classification accuracy include saliency maps ( Alqaraawi et al., 2020 ; Kim et al., 2019 ; Arun et al., 2021 ) , which can be used to visualize the signal through the spacial domain. That allows to inspect and identify sources of signal that might not be relevant to the computer vision problem at hand ( Simonyan, 2013 ; Pfau et al., 2019 ) . The domain of adversarial neural networks has also been studied in the context of irrelevant information that can skew the way CNNs work, yet with no direct link to the relevant information ( Hashimoto et al., 2020 ; Lin et al., 2020 ) . Some solutions has also been proposed to detect adversarial samples ( Wang et al., 2021 ; Pertigkiozoglou and Maragos, 2018 ) . The need of deep neural networks for large datasets led to the practice of data augmentation. Data augmentation has been used to correct for a variety of biases in datasets, especially biases related to the distribution of the data and fairness of the analysis ( Jaipuria et al., 2020 ; Sharma et al., 2020 ; McLaughlin et al., 2015 ; Iosifidis and Ntoutsi, 2018 ) . As the use of CNNs in various fields continues to grow, it is imperative to address these limitations, which also include the risk of classification bias. That reinforces the need to develop methods that can assist researchers who develop CNNs, and allow them to reduce the risk of unknown or unexpected biases. 2 Classification bias in benchmark datasets The term “bias” in the context of benchmark datasets for image classification has been used to describe numerous situations. Those may include unbalanced distribution of the samples in the classes, biases in the labeling of the images, biased selection of data, and more ( Torralba and Efros, 2011 ) . Unbalanced distribution of the samples in datasets is a well-known bias driven by an unequal number of samples in different classes. Machine learning can be biased by the number of samples a machine learning model is trained by, making more populated classes become more frequent in the classification outcomes. That leads to some classes with higher confusion rate than other classes, consequently leading to problems in the fairness and discrimination by machine learning algorithms. One of the common solutions to the problem is by balancing the classes through data augmentation ( Jaipuria et al., 2020 ; Sharma et al., 2020 ; McLaughlin et al., 2015 ; Iosifidis and Ntoutsi, 2018 ) . Labeling bias is a situation of incorrect or inconsistent labeling. It is related to contextual information bias, in which elements that are not the target objects are present in one class more frequently than in other classes, and can therefore be used by a classifier to

Read the complete text