Gender Fairness in Audio Deepfake Detection: Performance and Disparity Analysis

Aishwarya Fursule, Shruti Kshirsagar, Anderson R. Avila · 2026

Read Gender Fairness in Audio Deepfake Detection: Performance and Disparity Analysis by Aishwarya Fursule, Shruti Kshirsagar, Anderson R. Avila (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-09007.txt.

Introduction

Aishwarya Fursule and colleagues test gender fairness in audio deepfake detection. Fake voices grow ever more lifelike, raising fraud risks, yet gender bias in detection remains little studied. Training a ResNet-18 classifier on ASVspoof 5 with five fairness metrics, they find small overall error gaps between genders hide real disparities in error distribution that aggregate measures obscure.

Catalog record

CREATORAishwarya Fursule, Shruti Kshirsagar, Anderson R. Avila
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25530
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.09007

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Gender Fairness in Audio Deepfake Detection: Performance and Disparity Analysis Aishwarya Fursule, Shruti Kshirsagar, Anderson R. Avila Abstract Audio deepfake detection aims to detect real human voices from those generated by Artificial Intelligence (AI) and has emerged as a significant problem in the field of voice biometrics systems. With the ever-improving quality of synthetic voice, the probability of such a voice being exploited for illicit practices like identity thest and impersonation increases. Although significant progress has been made in the field of Audio Deepfake Detection in recent times, the issue of gender bias remains underexplored and in its nascent stage In this paper, we have attempted a thorough analysis of gender dependent performance and fairness in audio deepfake detection models. We have used the ASVspoof 5 dataset and train a ResNet-18 classifier and evaluate detection performance across four different audio features, and compared the performance with baseline AASIST model. Beyond conventional metrics such as Equal Error Rate (EER %), we incorporated five established fairness metrics to quantify gender disparities in the model. Our results show that even when the overall EER difference between genders appears low, fairness-aware evaluation reveals disparities in error distribution that are obscured by aggregate performance measures. These findings demonstrate that reliance on standard metrics is unreliable, whereas fairness metrics provide critical insights into demographic-specific failure modes. This work highlights the importance of fairness-aware evaluation for developing a more equitable, robust, and trustworthy audio deepfake detection system. I Introduction Audio deepfakes have become more prevalent due to recent advances in artificial intelligence and deep learning techniques [ 1 ] . These systems can generate speech that sounds very close to real human, making it difficult to distinguish between bonafide and spoof speech [ 2 ] . As a result, audio deepfakes are increasingly used for harmful purposes, including identity fraud, the creation and dissemination of false evidence, and the spread of misinformation [ 3 ] . To mitigate these risks, many audio deepfake detection models have been proposed in recent years [ 4 ] . Recent research focuses on improving detection accuracy, reducing error rate, and robustness against different spoofing attacks, often evaluated using benchmark datasets such as those from the Automatic Speaker Verification and Spoofing Countermeasures Challenge (ASVSpoof) [ 4 , 11 , 16 , 27 , 28 , 29 ] . While these initiatives enable fair comparisons of new detection approaches, solutions for audio deepfake detection often lack an explicit examination of whether performance differs between male and female speakers [ 11 , 10 , 15 ] . Most fairness studies in deepfake detection have focused on other domains, such as image and video-based systems. These studies suggest that performance disparities may exist across demographic groups [ 37 ] , even when standard metrics indicate no bias in these groups. This highlights the importance of evaluating fairness as a measure of how consistently the system behaves across all user groups, beyond traditional metrics [ 9 , 30 ] . This represents an important research gap in audio deepfake detection systems, as deployed systems are expected to perform equally across all user groups [ 23 ] , particularly with respect to gender [ 5 ] . Speech signals naturally vary between male and female speakers due to differences in pitch, vocal range, and speaking patterns [ 6 ] . If these variations are not adequately accounted for during training, the detection model may exhibit bias and perform unevenly across genders [ 7 ] . Thus, in this study, we investigate gender fairness in audio deepfake detection. We used the most recent ASVspoof Challenge dataset and applied four feature representations with the same base classifier, ResNet-18. Five fairness metrics in the AI literature are considered: Statistical Parity, Equal Opportunity, Equality of Odds, Predictive Parity, and Treatment Equality. [ 13 ] . In addition, we compared these results with AASIST [ 36 ] , the state-of-the-art, an end-to-end model for ASVspoof 5, which takes raw audio as input. In this experiment, we evaluated model performance by calculating the Equal Error Rate (EER) for audio deepfake detection. To assess fairness, we used fairness metrics to examine disparities in performance between genders. The goal of this study is to determine whether predictions are influenced by speaker gender under the same experimental conditions. II Related Work Fairness analysis has primarily focused on image and video modalities; systematic investigations of gender bias in audio deepfake detection systems have only recently emerged. For example, research in [ 15 ] , the authors evaluated machine learning and deep-learning models on a gender-balanced audio deepfake dataset and reported differences in detection performance between male and female speech, observing higher accuracy for female voices across several configurations. This research provides empirical evidence that gender characteristics might influence audio deepfake detection performance, although gender is treated as an evaluation factor rather than within a formal fairness framework. Likewise, [ 33 ] conducted the study examining bias in synthetic speech detector through their FairSSD framework. Their analysis revealed that most existing detectors exhibit significant gender bias, with systematically higher false positive rates for male speakers compared to female speakers. Moreover, [ 34 ] investigated gender-specific performance characteristics in their development of real-time detection systems for AI-generated speech. Their analysis revealed that the models trained on female audio significantly outperformed those trained on male audio. They attributed this finding to the expressive nature of female voice features and the presence of high-pitched artifacts in synthetic audio, which may be more easily detectable in the female vocal range. This suggests that certain deepfake generation techniques may interact differently with male and female vocal characteristics, producing artifacts that are more or less detectable depending on speaker gender. Existing audio deepfake detection studies rarely incorporate formal fairness metrics or gender-based evaluation, and most rely on global performance measures such as accuracy or EER. Recent forensic research by [ 35 ] examined deepfake detection using segmental speech features and confirmed that group-dependent performance differences exist for deepfake speech detectors. Their comprehensive review of the literature identified systematic demographic bias across gender, age, accent, and language dimensions. They documented that detection accuracy varies significantly by speaker gender even when using controlled samples, and emphasized that these biases can be amplified when detectors are trained on large, imbalanced corpora using metric-learning objectives. This amplification of training data bias leads to unequal error rates across speaker groups, a pattern that has serious implications for the fairness and reliability of detection systems. III Methodology Fig. 1: The five group fairness metrics adopted in this study. III-A Implication of Fairness Metrics in Gender-aware Audio Deepfake Detection In this study, fairness is evaluated with respect to gender, with two demographic groups: Female (F) and Male (M) speakers, denoted by G = Gender. The audio deepfake detection task is formulated as a binary classification problem, where Y=0 denotes spoofed (deepfake) speech and Y=1 denotes bonafide (genuine) speech. The predicted class label is denoted by \hat{Y} . Fairness metrics quantify model behavior across gender groups. Additionally, fairness metrics are computed using three approaches: (i) Mathematical Formulations derived from confusion matrices, (ii) the Fairlearn library, and (iii) the AIF360 framework. Fig. 2: Balanced gender distribution for the ASVspoof5 dataset across train, development,

Read the complete text