Exploring Gender Disparities in Automatic Speech Recognition Technology
Hend ElGhazaly, Bahman Mirheidari, Nafise Sadat Moosavi, Heidi Christensen · 2025
Read Exploring Gender Disparities in Automatic Speech Recognition Technology by Hend ElGhazaly, Bahman Mirheidari, Nafise Sadat Moosavi, Heidi Christensen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-18434.txt.
Introduction
Hend ElGhazaly and co-authors probe gender gaps in speech recognition in a 2025 paper. Most bias checks stop at head counts, yet training mix may bend accuracy in less plain ways. The team used the LibriSpeech set and the small Whisper model to vary how much male and female speech the training held.
Fair results peaked at set gender ratios that were not a plain half and half split. Pitch spread also swayed accuracy by a clear amount. The findings warn that fair speech tools need care in curating training data, not just in counting who is in it. Both mix and voice traits count.
Catalog record
| CREATOR | Hend ElGhazaly, Bahman Mirheidari, Nafise Sadat Moosavi, Heidi Christensen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-18434
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploring Gender Disparities in Automatic Speech Recognition Technology Hend ElGhazaly, Bahman Mirheidari, Nafise Sadat Moosavi, Heidi Christensen Abstract This study investigates factors influencing Automatic Speech Recognition (ASR) systems’ fairness and performance across genders, beyond the conventional examination of demographics. Using the LibriSpeech dataset and the Whisper small model, we analyze how performance varies across different gender representations in training data. Our findings suggest a complex interplay between the gender ratio in training data and ASR performance. Optimal fairness occurs at specific gender distributions of rather than a simple 50-50 split. Furthermore, our findings suggest that factors like pitch variability can significantly affect ASR accuracy. This research contributes to a deeper understanding of biases in ASR systems, highlighting the importance of carefully curated training data in mitigating gender bias. 1 Introduction Speech technologies have demonstrated their effectiveness and efficiency across various domains, however, they can inadvertently perpetuate biases. Bias refers to a performance gap discriminating against certain groups or individuals. For example, we consider an Automatic Speech Recognition (ASR) system biased when its recognition accuracy is much worse for one gender than the other. Despite extensive research on mitigating bias in AI, the concept of bias and fairness in speech technologies remains relatively underexplored. In speech research, studies on bias have included more emphasis on speakers’ demographics in training data such as gender, race, accents, and language variants Feng et al. 2024 ; Zhang et al. 2022 ; Meyer et al. 2020 ; Maison and Estève 2023 ; Koenecke et al. 2020 . This study investigates influences on gender bias in ASR systems beyond the speakers’ demographics. While prior research has mainly focused on gender imbalances Garnerin et al. 2021 ; Abushariah et al. 2012 , our study takes a comprehensive approach by examining not only the gender ratios in the training data but also the speech content readability, semantic similarity, and pitch distribution. Thus in this work we aim to examine the following research questions: • RQ1: How do different gender distributions in the training set affect ASR performance and fairness? • RQ2: What factors other than gender ratios influence the ASR performance? Our results reveal a nuanced relationship between gender ratios in training data and ASR performance, challenging the notion that balanced gender representation correlates with best performance. Furthermore, our findings suggest that factors like pitch variability significantly affect ASR accuracy, underscoring the importance of a holistic approach to dataset composition as a mitigation strategy. This research enhances the understanding of gender bias in ASR systems, emphasising the importance of accounting for diverse factors to achieve fairer speech recognition. 2 Methodology and Experimental Setup We investigate the effects of gender distribution, speech content and speakers’ pitch distributions in the training set on the ASR performance. The following subsections describe the datasets and ASR model used in the experiments. 2.1 Train and Test sets: LibriSpeech The LibriSpeech corpus is widely used in ASR research ( Panayotov et al. 2015 ) . The dataset includes recordings of individuals reading passages from books, which cover a range of genres and topics. It contains approximately 1000 hours of transcribed English audiobooks from LibriVox. The dataset is divided into train, dev, and test subsets. There are in total 5831 chapters from 1568 books read by 2484 speakers. The audio files in the dataset are chapters read by each speaker. All chapters in the dataset are unique but can be from the same books and multiple speakers read from the same book. The speakers’ names, gender, book titles, and chapter titles are provided in the dataset. Since the only speakers’ demographic provided is gender (denoted as female/male), we focused on investigating gender. The authors of LibriSpeech claim that the dataset is gender balanced at speaker level and there is no overlap between the subsets. The LibriSpeech corpus has two test sets; “TestClean” and “TestOther”. We report the evaluation results on the LibriSpeech “TestOther” set as it is considered a more challenging dataset because of lower speech quality and background noises, making it more representative of real-world scenarios. Table 1 summarizes the extracted number of chapters, hours and unique speakers per gender in the training and test sets. Training TestOther Chapters Hours Speakers Chapters Hours Speakers Women 2671 465.32 1128 46 2.65 17 Men 2795 495.73 1210 44 2.69 16 All 5466 961.05 2338 90 5.34 33 Table 1: Gender distribution in the original LibriSpeech training and test sets 2.2 Data Pre-processing We used the LibriSpeech corpus, described in Section 2.1 , to create 11 training subsets with varying gender distributions to investigate how increasing the number of women in the training set affects the ASR performance. The women’s percentage in the training set was changed incrementally from zero to 100. Only the number of audio files with female speakers changed in each training set. The number of audio files corresponds to the number of chapters in Table 1 , ensuring the content is different in each sample. As gender was the only speaker’s metadata provided, we evaluated the ASR performance based on the reported speakers’ gender (female/male) in the test set. Our focus on these two genders is only driven by their representation in the dataset used for our investigation. The same test set was used in all experiments to assess the performance with the same speakers. Table 2 outlines information on the gender distributions of audio samples used in each training subset and test set. Dataset Composition Women Men Total Audio Files Original train set 2671 2795 5466 0%women 0 2671 2671 10%women 267 2404 2671 … … … … 90%women 2404 267 2671 100%women 2671 0 2671 TestOther 44 46 90 Table 2: Number of audio files in training and test sets. 2.3 ASR System: Whisper Model We used the pre-trained Whisper small model Radford et al. 2022 as a baseline in our experiments; given its high performance and widespread use. While larger models potentially offer higher accuracy, they require significantly more processing time and computational power, which exceed our available resources. Thus, the small model was chosen to ensure feasibility within these constraints. The model was first fine-tuned using the original LibriSpeech training set and dev set for validation, with a focus on hyperparameter optimization. We specifically targeted the number of optimization steps that run during training, learning rate, and train batch size. After experimentation, the optimal settings were determined based on the minimum WER on the dev set, which are 3000 steps, a 5.00E-05 learning rate and a batch size of 16. These settings were used in all the fine-tuning experiments. 2.4 Evaluation Metrics We assessed the ASR model performance by measuring the Word Error Rate (WER) Klakow and Peters 2002 ; the lower the WER the better the performance. To evaluate the model’s fairness, we measured the gender bias, by computing the difference in WER between women and men speakers, as described in Feng et al. 2024 work. Besides the gender ratio in the training set, we also explored other factors including text content and speakers’ pitches to assess their potential impact on the results. For the text analysis, we measured the text difficulty in the train and test sets using the Flesch-Kincaid Grade Level readability metric. We also computed