Fairness in Dysarthric Speech Synthesis: Understanding Intrinsic Bias in Dysarthric Speech Cloning using F5-TTS
Anuprabha, Krishna, Anil Kumar · 2025
Read Fairness in Dysarthric Speech Synthesis: Understanding Intrinsic Bias in Dysarthric Speech Cloning using F5-TTS by Anuprabha, Krishna, Anil Kumar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05102.txt.
Introduction
People with dysarthria speak in ways that are hard to understand, and recordings of their speech are scarce. Voice cloning could multiply that data for assistive tools. Anuprabha and co-authors test F5-TTS, a current synthesis system, on the TORGO dataset of dysarthric speech. They score the clones for intelligibility, speaker likeness, and prosody. Fairness metrics then check whether quality shifts with severity level.
The system favored intelligibility over keeping the speaker's identity and rhythm. That tilt amounts to an intrinsic bias in the cloning. The authors argue synthesis for disabled speakers needs fairness checks built in. Their goal is assistive technology that serves severe and mild speakers alike.
Catalog record
| CREATOR | Anuprabha, Krishna, Anil Kumar |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-05102
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fairness in Dysarthric Speech Synthesis: Understanding Intrinsic Bias in Dysarthric Speech Cloning using F5-TTS Anuprabha, Krishna, Anil Kumar Abstract Dysarthric speech poses significant challenges in developing assistive technologies, primarily due to the limited availability of data. Recent advances in neural speech synthesis, especially zero-shot voice cloning, facilitate synthetic speech generation for data augmentation; however, they may introduce biases towards dysarthric speech. In this paper, we investigate the effectiveness of state-of-the-art F5-TTS in cloning dysarthric speech using TORGO dataset, focusing on intelligibility, speaker similarity, and prosody preservation. We also analyze potential biases using fairness metrics like Disparate Impact and Parity Difference to assess disparities across dysarthric severity levels. Results show that F5-TTS exhibits a strong bias toward speech intelligibility over speaker and prosody preservation in dysarthric speech synthesis. Insights from this study can help integrate fairness-aware dysarthric speech synthesis, fostering the advancement of more inclusive speech technologies. M Gurugubelli Vuppala International Institute of Information Technology-HyderabadIndia Samsung Research & Development Institute-BengaluruIndia keywords Dysarthria, F5-TTS, Fairness, Synthetic speech. † † email: [email protected], [email protected], [email protected] 1 Introduction Dysarthria, a motor speech disorder caused by neurological conditions, results in a speech that is often slurred, slow, or difficult to understand, particularly in its more severe forms [ 1 , 2 ] . Individuals with dysarthria often encounter communication challenges, making speech-based assistive technologies such as Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) systems vital for facilitating their interaction with digital devices. However, the development of robust dysarthric speech technologies is often hindered by data scarcity, as dysarthric speech varies widely in severity and speaker characteristics [ 3 ] . Data augmentation is widely explored in dysarthric speech applications to address this challenge of data scarcity [ 4 ] . Traditional approaches for data augmentation include speed, temporal, or spectral modifications [ 4 , 5 , 6 ] , GAN-based adversarial augmentation [ 7 , 8 ] , and voice conversion [ 9 , 10 , 11 ] . Additionally, advancements in end-to-end TTS and voice cloning systems have enabled the generation of high-quality synthetic speech with human-like qualities, including diverse speaker characteristics and styles. These advancements have been studied by [ 12 , 13 , 14 ] to synthesize dysarthric speech. By leveraging state-of-the-art voice cloning models, such as F5-TTS [ 15 ] , researchers can generate personalised dysarthric speech at varying severity levels, which has the potential to enhance the training of ASR models and other assistive speech technologies. Despite the utilization of neural speech synthesis systems for augmenting dysarthric speech, several critical questions remain unanswered regarding the effectiveness and fairness of dysarthric speech cloning, like: Can synthetic dysarthric speech maintain intelligibility? How does it impact ASR performance? How well the cloned speech retains speaker identity in replicating different severity levels of dysarthria? Do speech synthesis models exhibit bias across dysarthric severity levels in ways that could be disadvantageous to certain user groups? During dysarthric speech synthesis across severity levels, challenges related to fairness can manifest in various ways, such as intelligibility bias (comprehensibility of spoken content), speaker identity (preservation of speaker characteristics), and prosody inconsistencies (rhythmic and tonal aspects of speech). Due to this, speech data augmentation becomes biased, leading to skewed performance in ASR and making speech recognition less effective, particularly for severely impaired dysarthric speakers [ 16 ] . Moreover, assistive speech systems trained on biased synthetic data could reinforce disparities, thus negatively impacting accessibility and usability for individuals with dysarthria. Hence, it is very important to understand the intrinsic biases in dysarthric speech syntheses to incorporate fairness-aware augmentation strategies. Over the past decade, significant work has been done in assessing biases by focusing on age, gender, and geographic factors within ASR and spoken language understanding (SLU) systems [ 17 , 18 , 19 , 20 ] . To the best of our knowledge, this is the first study to analyse biases in synthetic dysarthric speech generated by a zero-shot voice cloning system. The key contributions of this work are as follows: • Assess the quality of synthesised dysarthric speech in terms of intelligibility, speaker similarity, and prosody preservation, using objective metrics like Word Error Rate (WER), Character Error Rate (CER), Speaker Similarity (SIM-o) and Auto Prosodic Consistency Protocol (AutoPCP). • Introduce a framework to quantify fairness in synthesising dysarthric speech based on Parity Difference (PD) and Disparate Impact (DI) metrics. • An in-depth fairness analysis is conducted on dysarthric speech generated by F5-TTS, examining variations across different severity levels, gender, and individual speakers using the TORGO dataset [ 21 ] . • Impact of incorporating voice-cloned samples on downstream tasks, such as detection and speech recognition of dysarthria. • Provision of key insights into the need for fairness-aware analysis in the generation of synthetic data to improve dysarthric speech technologies, particularly ASR and assistive communication systems. 2 Proposed Methodology This section introduces the proposed framework for evaluating performance variations in preserving information related to dysarthric severity during zero-shot synthetic speech generation. As illustrated in Figure 1, the framework follows a systematic approach, beginning with the generation of synthetic speech using F5-TTS, followed by an objective assessment, and concluding with the measurement of bias across different severity categories using two fairness metrics. Furthermore, this section outlines the formulation and methodology employed to derive the fairness metrics based on the objective assessment. Figure 1: Framework proposed for assessing fairness in dysarthric speech generation using F5-TTS 2.1 Speech Cloning using F5-TTS F5-TTS [ 15 ] , a zero-shot, non-autoregressive TTS system, is employed in this study to generate speech samples for dysarthric speakers. The model synthesizes speech using a few seconds of an audio prompt (reference speech sample), its correspoding transcription, and a text prompt for synthetic speech generation. F5-TTS leverages flow matching and the Diffusion Transformer (DiT), eliminating the need for components such as a duration predictor, text encoder, context-aware codec model, or phoneme alignment. Instead, it employs ConvNeXt V2 to align the text input with the speech length by padding it with filler tokens, followed by a de-noising process to generate high-quality, natural-sounding speech. Trained on 100K hours of publicly available multilingual data [ 22 ] , F5-TTS exhibits exceptional zero-shot performance, producing realistic speech while preserving naturalness, intelligibility, and speaker similarity. In this study, the F5-TTS base model 1 1 1 https://github.com/SWivid/F5-TTS is used to generate synthetic speech for speakers in the TORGO database. 2.2 Objective Assessment of Dysarthric Speech Cloning To objectively assess how well the generated speech preserves different levels of dysarthric severity in terms of intelligibility, speaker, and prosodic similarity, we evaluate it using the following criteria: • WER and CER : The intelligibility of the generated samples is assessed using both WER and CER. The addition of CER offers a detailed analysis by focusing on errors at the character level, which is particularly useful for capturing subtle speech distortions such as mispronunciations or missing characters often present in dysarthric speech [ 23 ] . Due to this, the evaluation process gains a more comprehensive understanding of the TTS system’s ability to handle the intricate nuances of dysarthric speech, ensuring a more accurate assessment of its overall performance on intelligibility. Additionally, to evaluate potential biases in the context of intelligibility, it is essential to incorporate the differences