Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech
Jaesung, Xiuwen, Minje, Chang D, Mark · 2026
Read Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech by Jaesung, Xiuwen, Minje, Chang D, Mark (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15988.txt.
Introduction
Bae and co-authors grade dysarthric speech severity with help from unlabeled data. Hand scoring of such speech costs much and scales poorly. Labeled sets stay small, which weakens direct model training. Their three-stage plan starts with a teacher model. The teacher pseudo-labels unlabeled dysarthric speech samples. Label-aware contrastive pretraining then spans many speakers and settings. A final fine-tune targets the severity task itself. Five unseen datasets cover varied causes and languages. The full stack reaches a mean rank score of 0.761.
Catalog record
| CREATOR | Jaesung, Xiuwen, Minje, Chang D, Mark |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50117 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.15988
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech Jaesung, Xiuwen, Minje, Chang D, Mark Abstract Dysarthric speech quality assessment (DSQA) is critical for clinical diagnostics and inclusive speech technologies. However, subjective evaluation is costly and difficult to scale, and the scarcity of labeled data limits robust objective modeling. To address this, we propose a three-stage framework that leverages unlabeled dysarthric speech and large-scale typical speech datasets to scale training. A teacher model first generates pseudo-labels for unlabeled samples, followed by weakly supervised pretraining using a label-aware contrastive learning strategy that exposes the model to diverse speakers and acoustic conditions. The pretrained model is then fine-tuned for the downstream DSQA task. Experiments on five unseen datasets spanning multiple etiologies and languages demonstrate the robustness of our approach. Our Whisper-based baseline significantly outperforms SOTA DSQA predictors such as SpICE, and the full framework achieves an average SRCC of 0.761 across unseen test datasets. Bae Zheng Kim Yoo Hasegawa-Johnson keywords dysarthria, speech quality assessment, data augmentation, weakly-supervised, contrastive learning † † address: 1 University of Illinois Urbana-Champaign, IL, USA 2 Korea Advanced Institute of Science & Technology, KR † † email: {jb82, xiuwenz2, minje, jhasegaw}@illinois.edu, [email protected] 1 Introduction Dysarthria is a motor speech disorder caused by neurological impairments, leading to substantial degradation in acoustic and perceptual characteristics. Accurate dysarthric speech quality assessment (DSQA) is essential for clinical diagnosis, early detection of progressive neurological conditions, rehabilitation monitoring, and the development of inclusive speech technologies, including pathological speech enhancement and automatic speech recognition. However, DSQA relies on expert clinical evaluation by speech-language pathologists (SLPs), limiting its scalability and accessibility in real-world settings. An automated and objective DSQA system would therefore complement SLP expertise by enabling continuous monitoring outside the clinic. Recent deep learning-based SQA models can reliably assess the perceptual correlates of algorithmic and channel distortion; in particular, non-intrusive SQA (NI-SQA) approaches estimate speech quality directly from the input signal without reference recordings. For example, DNSMOS [ 26 ] evaluates speech quality in real-world noisy and reverberant conditions, while UTMOS [ 29 ] predicts subjective quality for clean and enhanced speech. The growing availability of large-scale dysarthric datasets has further enabled the development of deep learning-based DSQA models. SpICE [ 36 ] is trained on 550,000 disordered speech samples from Project Euphonia [ 18 ] to automatically predict human-perceived intelligibility. More recently, [ 21 ] develops voice quality probes for dysarthric speech across seven perceptual dimensions, trained on 11,184 samples from the Speech Accessibility Project (SAP) [ 9 ] . These previous approaches show promising results; however, most evaluations have been conducted on English corpora, which may limit their generalizability to non-English datasets. Contrastive learning methods such as SimCLR [ 3 ] are a powerful self-supervised learning method that can form a structured latent representation space. It has also been widely adopted in speech technology, e.g., to train speech foundation models such as wav2vec 2.0 [ 1 ] and HuBERT [ 10 ] . This representation space is also proven to be effective for DSQA approaches, such as in [ 36 , 21 ] . However, to transfer the speech foundation model into the downstream DSQA task, a labeled dataset is required, and the availability of in-domain training data remains scarce. The Euphonia dataset [ 18 ] is not publicly released, limiting its use in research. In the SAP corpus [ 9 ] , only a small fraction of each participant’s speech, i.e., roughly 30 read sentences, is rated by speech-language pathologists, compared with 350–400 read sentences and 50–80 spontaneous speech samples in total. This limited amount of labeled datasets for dysarthric speech may limit the performance of DSQA models, especially in terms of robustness. To fully leverage the SAP dataset, which contains a small labeled subset and a much larger unlabeled corpus, we propose a three-stage framework. Additionally, we propose to adopt the large-scale typical speech corpus LibriSpeech [ 23 ] to enhance the speaker variability and acoustic environment of the training dataset and improve the robustness. In the first stage, a teacher regression model is trained on the limited labeled SAP subset using Whisper-large [ 25 ] as the speech encoder, then used to generate pseudo-labels for the extensive unlabeled SAP samples. In the second stage, these pseudo-labeled samples are then combined with Librispeech and used for weakly-supervised pretraining. During this stage, we employ a label-aware contrastive learning approach inspired by [ 13 ] to better align the representations with perceptual quality labels. In the final stage, the pretrained representation layers are fine-tuned on the labeled SAP subset for the downstream regression task. To demonstrate the effectiveness of our proposed methods, especially in terms of robustness to unseen test data, we build diverse test datasets with varying etiologies and languages, including UASpeech [ 14 ] , DysArinVox [ 40 ] , EasyCall [ 34 ] , EWA-DB [ 22 ] , and NeuroVoz [ 19 ] . Our baseline model, trained on the SAP labeled subset, achieves an utterance-level Spearman’s Rank Correlation Coefficient (SRCC) of 0.719 on the SAP test set and an average speaker-level SRCC of 0.732 on the cross-domain test sets. Our proposed three-stage framework consistently improves over the baseline, reaching an average SRCC of 0.761 on the cross-domain test sets while preserving performance on the SAP dataset. We further demonstrate that weak supervision is essential for harmonizing the SAP and LibriSpeech datasets and improving cross-domain robustness. Our contributions are summarized as follows. • We propose a three-stage framework that gradually improves the labeling quality of the SAP dataset by using pseudo-labeling, followed by representation learning with a contrastive objective. This method fully leverages the large portion of unlabeled data in the SAP dataset. • We propose to incorporate a large-scale typical speech dataset, LibriSpeech, into our DSQA model training pipeline, which improves the speaker and acoustic environment variability. • We evaluate our method with various cross-domain dysarthric speech datasets with diverse labeling metrics and languages, demonstrating the effectiveness and robustness of our proposed methods. • Through a rigorous ablation study, we demonstrate the importance of providing weak supervision during the representation learning stage and adopting the LibriSpeech dataset. • Model checkpoint and training code are publicly available. 1 1 1 https://github.com/JaesungBae/DA-DSQA 2 Related Works 2.1 Speech Quality Assessment Speech quality assessment (SQA) aims to predict perceived speech quality, typically represented by a mean opinion score (MOS). Traditional intrusive metrics such as PESQ [ 27 ] and signal-based measures like SI-SNR [ 28 ] and STOI [ 32 ] require reference signals, and even codec-oriented intrusive metrics like WARP-Q [ 12 ] share this limitation, whereas modern generative applications (e.g., text-to-speech (TTS) and speech enhancement) increasingly rely on non-intrusive SQA (NI-SQA) methods. With the availability of large-scale MOS datasets and benchmarks, deep learning approaches have become the dominant paradigm for NI-SQA. Representative systems include DNSMOS [ 26 ] , which predicts multi-dimensional MOS for the deep noise suppression (DNS) task, and UTMOS [ 29 ] , which employs a strong-weak learner framework to model human perception of the naturalness of synthetic speech and achieved state-of-the-art performance in the VoiceMOS Challenge 2022 [ 11 ] . More recently, [ 8 ] introduced a training-free NI-SQA metric based