CHSER: A Dataset and Case Study on Generative Speech Error Correction for Child ASR
Natarajan, Zilai, Kaiyuan, Mohan, Abeer · 2025
Read CHSER: A Dataset and Case Study on Generative Speech Error Correction for Child ASR by Natarajan, Zilai, Kaiyuan, Mohan, Abeer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18463.txt.
Introduction
Natarajan Balaji Shankar and co-authors introduce a 2025 data set for fixing speech recognition errors in child speech. Systems that turn speech into text often fail on children, whose voices and word use differ from adults. Few child speech sets exist, so error rates stay high.
The new set, called CHSER, holds two hundred thousand pairs of flawed output and correct text. The pairs span many ages and speaking styles. Training a correction model on the set cut word errors by up to 28.5 percent on new systems. The method fixed word swaps and missed words well. It still struggled with added words and child speech breaks.
Catalog record
| CREATOR | Natarajan, Zilai, Kaiyuan, Mohan, Abeer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18463
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CHSER: A Dataset and Case Study on Generative Speech Error Correction for Child ASR Natarajan, Zilai, Kaiyuan, Mohan, Abeer Abstract Automatic Speech Recognition (ASR) systems struggle with child speech due to its distinct acoustic and linguistic variability and limited availability of child speech datasets, leading to high transcription error rates. While ASR error correction (AEC) methods have improved adult speech transcription, their effectiveness on child speech remains largely unexplored. To address this, we introduce CHSER, a Generative Speech Error Correction (GenSEC) dataset for child speech, comprising 200K hypothesis-transcription pairs spanning diverse age groups and speaking styles. Results demonstrate that fine-tuning on the CHSER dataset achieves up to a 28.5% relative WER reduction in a zero-shot setting and a 13.3% reduction when applied to fine-tuned ASR systems. Additionally, our error analysis reveals that while GenSEC improves substitution and deletion errors, it struggles with insertions and child-specific disfluencies. These findings highlight the potential of GenSEC for improving child ASR. 1 1 1 This work was supported in part by the NSF. Balaji Shankar Wang Zhang Shi Alwan University of California, Los AngelesUSA keywords Child Automatic Speech Recognition, Error Correction, Disfluency Detection † † email: balaji1312,zilaiwang2001,kaiyuanzhang,[email protected], [email protected] 1 Introduction Speech Foundation Models (SFMs) have advanced Automatic Speech Recognition (ASR), demonstrating state-of-the-art performance and strong zero-shot generalization [ 1 , 2 , 3 , 4 , 5 ] . Their success stems from large-scale pretraining and transformer-based architectures, yet they struggle with child speech, exhibiting significantly high error rates [ 6 ] . The performance gap between child and adult ASR arises from the distinct characteristics of child speech, including higher pitch, greater intra- and inter-speaker variability, and developmental changes [ 7 , 8 , 9 , 10 ] . Additionally, child speech contains greater disfluencies (e.g., repetitions, restarts), grammatical inconsistencies, and non-standard word usage, which degrade ASR accuracy [ 11 ] . Variability across speaking styles, such as scripted vs. spontaneous speech, further complicates generalization, necessitating effective post-processing to enhance transcription accuracy. Traditional ASR error correction (AEC) methods focus on rescoring and hypothesis selection, including n-gram-based rescoring [ 12 ] , consensus decoding [ 13 ] , and neural rescoring with RNNs and transformers [ 14 , 15 ] . More recent AEC approaches leverage machine translation models for domain adaptation [ 16 ] , fine-tuned RoBERTa models for readability improvements [ 17 ] , and crossmodal methods incorporating both audio and text representations [ 18 ] . Confidence-based correction strategies also integrate utterance rejection for improved robustness [ 19 ] . Generative speech error correction (GenSEC) has emerged as a promising approach, leveraging large language models (LLMs) to refine ASR outputs by learning error patterns from large-scale paired data. Datasets such as HyPoradise [ 20 ] and RobustHyPoradise [ 21 ] serve as benchmarks, demonstrating that fine-tuned generative models significantly enhance transcription accuracy. GenSEC exploits full N-best lists for improved hypothesis selection. LoRA-based [ 22 ] fine-tuning of Llama 2 [ 23 ] and Flan T5 [ 24 ] models enables efficient adaptation [ 20 , 25 ] without requiring full retraining. Other approaches include language-space noise embeddings for robustness [ 21 ] , integration of ASR embeddings with LLMs [ 26 ] , lattice-constrained decoding [ 27 ] , retrieval-augmented generation for out-of-domain correction [ 28 ] , and multi-expert models for ASR and speech translation [ 29 ] . However, these approaches are trained on adult speech, limiting their effectiveness for child-specific disfluencies and linguistic patterns. To address this gap, we introduce CHSER, a GenSEC dataset designed to enhance ASR transcription accuracy for child speech. Our dataset comprises 200k child speech ASR hypothesis pairs across diverse age groups (4–14 years) and speaking styles (scripted, spontaneous), enabling models to learn child-specific error patterns. We evaluate existing GenSEC methods to establish baselines and analyze their limitations in handling child speech variability. Furthermore, we assess the adaptability of child-specific GenSEC models in both zero-shot and fine-tuned settings, demonstrating their impact on improving child ASR performance across different domains. Our contributions can be summarized as follows: • We introduce the first large-scale dataset for child ASR Generative Speech Error Correction, comprising 200K hypothesis-transcription pairs spanning multiple age groups (4–14 years old) and speaking styles (scripted and spontaneous). • Through systematic evaluation, we address four key research questions: (RQ1) The effectiveness of existing GenSEC methods on child speech, (RQ2) The robustness of child-specific error correction models across different domains, (RQ3) The applicability of GenSEC models to fine-tuned ASR systems of differing architectures, and (RQ4) Whether GenSEC alters the nature of ASR errors by analyzing error distribution shifts post correction. 2 2 2 Our dataset, code, and models are available at https://github.com/balaji1312/CHSER 2 CHSER Dataset To construct a benchmark for child speech ASR error correction, we compile a dataset from multiple publicly available child speech corpora, ensuring diversity in age ranges and speaking styles. Our dataset consists of 200K ASR hypothesis-transcription pairs sourced from four major child speech datasets: MyST [ 30 ] , CMU Kids [ 31 ] , CSLU OGI Kids [ 32 ] , and CHILDES English OCSC Corpus [ 33 ] . Similar to the procedure used for adult speech in [ 20 ] , we generated N-best lists of sentence hypotheses using the Whisper-base.en model [ 4 ] . To improve hypothesis-transcription pair quality, we apply the following filtering steps: remove repeated utterances, retain only the top 5 hypotheses with the highest probabilities in the N-best list, exclude transcriptions shorter than three words, and discard utterances containing words outside the Whisper tokenizer vocabulary. The CHSER dataset in total contains over 200K ASR hypothesis-transcription pairs and is summarized in Table 1 . All dataset divisions into train, dev, and test sets follow the original splits in their respective source datasets. Below, we provide an overview of each source dataset: The My Science Tutor (MyST) corpus [ 30 ] is a 400-hour English speech dataset collected from 1,371 students in grades 3–5 (ages 8-11). It features spontaneous, unscripted conversations with a virtual science tutor on topics such as physics, geography, and biology. Of this, we include 64,187 utterances in the train set, 10,458 in the dev set, and 11,591 in the test set. The CMU Kids Corpus [ 34 ] consists of read-aloud sentences from 6–11-year-old children. It includes 24 male and 52 female speakers with a total of 5,180 utterances recorded over 9 hours of speech. We incorporate 3,784 utterances in the train set, 642 in the dev set, and 754 in the test set. The CSLU OGI Kids Corpus [ 32 ] contains speech from 1,100 children (ages 4–14) across both scripted and spontaneous speech conditions. The read speech section consists of simple words, sentences, and digit strings. The spontaneous speech section includes alphabet recitation and unscripted monologues. We utilize both sections in CHSER, with 18,121 utterances (scripted) and 3,730 utterances (spontaneous) in the train set, 1,971 and 451 in the dev set, and 5,795 and 457 in the test set, respectively. The OCSC corpus contains 156 hours of speech from 303 primarily monolingual Engish speaking children aged 4–9. The dataset includes a mix of scripted and spontaneous speech, covering structured tasks such as reading passages, and letter and number identification,