Clinical Annotations for Automatic Stuttering Severity Assessment
Ana Rita, Rufael, Hawau Olamide, Hamdan, Anelise, Inma, Elsa Marta, Gonçalo, Hanan · 2025
Read Clinical Annotations for Automatic Stuttering Severity Assessment by Ana Rita, Rufael, Hawau Olamide, Hamdan, Anelise, Inma, Elsa Marta, Gonçalo, Hanan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00644.txt.
Introduction
Ana Rita Valente and co-authors present a dataset paper from 2025 on speech and stuttering. The team worked to improve a shared speech dataset called FluencyBank. They added a new way to label stuttering that follows rules used by trained clinicians in real practice.
Expert clinicians were hired to label the speech data by hand. Their labels cover sounds and video, and they mark stuttering moments, other behaviors, and scores for tension. The team also built a test set where experts agreed on the labels together. Tests with the new labels show how hard the task is. Good training and scoring of computer models need real clinical skill.
Catalog record
| CREATOR | Ana Rita, Rufael, Hawau Olamide, Hamdan, Anelise, Inma, Elsa Marta, Gonçalo, Hanan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00644
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Clinical Annotations for Automatic Stuttering Severity Assessment Ana Rita, Rufael, Hawau Olamide, Hamdan, Anelise, Inma, Elsa Marta, Gonçalo, Hanan Abstract Stuttering is a complex disorder that requires specialized expertise for effective assessment and treatment. This paper presents an effort to enhance the FluencyBank dataset with a new stuttering annotation scheme based on established clinical standards. To achieve high-quality annotations, we hired expert clinicians to label the data, ensuring that the resulting annotations mirror real-world clinical expertise. The annotations are multi-modal, incorporating audiovisual features for the detection and classification of stuttering moments, secondary behaviors, and tension scores. In addition to individual annotations, we additionally provide a test set with highly reliable annotations based on expert consensus for assessing individual annotators and machine learning models. Our experiments and analysis illustrate the complexity of this task that necessitates extensive clinical expertise for valid training and evaluation of stuttering assessment models. Valente Marew Toyin Al-Ali Bohnen Becerra Soares Leal Aldarmaki University of AveiroPortugal SpeechCare Center MBZUAIUAE Instituto Brasileiro de FluênciaBrazil keywords stuttering assessment, disfluency detection † † email: [email protected], [email protected] 1 Introduction Stuttering is a neurodevelopmental and multidimensional communication disorder that begins early in speech and language development [ 1 , 2 , 3 ] . It is characterized by involuntary disruptions in speech fluency, such as repetitions, prolongations, and blocks, which are inconsistent and variable [ 2 , 4 , 5 ] . In addition to speech disruptions, stuttering also includes various behavioral, emotional, and cognitive components. Behaviorally, stuttering is marked by speech interruptions, muscle tension, and other behaviors related to speech difficulties (e.g. escape behaviors). These speech production challenges can trigger emotional responses such as fear, guilt, shame, or anxiety related to speaking and anticipation of stuttering. Stuttering can negatively affect several areas of daily life for both children and adults, who often report feelings of “loss of control” [ 6 , 7 ] . The affective, behavioral, and cognitive aspects of stuttering can limit their interactions in personal, social, and professional contexts [ 8 , 9 ] . Several cognitive factors play a role in stuttering, such as avoidance behaviors, intended to alleviate emotional and behavioral stress [ 2 ] . Understanding the multiplicity of stuttering disorder involves the integration of all the components mentioned above in a comprehensive assessment process [ 4 , 10 ] . Specifically, with regard to the behavioral component, it is crucial to identify the types of disfluencies and to determine their frequency, duration, and the tension with which they are produced. Accurately characterizing these disfluencies is essential for determining the severity of stuttering, establishing treatment goals, tracking progress throughout the intervention, and monitoring post-treatment results [ 5 ] . However, the variable nature of this fluency disorder complicates the assessment process. Stuttering severity fluctuates across different speaking situations, with various interlocutors, audience sizes, or conversation topics [ 1 , 5 , 11 ] . Traditionally, the assessment of stuttering moments in Adults Who Stutter (AWS) is conducted using audio-visual recordings of communicative situations, typically in a therapeutic setting [ 12 , 13 ] . The analysis is based on the evaluators’ perceptions and the individuals’ self-reports of difficulties. Moreover, it is still predominantly performed manually or with limited use of technology, making it challenging, time-consuming, and reducing its reliability and validity [ 14 ] . Several commendable efforts to advance automatic processing methods for stuttered speech exist, including the FluencyBank dataset [ 15 ] which includes annotated videos of adults who stutter. However, annotations of these samples include only the primary disfluency type, which is insufficient for clinically-accurate severity assessment. Furthermore, existing annotation efforts rely on non-expert listeners [ 16 ] which results in unreliable annotations. In this work, we propose a comprehensive multi-dimensional annotation scheme for stuttering severity assessment that aligns with clinical practice. We hire expert Speech-Language Pathologists (SLPs) to perform the annotations on the public segments of the FluencyBank dataset. We analyze inter-annotator agreement, and provide consenus-based annotations for the test set. The resulting annotations can be used for training and validating multi-modal, multi-label automatic stuttering assessment systems. The annotations will be made publicly available to align stuttering research efforts with clinical use cases. 2 Related Work Research on automatic stuttering detection, also called disfluency detection, has been facilitated by several publicly available data sets that include audio or video recordings of adults who stutter. FluencyBank [ 15 ] is a database that includes audio/video recordings from various participants, including children and adults who stutter (C/AWS). FluencyBank is a component of the broader TalkBank project, which serves as a comprehensive resource for studying fluency disorders. The stuttering data is associated with text transcripts with disfluency markers, and has been influential in facilitating research in automatic stuttering detection. However, inaccuracies in the temporal alignment of transcriptions and disfluency annotations have been noted [ 17 , 16 ] . The Stuttering Events in Podcasts (SEP-28k) dataset [ 16 ] contains over 28,000 clips labeled with five stuttering types: blocks, prolongations, sound repetitions, word repetitions, and interjections. The audio, sourced from public podcasts, features interviews with and by people who stutter. The data set was divided into 3-second audio segments, which are then annotated by nonclinical annotators, and the same annotation methodology was applied to the FluencyBank interview section. From a clinical perspective, these annotations are not suitable for stuttering assessment as they rely solely on short segments of audio by annotators who lack clinical expertise. UCLASS [ 18 ] provides audio recordings of persons who stutter along with orthographic transcriptions with time markers, and stuttering types were annotated by speech-language pathologists. While these annotations are more reliable that previous efforts, they lack the additional visual dimensions necessary for accurate assessment. Complementary efforts include the LibriStutter dataset [ 19 ] , which is an artificially generated dataset derived from the public LibriSpeech ASR corpus, and includes 5 stuttering types. Kassel State of Fluency (KSoF) [ 20 ] is a therapy-based dataset designed to monitor speech behavior over time, particularly for individuals who have undergone stuttering therapy. It includes over 5,500 clips of German speech labeled with six stuttering-related event types, including a unique category for speech modifications specific to therapy. While such resources are valuable for building robust automatic speech recognition systems that are inclusive of persons who stutter, they are limited in their clinical applicability. As described in the introduction, stuttering severity assessment in a therapeutic setting is a complex, multi-dimensional process that involves the identification of behavioral elements beyond the auditory; none of the existing annotations for stuttering include the full-range of clinical annotations needed for accurate stuttering severity assessment. 3 Annotation Methodology Stuttering moments, defined as interruptions in speech that represent the observable aspect of the stuttering disorder, can be classified based on frequency, duration, disfluency type, and the type/severity of secondary behaviors [ 2 ] . The combination of these quantitative behavioral measures (i.e., frequency, duration, and severity of secondary behaviors) is used to determine stuttering severity [ 5 , 13 ] . Frequency is typically expressed as the percentage of stuttered syllables, calculated by determining the total number of syllables produced by the speaker and identifying those in which stuttering occurs [ 5 ] . Duration refers to the amount