Beyond Binary: Speech Representations Across the Cognitive Score Hierarchy
Serli, Roshan Prakash, Christian, Lydia, Gerhard, Daniela, Sam, Paula Andrea, Kerstin · 2026
Read Beyond Binary: Speech Representations Across the Cognitive Score Hierarchy by Serli, Roshan Prakash, Christian, Lydia, Gerhard, Daniela, Sam, Paula Andrea, Kerstin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27189.txt.
Introduction
Kopar and co-authors test how speech signals line up with staged scores for mild cognitive impairment. They use 5754 German assessment recordings across six tasks and three score levels: task, domain, and global. Learned speech embeddings beat hand-built acoustic features at the lower levels, but that edge flips for impairment classification. Freer tasks lose signal as levels rise, while tightly set tasks gain, a split the authors read as specialist versus generalist patterns.
Catalog record
| CREATOR | Serli, Roshan Prakash, Christian, Lydia, Gerhard, Daniela, Sam, Paula Andrea, Kerstin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20592 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.27189
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Binary: Speech Representations Across the Cognitive Score Hierarchy Serli, Roshan Prakash, Christian, Lydia, Gerhard, Daniela, Sam, Paula Andrea, Kerstin Abstract This study examines the relationship between speech representations and the hierarchical structure of cognitive assessment in mild cognitive impairment. Utilizing 5,754 German neuropsychological assessment recordings, we evaluate six cognitive tasks across three score levels: task, domain, and global levels. We compare hand-crafted acoustic features with self-supervised learning (SSL) embeddings. Results show that although SSL representations generally outperform hand-crafted features at lower levels, this trend reverses for MCI classification. Furthermore, task-specific constraints influence performance: tasks with greater response freedom exhibit performance dilution as hierarchical levels increase, suggesting “specialist” representations, whereas the performance of highly structured tasks increases toward higher levels, suggesting “generalist” representations. These findings show links between task constraints and assessment hierarchy in automated clinical speech analysis. Kopar Rane Mychajliw Federmann Eschweiler Berg Gijsen Perez-Toro Ritter \usetikzlibrary shapes.geometric, arrows.meta, positioning keywords hierarchical cognitive assessment, mild cognitive impairment, neuropsychological test battery, clinical speech analysis † † address: 1 Hertie Institute for AI in Brain Health, University of Tübingen, Tübingen, Germany 2 Tübingen AI Center, University of Tübingen, Tübingen, Germany 3 Department of Psychology, Humboldt-Universität zu Berlin 4 Geriatric Center, Tübingen University Hospital, Tübingen, Germany 5 Tübingen Center for Mental Health (TüCMH), Department of Psychiatry and Psychotherapy, Tübingen University Hospital, Tübingen, Germany 6 German Center for Mental Health (DZPG), Partner Site Tübingen, Tübingen, Germany 7 Department of Neurology, University Medical Center Schleswig-Holstein and Kiel University, Kiel, Germany 8 Center for Neurology, University Hospital Tübingen and Hertie Institute for Clinical Brain Research, Tübingen, Germany 9 Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany 10 Charité–Universitätsmedizin, Department of Psychiatry and Psychotherapy, Berlin, Germany † † email: [email protected], [email protected], [email protected] 1 Introduction Mild cognitive impairment (MCI) is a clinical syndrome characterized by cognitive decline exceeding normal aging [ 19 , 23 ] . As a prodromal stage of dementia, most commonly Alzheimer’s disease, it represents a critical window for early intervention [ 1 ] . Despite its clinical relevance, MCI remains substantially underdiagnosed [ 3 ] . Clinical diagnosis typically relies on standardized neuropsychological assessments. The Consortium to Establish a Registry for Alzheimer’s disease (CERAD+) is a well-validated, multi-domain neuropsychological battery assessing language, memory, executive function, and visuospatial abilities [ 16 ] , generating structured task-, domain-, and global-level scores. In routine practice, shorter instruments such as the Mini-Mental State Examination (MMSE) are frequently used for efficient screening. Together, these instruments form the backbone of clinical cognitive assessment. Automated speech analysis has emerged as a promising complement to traditional testing [ 13 ] . However, current approaches face three methodological bottlenecks: (i) focusing on binary classification (Alzheimer’s Disease vs. healthy controls (HC)), which lacks sensitivity to the subtle cognitive change characteristics of MCI [ 15 ] ; (ii) reliance on English-centric, single-task datasets, limiting generalizability [ 10 ] ; and (iii) modeling clinical scores as independent, flat targets, thereby ignoring the hierarchical structure inherent to standardized cognitive assessment. In this paper, we address these gaps using 5,754 recordings collected during five CERAD+ tasks and one MMSE screening task from an elderly German cohort. Beyond binary classification, we model the hierarchical organization of clinical scores, linking acoustic features to task-, domain-, and global-level scores. This enables a fine-grained analysis of the relationship between speech and cognitive decline across multiple diagnostic and screening tasks. Our analysis reveals a task-dependent pattern: for tasks with more open-ended responses, the predictive power of acoustic features decreases from task-level to domain-level and global scores, whereas for more constrained tasks, it increases across these levels. To the best of our knowledge, this is the first study to examine how acoustic feature predictiveness varies across hierarchical levels of the German CERAD+ battery in the context of MCI. To support future research, we publicly release our code. 1 1 1 https://github.com/anon-interspeech/anon-interspeech-2026.git Figure 1: Our workflow for predicting hierarchical cognitive scores from speech. Using speech-derived acoustic features, independent models predict targets at three levels: (1) Level 1 (Individual Tests) : task-level scores (e.g., Phonemic Fluency (PF), the number of valid words beginning with “S”, with scores ranging from [0,\infty) ); (2) Level 2 (Cognitive Domains) : domain-level composite scores with speech-to-drawing task ratios of 1:0 (Language, LAN), 1:1 (Memory, MEM), and 0:1 (Executive, EXE; Visuospatial, VIS) [ 21 ] ; and (3) Level 3 (Global Status) : global-level scores including the CERAD+ total score (modeled as continuous and binary with a threshold of 85) [ 7 ] and binary MCI status (defined as more than 1.5 standard deviations below the normative cohort) [ 2 ] . Arrows denote workflow from shared acoustic feature representations to independent models predicting hierarchical targets. 2 Methods 2.1 Dataset and Quality Control We used speech recordings from the TREND study 2 2 2 All participants provided written informed consent. The study was approved by the Ethics Committee of (anonymized) [ 26 ] , comprising one MMSE screening task and five CERAD+ diagnostic tasks: Word List Recognition (RW), Boston Naming Test (BNT), Word List Recall (RL), Verbal Fluency (VF), and Phonemic Fluency (PF). Our analysis follows the inherent three-level hierarchy of clinical assessment, as summarized in Fig. 1 : Level 1 (Individual Tests) comprises raw task-level scores (e.g., PF: number of valid words beginning with “S”). Level 2 (Cognitive Domains) aggregates these task-level scores into domain-level composite measures [ 21 ] , including Language (LAN), Memory (MEM), Executive Function (EXE), and Visuospatial Ability (VIS). These domains differ in their reliance on verbal versus drawing-based tasks (LAN: 1:0; MEM: 1:1; EXE and VIS: 0:1). Although EXE and VIS are traditionally assessed through drawing-based tasks, we evaluate the cross-domain predictive performance of speech by using acoustic features from verbal tasks to predict these non-verbal targets. This tests whether speech contains information that generalizes beyond verbal cognitive domains (MEM, LAN). Level 3 (Global Status) includes the global-level scores: CERAD+ total score (continuous and thresholded at 85) [ 7 ] and clinical MCI status ( >1.5 SD below demographically adjusted normative cohort) reported by Berres et al. [ 2 ] . To ensure diagnostic and acoustic integrity, we excluded non-native speakers, incomplete profiles, and MCI-to-HC reverters. Additionally, we performed acoustic quality assessment, enforcing constraints on duration (minimum 15 s), energy (RMS >-55 dBFS), digital clipping ( <1.5\% ), and signal-to-noise ratio (SNR >10 dB), estimated using reference-free, quantile-based methods [ 12 , 20 , 17 ] . Conditional inconsistencies between metrics (e.g., high speech activity ratio with low SNR) were manually reviewed. These filtering steps yielded 959 sessions (698 HC, 261 MCI) from 593 participants. 2.2 Preprocessing and Diarization Optimization To find the optimal preprocessing hyperparameters, we used a manually transcribed ground-truth subset ( N=89 ; \approx 9\% of the corpus, available only in two fluency tests: Phonemic (PF) and Verbal (VF)). We conducted a grid search of >2{,}500 combinations using participant-disjoint tuning and validation splits. Hyperparameter tuning was based on diarization and Jaccard error rates (DER, JER), purity (PUR), and coverage (COV), with a 250 ms collar [ 5 ] . The resulting pipeline applied a 6th-order Butterworth high-pass filter ( f_{c}=100 Hz) [ 6 ] , spectral-gating noise suppression ( \alpha=0.3 ) [ 4 ] , and loudness normalization ( -23 LUFS) [