Speech-Based Prioritization for Schizophrenia Intervention
Gowtham Premananth, Philip Resnik, Sonia Bansal, Deanna L.Kelly, Carol Espy-Wilson · 2025
Read Speech-Based Prioritization for Schizophrenia Intervention by Gowtham Premananth, Philip Resnik, Sonia Bansal, Deanna L.Kelly, Carol Espy-Wilson (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-03086.txt.
Introduction
Premananth, Philip Resnik, Sonia Bansal, Deanna Kelly, and Carol Espy-Wilson rank people with schizophrenia by symptom severity using speech. The aim is to help clinics decide who needs care first. Many patients wait too long for assessment because expert time is scarce. Their 2025 model compares patients pairwise instead of scoring each alone, using articulatory and acoustic features drawn from recordings.
Pairwise wins feed a Bradley-Terry model, a standard way to turn comparisons into a ranking. On ranking measures, the approach beat earlier regression models that predicted severity scores directly. The authors frame the tool for triage in settings with few clinicians. It is a scalable, remote read of speech that puts the most severe cases first, without claiming to replace diagnosis.
Catalog record
| CREATOR | Gowtham Premananth, Philip Resnik, Sonia Bansal, Deanna L.Kelly, Carol Espy-Wilson |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-03086
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Speech-Based Prioritization for Schizophrenia Intervention Gowtham Premananth, Philip Resnik, Sonia Bansal, Deanna L.Kelly, Carol Espy-Wilson Abstract Millions of people suffer from mental health conditions, yet many remain undiagnosed or receive delayed care due to limited clinical resources and labor-intensive assessment methods. While most machine-assisted approaches focus on diagnostic classification, estimating symptom severity is essential for prioritizing care, particularly in resource-constrained settings. Speech-based AI provides a scalable alternative by enabling automated, continuous, and remote monitoring, reducing reliance on subjective self-reports and time-consuming evaluations. In this paper, we introduce a speech-based model for pairwise comparison of schizophrenia symptom severity, leveraging articulatory and acoustic features. These comparisons are used to generate severity rankings via the Bradley-Terry model. Our approach outperforms previous regression-based models on ranking-based metrics, offering a more effective solution for clinical triage and prioritization. † † address: 1 Institute for System Research, Department of Electrical and Computer Engineering, University of Maryland, College Park, USA 2 Institute for Advanced Computer Studies, University of Maryland, College Park, USA 3 School of Medicine, University of Maryland, USA 1 Introduction Mental health conditions affect millions globally [ 12 ] , affecting not only individuals but also the ones around them. While advancements in medical science have led to effective intervention strategies such as medication, therapy, and lifestyle modifications, timely and accurate assessment and diagnosis remain a critical challenge. Many individuals go undiagnosed or receive delayed care due to limited access to healthcare resources: in the U.S., more than a third of the population live in federally designated Mental Health Professional Shortage Areas [ 7 ] . This underscores the need for intelligent systems that can identify individuals requiring clinical intervention and assist healthcare professionals in prioritizing care. The rise of artificial intelligence (AI) and deep learning has led to the development of AI-driven early diagnosis systems [ 9 ] and remote healthcare monitoring solutions [ 2 ] . However, these models often require large, high-quality datasets, which are scarce in the medical field due to privacy constraints. This limitation is particularly challenging when the task extends beyond classification to severity estimation, which is essential for intervention prioritization. Among various AI-driven approaches, speech-based analysis offers a non-invasive, scalable, and cost-effective solution for mental health assessment. Speech carries valuable information related to cognitive, emotional, and neurological states, supporting AI based approaches as a promising tool for measuring symptoms severity in conditions like depression [ 18 ] , schizophrenia [ 15 ] , and post traumatic stress disorder [ 10 ] . Unlike traditional diagnostic methods, speech-based AI models enable automated, remote-friendly, and continuous monitoring, making early intervention more accessible, particularly in underserved regions. Most AI-based mental health studies focus on classification, determining whether an individual has a disorder or specific symptom [ 19 , 16 , 17 ] . However, in clinical practice, diagnosis alone is often insufficient, as patients with the same condition can vary widely in symptom severity and care needs. A more critical need is to rank individuals by psychiatric symptom severity to help clinicians prioritize those requiring urgent intervention. Estimating severity enables a more nuanced understanding of patient needs, allowing for better resource allocation and more timely treatment. Integrating severity assessment into AI systems moves beyond binary classification toward a dynamic, patient-centered model that improves overall care management. In this study, we introduce a speech-based approach for schizophrenia intervention prioritization using symptom severity. Using articulatory and acoustic speech features, we develop a pairwise comparison prediction model, whose outputs are fed into the Bradley-Terry model [ 5 ] to generate a severity ranking. By framing severity estimation as a ranking problem rather than a binary classification task, this method provides a more nuanced and clinically relevant assessment, aiding healthcare providers in prioritizing urgent cases and improving patient outcomes. The key contributions of this paper are: 1. A speech-based pairwise comparison prediction model leveraging Vocal Tract Variable(TV)-based articulatory features and pre-trained self-supervised acoustic features to determine which of two sessions exhibits higher severity. 2. A Bradley-Terry algorithm-based ranking framework that processes pairwise comparison predictions to rank sessions by symptom severity, aiding clinicians in prioritizing patients especially when resources are limited. Figure 1: Speech-based Pairwise comparison prediction model with acoustic and articulation fusion approach (The model architecture consists of only two branches: acoustic and articulatory. Inputs from both sessions are processed through these same branches. The figure shows four branches simply to illustrate that inputs from two sessions pass through the acoustic and articulatory branches, after which their outputs are fused to generate the pairwise comparison prediction.) 2 Method This section outlines the dataset, feature extraction, and the models and algorithms used in this study. 2.1 Dataset The dataset used in this study was collected through a collaborative effort between the University of Maryland School of Medicine and the University of Maryland, College Park, as part of research on assessing mental health disorders such as major depressive disorder and schizophrenia [ 11 ] . It consists of audio and video recordings of in-person, unstructured clinical interviews conducted over time, in which interviewers engaged subjects in open-ended conversations guided by their spontaneous responses rather than a predefined set of questions. In addition to these recordings, the dataset includes clinical evaluation scores obtained prior to each session. For this work, we focused on a subset of the dataset comprising 140 sessions from 39 unique subjects, including both healthy controls and subjects with schizophrenia. Only the audio recordings were used, and symptom severity was measured using evaluation scores based on the 18-item Brief Psychiatric Rating Scale (BPRS) [ 13 ] . The BPRS, widely regarded as a gold standard in psychiatry, provides a validated measure of symptom severity in schizophrenia. In the sessions used for this study, BPRS scores ranged from 19 to 62. 2.2 Feature extraction The audio recordings in the dataset consist of speech from both the interviewer and the subject. Therefore initially the speech from the session was diarized and then segmented into 40-second segments. The 40-second segments were used for articulatory and acoustic feature extraction. For articulatory features, a total of six Vocal Tract Variables (TVs) that represent the temporal and spatial movements of the articulators in the vocal tract (lips, tongue tip, and tongue body) were extracted using an acoustic-to-articulatory speech inversion system [ 3 ] . In addition to the TVs, articulatory source features aperiodicty and periodicity were extracted using an Aperioidicity Periodicity Pitch detector [ 6 ] . From the combination of TVs and source features, a Full Vocal Tract Coordination (FVTC) structure that portrays phasing between articulatory gestures was calculated using channel-delay correlations [ 8 ] . As previous work on speech-based schizophrenia severity estimation [ 14 ] has shown that concise articulatory representations extracted from FVTCs, using a Vector Quantized-Variational Auto Encoder (VQ-VAE) [ 20 ] based representation learning model has outperformed the use of stacked FVTCs, we also used concise articulatory representations as articulatory features for all our models. We extracted these concise representations from the same model introduced in [ 14 ] . Acoustic features were extracted using a pre-trained Wav2Vec2.0 [ 4 ] self-supervised model. 2.3 Model for speech-based pairwise comparison The pairwise comparison prediction model leverages both articulatory and acoustic representations to assess symptom severity. For