Multimodal Biomarkers for Schizophrenia: Towards Individual Symptom Severity Estimation

Gowtham, Philip, Sonia, Deanna, Carol · 2025

Read Multimodal Biomarkers for Schizophrenia: Towards Individual Symptom Severity Estimation by Gowtham, Philip, Sonia, Deanna, Carol (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16044.txt.

Introduction

Gowtham Premananth, Philip Resnik, Sonia Bansal, Deanna Kelly, and Carol Espy-Wilson argue that schizophrenia assessment should gauge symptom severity. It should not just detect the disorder. Deep learning studies most often treat schizophrenia as a task of sorting, as present or absent. That makes a complex condition too simple. It also limits clinical use, since care rests on which symptoms are severe.

The study shifts focus to gauges of severity for each symptom, with a multimodal path. It joins speech, video, and text inputs. It builds models for each mode on its own and a frame that combines them. To catch a more detailed symptom profile can sharpen diagnostic precision. It can also support care that is tailored to the person. The team casts the path as a scalable, objective tool for mental health work.

Catalog record

CREATORGowtham, Philip, Sonia, Deanna, Carol
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-16044

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multimodal Biomarkers for Schizophrenia: Towards Individual Symptom Severity Estimation Gowtham, Philip, Sonia, Deanna, Carol Abstract Studies on schizophrenia assessments using deep learning typically treat it as a classification task to detect the presence or absence of the disorder, oversimplifying the condition and reducing its clinical applicability. This traditional approach overlooks the complexity of schizophrenia, limiting its practical value in healthcare settings. This study shifts the focus to individual symptom severity estimation using a multimodal approach that integrates speech, video, and text inputs. We develop unimodal models for each modality and a multimodal framework to improve accuracy and robustness. By capturing a more detailed symptom profile, this approach can help in enhancing diagnostic precision and supports personalized treatment, offering a scalable and objective tool for mental health assessment. Premananth Resnik Bansal L.Kelly Espy-Wilson University of Maryland, College ParkUSA University of Maryland, College ParkUSA University of MarylandUSA keywords Schizophrenia, Symptom severity, Vocal tract variables † † email: [email protected], [email protected],[email protected], [email protected], [email protected] 1 Introduction Schizophrenia is a complex and heterogeneous mental health disorder that affects approximately 24 million people worldwide [ 1 ] . Despite its significant impact, many individuals with schizophrenia and other mental health conditions remain undiagnosed or experience delays in receiving appropriate treatment. Limited access to mental healthcare, particularly in under-served and rural areas, exacerbates this issue, as mental health specialists and resources are often scarce [ 2 ] . Even when care is accessible, the diverse and overlapping symptoms of schizophrenia pose a diagnostic challenge, frequently leading to misdiagnosis or prolonged delays in treatment. Traditional diagnostic approaches rely heavily on subjective clinical assessments, which can vary between practitioners and lack consistency. These challenges underscore the urgent need for accessible, scalable, and objective measures that can aid in symptom assessment by identifying reliable biomarkers and clinical subtypes that can capture pathological homology missed by conventional diagnostic assessments. Advancements in this area would support early and accurate identification and characterization of schizophrenia symptom profiles, ultimately yielding meaningful neurobiological and behavioral targets for treatments, etiological investigations and patient outcomes. One promising avenue for objective mental health assessment lies in multimodal analysis, integrating speech, video,and text-based inputs [ 3 , 4 , 5 ] . Recent research has demonstrated that speech patterns, facial expressions, and linguistic features extracted from transcripts can serve as valuable, non-invasive indicators of mental health conditions, particularly in disorders such as major depressive disorder [ 6 ] , post-traumatic stress disorder [ 7 ] , and schizophrenia [ 8 ] . Accordingly, these multimodal objective signals can be used in order to identify clinical subtypes that converge on symptom domains as assessed by more subjective clinical interviews. Speech-based methods analyze vocal patterns, acoustic features, and articulatory characteristics, while video analysis can capture facial expressions and gaze behavior indicative of underlying symptoms. Additionally, text from transcripts of natural speech or clinical interviews can provide insights into linguistic coherence, thought patterns, and semantic anomalies associated with schizophrenia. Integrating speech, video, and text-based modalities could improve the robustness and accuracy of symptom detection, allowing clinicians to tailor treatment strategies based on specific symptom severity. However, much of the existing research has framed these approaches as binary classification tasks—focusing on determining whether an individual has a disorder—rather than capturing the nuances of specific symptoms. Given symptomatology that schizophrenia is a highly heterogeneous disorder, shifting the focus toward estimating individual symptom profiles that are associated with objective through multimodal inputs could offer a more precise and clinically relevant approach, enabling more targeted interventions and personalized treatment strategies. While some studies have explored overall severity estimation [ 9 ] , this remains a relatively small subset of the field. A major challenge in multimodal assessment is that many mental health disorders share overlapping symptoms, making a simple classification or severity score insufficient for clinical decision-making. To enhance the practical utility of these models, the focus should shift toward individual symptom severity estimation, which provides a more granular understanding of a patient’s condition. One critical consideration in developing such systems is whether to use multiple models for different symptoms or a single unified model capable of estimating all symptoms simultaneously. A unified model offers several advantages. First, schizophrenia symptoms often co-occur and influence one another, meaning that a model trained to recognize all symptoms collectively may better capture these inter-dependencies. Second, training and deploying separate models for each symptom would be computationally expensive and inefficient, requiring significant resources for both development and real-time inference. Third, a single model ensures greater consistency in symptom estimation, reducing variability that might arise when combining predictions from multiple independent models. Lastly, integrating all symptoms into a unified framework allows for a more holistic and clinically meaningful assessment, aligning more closely with real-world diagnostic practices where clinicians evaluate symptoms in conjunction rather than in isolation. In this study we focus on doing an individual symptom severity estimation based on the symptoms exhibited by schizophrenia subjects. The main contributions of this study are: 1. Unimodal models using speech,video, and text inputs to estimate individual schizophrenia symptoms. 2. A multimodal framework involving speech,video, and text inputs to better perform individual schizophrenia symptoms estimation. 3. We analyze the extent to which different modalities capture specific symptoms, providing insights into the strengths and limitations of each modality in symptom assessment. By adopting a unified model for symptom estimation, this study aims to improve the precision and efficiency of schizophrenia assessment. The findings could contribute to the development of more objective, scalable, and clinically useful diagnostic tools, ultimately aiding in earlier detection and better-informed treatment planning. 2 Dataset The dataset used in this study is a dataset that was collected as a part of an interdisciplinary mental health assessment project jointly carried out by the University of Maryland School of Medicine and the University of Maryland - College Park [ 10 ] . The dataset contains English audio and video recordings of subjects who were interviewed during their in-person clinic visits. The dataset contains data from a total of 140 sessions produced by 39 different subjects. Before their interview sessions they were also evaluated based on mental health assessment questionnaires like Hamilton Depression rating scale (HAMD) [ 11 ] and Brief Psychiatric Rating Scale (BPRS) [ 12 ] to quantify their symptom severity before their sessions. In this study we are focusing our experiments based on the 18-item BPRS scores as they look into the psychiatric symptoms presented by the subjects which are good indicators for schizophrenia. In the 18-item BPRS scale all of these symptoms are individually rated on a scale of 1-7 that goes up with increasing severity of the particular symptom. For example, a score of 1 stands for symptom not present, a score of 4 meaning moderate severity, and a score of 7 stands for extremely severe symptom. Table. 1 gives a summary of the 18 symptoms and the frequency of the symptom scores across all the sessions in the dataset. Table 1: BPRS symptoms and the frequency of the symptom scores in the dataset 1 2 3 4 5 6 Somatic Concern 45 44 39 10 2 0 Anxiety 25 39 52 20 2 2 Guilt 82 33 20 4 0 1 Grandiosity 82 24 4 11 11 8 Depression 67

Read the complete text