Suicide Risk Assessment Using Multimodal Speech Features: A Study on the SW1 Challenge Dataset

Ambre, Ilias, Guillaume, Gwenolé · 2025

Read Suicide Risk Assessment Using Multimodal Speech Features: A Study on the SW1 Challenge Dataset by Ambre, Ilias, Guillaume, Gwenolé (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13069.txt.

Introduction

Ambre Marie, Ilias Maoudj, Guillaume Dardenne, and Gwenolé Quellec study suicide risk from speech in teens. Their paper, from twenty twenty-five, uses a shared data set from a speech wellness contest. They join the words a teen says with the sound of the voice. The goal is a tool that can flag risk early.

The team turns speech to text, then scores both text and sound with trained models. They also add hand-built voice traits, such as pitch. They test three ways to mix these streams of data. A weighted mix worked best, with fair scores on test data. A gap to new data shows the method still needs work.

Catalog record

CREATORAmbre, Ilias, Guillaume, Gwenolé
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE24 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-13069

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Suicide Risk Assessment Using Multimodal Speech Features: A Study on the SW1 Challenge Dataset Ambre, Ilias, Guillaume, Gwenolé Abstract The 1st SpeechWellness Challenge conveys the need for speech-based suicide risk assessment in adolescents. This study investigates a multimodal approach for this challenge, integrating automatic transcription with WhisperX, linguistic embeddings from Chinese RoBERTa, and audio embeddings from WavLM. Additionally, handcrafted acoustic features -including MFCCs, spectral contrast, and pitch-related statistics- were incorporated. We explored three fusion strategies: early concatenation, modality-specific processing, and weighted attention with mixup regularization. Results show that weighted attention provided the best generalization, achieving 69% accuracy on the development set, though a performance gap between development and test sets highlights generalization challenges. Our findings, strictly tied to the MINI-KID framework, emphasize the importance of refining embedding representations and fusion mechanisms to enhance classification reliability. MARIE MAOUDJ DARDENNE QUELLEC LaTIMUMR INSERM 1101, Brest, France Université de Bretagne OccidentaleBrest, France INSERMFrance keywords suicide risk assessment, multimodal learning, speech embeddings, linguistic analysis † † email: [email protected], [email protected], [email protected], [email protected] 1 Introduction Suicide is recognized as a significant global health issue, ranking among the leading causes of death in adolescents [ 1 , 2 ] . Early identification of individuals at risk is critical for timely intervention, yet reliable and accessible risk assessment remains a persistent challenge. Traditional suicide risk (SR) assessments are based primarily on clinical interviews and self-report questionnaires, both of which have significant limitations. Self-report assessments often lead to misclassification of suicidal behaviors, requiring follow-up questions to improve precision [ 3 ] . Clinical interviews, while more exhaustive, require substantial human expertise and resources, making them difficult to scale [ 4 ] . Additionally, the phrasing of interview questions can introduce bias, as negatively framed questions may lead patients to under-report suicidal ideation [ 5 ] . Speech analysis has been studied for its potential in classifying and predicting depression and suicidality, offering objective markers supporting clinical assessments [ 6 ] . On the linguistic level, suicidal individuals tend to use more absolutist words and exhibit dichotomous thinking, characterized by polarized and negative linguistic structures [ 7 , 8 ] . On the paralinguistic level, suicidal speech often demonstrates altered prosodic characteristics, including changes in pitch, rhythm, and speech fluency [ 9 ] . By analyzing linguistic and paralinguistic markers, speech-based systems can provide objective tools for SR evaluation, helping clinicians identify individuals at risk [ 10 , 11 ] . In response to this need, the 1st SpeechWellness Challenge was launched as part of Interspeech 2025. This challenge focuses on speech-based SR classification, providing a dataset of speech recordings from anonymized Chinese adolescents, with 50% identified as having SR based on validated psychological scales. In this work, we present a multimodal approach for assessing SR, integrating automatic transcription, text embeddings, audio embeddings, and acoustic feature extraction. We explore different fusion strategies for combining these modalities and evaluate their effectiveness. Our results provide insights into the strengths and limitations of multimodal learning for speech-based SR classification, and highlight areas for future research and improvement. 2 Materials and methods Each submission differs in its choice of embeddings and fusion strategies: (1) Uses only audio (WavLM) and text (RoBERTa) embeddings, with an early fusion approach; (2) Introduces acoustic features (MFCCs) alongside audio and text, with a modality-specific fusion strategy; (3) Expands acoustic embeddings by incorporating spectral and pitch features, while using a weighted attention fusion mechanism. Figure 1 provides an overview of the final processing pipeline, including transcription, embedding extraction, and fusion for classification. Figure 1: Architecture of the Submission 3 classification pipeline. 2.1 Dataset The dataset used in this study is the SW1 Challenge Dataset, provided as part of the 1st SpeechWellness Challenge at Interspeech 2025 [ 12 ] . It consists of speech recordings from 600 anonymized Chinese adolescents aged 10 to 18 years. Participants were categorized into two groups: 300 subjects diagnosed with SR and 300 non-risk subjects, based on clinical evaluations using the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID). Speech data was collected through interview tasks designed to capture linguistic and paralinguistic aspects of speech. Participants completed three tasks: an emotional regulation discussion, a standardized passage reading, and an image-based expression description. The dataset included 400/100/100 samples in the training (train), development (dev) and test sets respectively. The sets were balanced and the labels in the test set remain undisclosed, ensuring an unbiased final evaluation. In this study, we adhered to the original split of the dataset, using the train and dev sets for model development and relying on the test set for the final submission. 2.2 Transcription To incorporate both linguistic and acoustic information into the classification process, the audio recordings were transcribed into Mandarin text using the WhisperX [ 13 ] large-v2 model. By transcribing speech into structured text, we aimed to extract linguistic patterns that complement acoustic features. Previous studies have shown that multimodal models, which combine text and audio modalities, can outperform single-modal approaches by capturing a broader range of behavioral and emotional cues [ 14 ] . 2.3 Embeddings To build a thorough representation of speech data, embeddings were extracted from audio, text, and acoustic features. Each type captures distinct but complementary aspects of the recordings. The following sections describe the extraction process for each type of embedding. 2.3.1 Audio Embeddings To extract audio-based representations of speech, audio embeddings (AuE) were generated using WavLM [ 15 ] , a self-supervised model capturing high-level speech features relevant to emotion and classification. Three AuE strategies were tested for SR detection. In the first approach, WavLM Base+ was used to extract embeddings from non-overlapping 10-second chunks of speech. The embeddings were extracted from the last hidden state of WavLM, and the mean of all chunks was used as the final representation for each subject. The second approach employed WavLM Large, which has a greater number of parameters and a more expressive feature space. Windows with a 50% overlap were also introduced to maintain continuity in speech and prevent information loss at chunk boundaries. Overlapping embeddings were aggregated via mean pooling. The third approach allowed the model to capture short-term variations by reducing the chunk size to one second. While resulting in a higher dimensional representation, it provided a more detailed temporal view of speech features. Each of these embedding strategies offered different compromises between information granularity, computational efficiency, and context preservation. The objective was to identify which approach provided the most discriminative features to improve model performance. 2.3.2 Text Embeddings To encode the linguistic aspects of speech, text embeddings (TE) were extracted from the transcriptions using a transformer-based language model. These embeddings capture semantic and syntactic structures, allowing the model to differentiate between speech patterns associated with SR and those without. Given that the dataset consists of Mandarin transcriptions, pretrained Chinese RoBERTa [ 16 ] models were used to generate text representations. For embedding extraction, two RoBERTa versions were tested: the base (efficient) and the large (richer representation) versions. Each transcription was tokenized using the RoBERTa tokenizer, which splits the text into subword units. Then, the input was passed through the transformer model, generating contextualized word embeddings. Finally, the embeddings were aggregated into a single vector representation for each transcription using different pooling

Read the complete text