Speech as a biomarker for supported diagnosis of major depressive disorder using self-supervised representations Authors: Yunhan Lin, Biman Najika Liyanage, Chuan Shi, Zhengwen Zhu, Yutao Sun, Yaoyao Sun, Ziwei Sunny Zhang, Jun Yang, Chenyang Xu, Yundan Liao, Yanbao Tao, Zongfeng Li, Guorui Zhao, Weihua Yue Published: 2026 ABSTRACT The diagnosis of major depressive disorder (MDD) urgently requires objective biomarkers for clinical translation. In this study, we established a multicenter cohort to date (N = 1,816; comprising 910 MDD patients and 906 healthy control subjects), using 23,608 standardized speech samples. Based on 6,373 acoustic-prosodic features, we develop a deep learning framework that employs a self-supervised architecture to leverage speech biomarkers. We perform a systematic comparative analysis among pretrained foundation models, including WavLM and HuBERT, and traditional acoustic features extracted from openSMILE. Model performance is evaluated using the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. Our framework achieves an AUC of 0.932 in internal validation (n = 333), significantly outperforming conventional methods, while maintaining efficacy in external validation (n = 160, AUC = 0.879). Self-supervised representations demonstrate robust diagnostic accuracy compared to other models. Leveraging the large speech biomarker dataset, our findings provide a rapid, cost-effective, and non-invasive approach for assisted depression assessment. Introduction In the post-pandemic era, depressive disorders have become increasingly prevalent. Although their incidence varies among countries1, approximately 20% of individuals are estimated to experience depressive disorders at some point in their lives2, with a rising trend observed globally3. The World Health Organization (WHO) predicts that by 2030, depressive disorders will emerge as the leading global cause of disease burdeny4. Major depressive disorder not only affects a significant number of people but also imposes a substantial economic cost, underscoring the critical need for early diagnosis and treatment. This is particularly urgent in low- and middle-income countries, where healthcare resources are often limited. In current clinical practice, the diagnosis of depression primarily relies on patients’ self-reporting and professional clinical evaluations. However, this approach faces several challenges, including (but not limited to) a shortage of mental health professionals5,6, inaccuracies in primary care settings7, time-consuming evaluation processes, the social stigma that leads to concealed symptoms, and hesitancy to seek medical help. These limitations highlight an urgent need for novel, objective, and user-friendly diagnostic methods. Fortunately, advances in scientific research and technology have led to the discovery of several new biomarkers8–16 in recent years, offering potential targets for depression screening. Among emerging digital phenotyping approaches, speech features stand out as a promising biomarker category due to their advantages of being rapid, non-invasive, privacy-preserving, and objective. The field of speech-based depression detection has evolved significantly since its inception. The earliest systematic recognition of depression-related vocal changes can be traced back to 1921, when Emil Kraepelin observed that patients with depression often exhibited more monotonous speech, reduced sound intensity, and slower speech rate17. These seminal observations laid the theoretical foundation for subsequent scientific investigations. Building upon these early insights, initial research primarily focused on handcrafted acoustic features. These pioneering studies identified unique expression patterns in the speech of individuals with depression18–23, establishing pitch, intensity, speech rate, pauses, and spectral characteristics as crucial biomarkers for depression detection. With advances in computational methods, the advent of machine learning marked a transformative period in the field. Williamson et al. 24,25 achieved a significant milestone by demonstrating the feasibility of depression severity using acoustic features. During this era, traditional machine learning approaches relied heavily on handcrafted features, including prosodic, spectral, voice quality, and linguistic measures. The main classifiers employed during this period were Support Vector Machines (SVM)26, Random Forest (RF)27,28, and Gaussian Mixture Models (GMM)29. The rapid advancement of artificial intelligence, particularly the rise of deep learning revolutionized speech-based depression detection by providing diverse neural network architectures to handle complex speech patterns. Convolutional neural networks (CNN) excelled at processing speech spectrograms and automatically learning local spectral patterns30. Complementing this approach, recurrent neural networks (RNN) and their variants, notably long short-term memory networks (LSTM) networks and gated recurrent units (GRU), proved especially effective at modeling temporal characteristics of speech31,32. Building on these developments, Harati et al.33 made important contributions through encoder-based transfer learning, establishing technical foundations for subsequent research. Most recently, recent advances in self-supervised learning and foundation models have brought transformative changes to the field. These models are pre-trained on large-scale, unlabeled speech data to learn rich speech representations, which are then fine-tuned for specific tasks. Key developments in this area include Wav2Vec34, HuBERT35, WavLM36, and Whisper37. Demonstrating the practical value of these innovations, Campbell et al. 38 leveraged deep learning-based pre-trained models to extract speech representation features, successfully applying them to depression severity classification. Riad et al. 39 further showed that automated speech analysis could achieve clinically relevant performance across multiple mental health conditions. Expanding on this foundation, more recent studies by Gómez-Zaragozá40 et al. and Maji et al. 41 have explored cross-task and cross-language applications of large-scale foundation models for depression detection. Additionally, Dumpala et al. 42 proposed methods for detecting individual depression symptoms using self-supervised embeddings. Collectively, these developments demonstrate that modern self-supervised models are capable of capturing subtle acoustic patterns that traditional handcrafted features might miss, providing a powerful and robust technical foundation for speech-based depression detection. Despite these promising developments, a significant gap persists in translating speech-based depression detection into clinical practice. Most existing studies are limited by small sample sizes, single-site data collection, and a lack of robust external validation using independent clinical populations40,43. Furthermore, there is a clear need for robust models that can reliably differentiate individuals with and without depression across various contexts and settings. To address these critical gaps, we developed and validated a comprehensive speech-based screening system that leverages self-supervised learning representations. Our study presents a novel deep-learning-based model designed to diagnose depression using self-supervised features extracted from speech segments. Results Demographic characteristics Our study was conducted in two distinct phases, as illustrated in Fig. 1a. In the discovery phase, data were collected from 811 patients with depression and 845 individuals without depression. Eighty percent of this cohort (1323 participants) was used for model development, with the remaining 20% (333 participants) set aside for internal validation. For the subsequent external validation phase, a separate cohort of 123 outpatients with mood disturbances was evaluated. Of these, 80 patients diagnosed with major depressive disorder were included as the primary external validation group, while 43 were excluded according to predefined criteria. An additional comparison group of 94 individuals, including 80 individuals without depression, was assembled, with 14 samples excluded. This resulted in a final external validation dataset comprising a total of 160 participants.Fig. 1Study design and model architecture.a Flowchart of the two-stage study design. The discovery cohort (n = 1323) was split into a training set (80%) and an internal validation set (20%). The external validation cohort (n = 160) consisted of outpatients with mood disturbances and individuals without depression from independent clinical centers. b Diagram of the depression diagnosis process using voice recordings, implemented through a suite of deep learning models, and a schematic of the deep learning framework. process used to diagnose depression from voice recordings employing a suite of deep learning models and a schematic of the deep learning framework. The extracted features are concatenated with gender embeddings derived from a fine-tuned wav2vec model and then input into an adversarial training pipeline to isolate depression-related paralinguistic features from speaker-specific characteristics. Source data are provided as a Source Data file. Study design and model architecture. a Flowchart of the two-stage study design. The discovery cohort (n = 1323) was split into a training set (80%) and an internal validation set (20%). The external validation cohort (n = 160) consisted of outpatients with mood disturbances and individuals without depression from independent clinical centers. b Diagram of the depression diagnosis process using voice recordings, implemented through a suite of deep learning models, and a schematic of the deep learning framework. process used to diagnose depression from voice recordings employing a suite of deep learning models and a schematic of the deep learning framework. The extracted features are concatenated with gender embeddings derived from a fine-tuned wav2vec model and then input into an adversarial training pipeline to isolate depression-related paralinguistic features from speaker-specific characteristics. Source data are provided as a Source Data file. This study utilizes real-world clinical data. As shown in Supplementary Fig. 1, the distributions of age and sex differed significantly (p < 0.05) between the depression and healthy control groups, a finding consistent with previous epidemiological reports on depression. In the external validation set, statistical analyses showed no significant between-group differences in BMI, ethnicity, or lifestyle habits (Supplementary Table 1). Voice feature analysis We analyzed 6373 features extracted using the openSMILE toolkit across three distinct datasets to compare their values between different emotion groups. The average mean effect size for each feature was calculated using the weighted mean effect size. This analysis identified 26 features that effectively distinguished between emotion groups across all three feature sets, exhibiting significant discriminative power (adjusted p < 0.05, effect size > 0.5). The top 20 features are displayed in Fig. 2a. MFCC features represented the largest group (9 features), predominantly consisting of delta-coefficient statistics, including quartiles, interquartile ranges, and kurtosis measures across various frequency bands. Auditory spectrum features formed the second major category, which was further subdivided into filtered spectral characteristics and L1-norm-based spectral measures. Among these, a spectral measure capturing the standard deviation of peak distances in L1-normalized auditory spectrum (corresponding to the openSMILE feature “audspec_lengthL1norm_sma_peakDistStddev”) showed the largest effect size in distinguishing depressed from healthy individuals (Weighted Mean Effect Size = 0.749). Additionally, RMS energy-based features (2 features) contributed to the set of discriminative features.Fig. 2Population-level analysis of voice features.a Performance of Features in Emotion Detection Across Multiple Datasets: The top 20 features that distinguishing emotional groups show effect sizes in the training set (red circles), internal validation set (green squares), and external validation set (blue triangles). Diamonds indicate the weighted mean effect sizes across all three datasets. All features demonstrated significant discriminative power (p-adjusted <0.05) and consistent performance across independent validation cohorts. Features are ranked 1-20 by the magnitude of their weighted mean effect size. Effect sizes were calculated using Cohen’s d. Statistical significance was assessed using two-sided independent t-tests or Mann-Whitney U tests (depending on normality and variance homogeneity), with false discovery rate (FDR) correction using the Benjamini-Hochberg method. b Performance of Feature in Gender Classification Across Multiple Datasets: the top 20 gender-discriminative features show effect sizes in the training set (red circles), internal validation set (green squares), and external validation set (blue triangles). Diamonds represent weighted mean effect sizes across all three datasets. Features primarily from MFCC spectral domain exhibited significant ability to classify gender. Statistical methods are the same as in (a). c Age Distribution Analysis Across Demographic Strata: Scatter plots with marginal histograms showing the relationship between participant age (x-axis, 18–65 years) and feature values (y-axis) for three representative acoustic features. Data are shown for all participants (purple, left), male participants (blue, middle), and female participants (red, right). Marginal histograms depict the distributions of age (top) and feature values (right). Source data are provided as a Source Data file. Population-level analysis of voice features. a Performance of Features in Emotion Detection Across Multiple Datasets: The top 20 features that distinguishing emotional groups show effect sizes in the training set (red circles), internal validation set (green squares), and external validation set (blue triangles). Diamonds indicate the weighted mean effect sizes across all three datasets. All features demonstrated significant discriminative power (p-adjusted <0.05) and consistent performance across independent validation cohorts. Features are ranked 1-20 by the magnitude of their weighted mean effect size. Effect sizes were calculated using Cohen’s d. Statistical significance was assessed using two-sided independent t-tests or Mann-Whitney U tests (depending on normality and variance homogeneity), with false discovery rate (FDR) correction using the Benjamini-Hochberg method. b Performance of Feature in Gender Classification Across Multiple Datasets: the top 20 gender-discriminative features show effect sizes in the training set (red circles), internal validation set (green squares), and external validation set (blue triangles). Diamonds represent weighted mean effect sizes across all three datasets. Features primarily from MFCC spectral domain exhibited significant ability to classify gender. Statistical methods are the same as in (a). c Age Distribution Analysis Across Demographic Strata: Scatter plots with marginal histograms showing the relationship between participant age (x-axis, 18–65 years) and feature values (y-axis) for three representative acoustic features. Data are shown for all participants (purple, left), male participants (blue, middle), and female participants (red, right). Marginal histograms depict the distributions of age (top) and feature values (right). Source data are provided as a Source Data file. In light of the known variations in gender and age distribution across different emotion groups in real-world settings, we further evaluated the influence of gender on acoustic features across all datasets. A total of 510 features demonstrated a significant ability to distinguish between gender groups across all three feature sets (adjusted p < 0.05, effect size > 0.5). The top 20 discriminative features are presented in Fig. 2b, with a measure summarizing the typical (median) speech pitch (corresponding to the openSMILE feature “F0final_sma_quartile2”) showing the largest effect size. Notably, features derived from fundamental frequency and the MFCC category, which characterize the spectral shape of the speech signal—including those corresponding to the 10th, 13th, and 14th MFCC components (mfcc_10, mfcc_13, and mfcc_14), also exhibited strong discriminative performance. MFCC features model the spectral characteristics of human auditory perception, where higher-order coefficients capture fine-grained spectral details relevant to distinguishing between male and female vocal traits. Importantly, our comparative analysis reveals significant overlap between the features that varied across emotion groups and those that distinguish gender groups. This overlap suggests that gender functions as a confounding factor in the relationship between vocal features and emotional expression. Consequently, these findings underscore the importance of accounting for gender during the training of models designed to utilize speech characteristics for emotional and psychological analyses. Finally, when examining the influence of age on feature values, we observed only a weak correlation (Fig. 2c). Specifically, only three features exhibited an absolute correlation coefficient exceeding 0.3. Among these, the strongest association was observed for a measure quantifying the proportion of time during which the voice is strongly present in the signal (corresponding to the openSMILE feature “voicingFinalUnclipped_sma_upleveltime90”), which showed a negative correlation coefficient of −0.330. Model performance This study identified consistent acoustic differences between individuals with depression and a control group across three datasets, as illustrated in Fig. 3a. These distinctions were captured by specific speech features spanning spectral, prosodic, voicing, and temporal domains. Key discriminative features captured complementary aspects of the speech signal, including lower‑quartile dynamics of MFCCs, upper‑quartile energy changes in the auditory spectrum, asymmetry in the distribution of speech pitch, variability in the strength of voicing, and the proportion of time with rapid zero‑crossings reflecting high‑frequency, noise‑like components. Together, these features showed substantial discriminative power in distinguishing patients with depression from individuals without depression.Fig. 3Model performance.a SHAP dependence plots of the top 10 acoustic features ranked by mean absolute SHAP value, as observed in the training set (left), internal validation set (center), and external validation set (right). Each point corresponds to an individual sample, colored by feature (pink: high, blue: low), with the SHAP value on the x-axis indicating impact on model output for depression classification. b Performance comparison between the CatBoost-openSMILE Model and the Whisper-Emb Model on both internal and external validation datasets. Error bars represent 95% confidence intervals. For the Whisper-Emb model, CIs were calculated using bootstrap resampling (n = 1000); for the CatBoost-openSMILE model, CIs were derived from 10-fold cross-validation results. c ROC curves for the Whisper-Emb, HuBERT-Emb, and WavLM-Emb models on the external validation set (left) and internal validation set (right). Area under the curve (AUC), sensitivity, and specificity are annotated at optimal cutoff points. Source data are provided as a Source Data file. Model performance. a SHAP dependence plots of the top 10 acoustic features ranked by mean absolute SHAP value, as observed in the training set (left), internal validation set (center), and external validation set (right). Each point corresponds to an individual sample, colored by feature (pink: high, blue: low), with the SHAP value on the x-axis indicating impact on model output for depression classification. b Performance comparison between the CatBoost-openSMILE Model and the Whisper-Emb Model on both internal and external validation datasets. Error bars represent 95% confidence intervals. For the Whisper-Emb model, CIs were calculated using bootstrap resampling (n = 1000); for the CatBoost-openSMILE model, CIs were derived from 10-fold cross-validation results. c ROC curves for the Whisper-Emb, HuBERT-Emb, and WavLM-Emb models on the external validation set (left) and internal validation set (right). Area under the curve (AUC), sensitivity, and specificity are annotated at optimal cutoff points. Source data are provided as a Source Data file. To establish a performance baseline, we first evaluated a demographics-only model that utilized only age and gender information. This baseline model achieved AUCs of 0.647 on internal validation and 0.711 on external validation. As shown in Fig. 3b, the Whisper-Emb Model demonstrates consistently higher performance and greater stability across both internal and external validation datasets compared to the CatBoost-openSMILE Model. Specifically, on internal validation, Whisper-Emb Model shows a 9.4% higher F1-score, a 11.7% increase in sensitivity, a 6.0% improvement in specificity, and a 8.3% higher AUC compared to CatBoost-openSMILE Model. On external validation, it maintains this advantage, with an 10.5% higher F1-score, a 13.6% increase in sensitivity, a 4.9% improvement in specificity, and a 6.2% higher AUC. Among the self-supervised learning models, Whisper-Emb Model demonstrated superior performance in depression detection through speech analysis, outperforming both the WavLM-Emb Model and HuBERT-Emb Model across internal and external validation settings (Fig. 3c). It achieved an AUC of 0.932 for the internal validation and 0.879 for the external validation, notably surpassing the other models. Detailed robustness metrics are provided in Supplementary Data 1. In contrast, both the WavLM-Emb Model and HuBERT-Emb Model exhibited lower performance on the external validation set compared to the internal validation set, with DeLong test p-values of 0.007 and 0.001, respectively. The diagnostic advantage of the Whisper-Emb Model was further underscored by its significantly higher AUC relative to the other models (p = 0.001). During the external validation, Whisper achieved a sensitivity of 82.5% [CI 0.742, 0.908] and specificity of 83.8% [CI 0.757, 0.918]. Additional details are available in Supplementary Tables 2 and 3. Based on the superior performance of the Whisper-Emb Model compared to all other evaluated models, we further assessed its clinical utility through calibration and decision curve analyses using the external validation cohort. The model demonstrated good calibration (Brier score = 0.144) and offered substantial clinical net benefit, confirming its potential for clinical screening applications. Detailed results are provided in Supplementary Note 1. Subgroup and sensitivity analyses To evaluate how different experimental paradigms affect the performance of speech-based models, we analyzed their accuracy across multiple dimensions during both internal and external validation. Initially, to investigate the effect of audio length on model performance, we used audio segments ranging from 20 to 120 s in 20-s increments. Performance metrics were averaged over 10 seeds, with each model trained for 100 epochs. The results, presented in Fig. 4a, show that model performance improved as the speech duration increased and began to plateau after 80 s. This suggests that at least 80 s of audio are needed to capture stable acoustic signals for depression detection. Longer speech segments appear to enhance the detectability of the psychomotor retardation (a clinical term describing slowed motor and cognitive activity) in speech acoustic features, which remain consistent over time and serve as a stable biomarker for depression screening. These findings align with prior studies examining speaking duration, where longer audio has also been associated with improving model performance44,45,.Fig. 4Performance analysis of depression detection models.a Line plots illustrating the influence of audio duration (20 s, 40 s, 80 s, 120 s) on classification performance (AUC, sensitivity, and specificity) for the Whisper-based model in internal validation (left) and external validation (right). b ROC curves comparing model performance on internal (green) and external (pink) validation sets using exclusively open-ended questions (left) and close-ended questions (right), with sensitivity, specificity, and AUC values annotated. c ROC curves for internal (green) and external (pink) validation, stratified by the emotional valence of questions: negative (left), neutral (middle), and positive (right). Each panel reports the corresponding sensitivity, specificity, and AUC values. Source data are provided as a Source Data file. Performance analysis of depression detection models. a Line plots illustrating the influence of audio duration (20 s, 40 s, 80 s, 120 s) on classification performance (AUC, sensitivity, and specificity) for the Whisper-based model in internal validation (left) and external validation (right). b ROC curves comparing model performance on internal (green) and external (pink) validation sets using exclusively open-ended questions (left) and close-ended questions (right), with sensitivity, specificity, and AUC values annotated. c ROC curves for internal (green) and external (pink) validation, stratified by the emotional valence of questions: negative (left), neutral (middle), and positive (right). Each panel reports the corresponding sensitivity, specificity, and AUC values. Source data are provided as a Source Data file. We also compared model performance between open-ended and closed-question scenarios using the external validation cohort. As shown in Fig. 4b, models evaluated with open-ended questions demonstrated higher sensitivity (77.5% vs. 70.0%) but lower specificity (73.8% vs. 78.7%) compared to those tested with closed questions, while their AUC values remained comparable (0.831 vs. 0.848). This indicates that the overall accuracy and sensitivity of the open-question model were not significantly different from those of the closed-question model (p = 0.606). In a further analysis, we evaluated how different emotional stimuli affect model performance. As illustrated in Fig. 4c, models trained exclusively on negative emotional stimuli showed a sensitivity of 77.5%, a specificity of 73.8%, and an AUC of 0.828. By contrast, models trained solely on positive stimuli yielded a sensitivity of 70.0%, a specificity of 78.7%, and an identical AUC of 0.840. Models trained on neutral stimuli produced a sensitivity of 72.5% and a specificity of 76.2%, and an AUC of 0.838. The overlapping confidence intervals for sensitivity and specificity across different emotional conditions suggest that the type of emotional stimulus has a comparable effect on the model performance. Furthermore, the Delong test p-value indicates that the emotional nature of the stimuli does not significantly influence detection efficacy. Additional details are provided in Supplementary Tables 4 and 5. To evaluate model performance across demographic subgroups, we stratified the external validation cohort by gender and age. The analysis indicated strong fairness, with Disparity Ratios of 0.973 and 0.949, respectively (detailed results in Supplementary Note 2). Furthermore, we assessed the model across different depression subtypes, including those with melancholic features, anxiety, or suicide risk. No significant difference in accuracy was observed among these groups (p > 0.5). This consistency across subtypes underscores the robustness of speech biomarkers in detecting depression, regardless of specific depressive characteristics. Additional details are provided in Supplementary Table 6. Evaluation of depression severity prediction In this study, we investigated the ability of our model to accurately predict depression severity using only patient speech recordings. Leveraging an external dataset with available clinical assessments, including the Hamilton Depression Rating Scale (HAMD), the Hamilton Anxiety Rating Scale (HAMA), the Zung Self-Rating Depression Scale (SDS) and PHQ-9, we compared these clinically derived scores with severity predictions generated by our model. This comprehensive analysis enabled us to assess the correlations between model predictions and both HAMD and self-reported (SDS and PHQ-9) depression severity. As shown in Fig. 5, model predictions demonstrated a moderate correlation with HAMD scores (r = 0.662, 95% CI: 0.563–0.747, p < 0.001). The mean absolute error was 10.746 (95% CI: 9.340–12.150), and Bland-Altman analysis revealed a mean bias of 10.709 with 95% limits of agreement ranging from −7.425 to 28.843. These differences largely reflect the expected scale discrepancy between model output probabilities (0-1) and HAMD scoring range (0-52, see Supplementary Fig. 2). Compared with self-reported scales, our model’s severity predictions correlated more strongly with clinician-administered measures such as HAMD and HAMA than with self-reported instruments like PHQ-9 and SDS.Fig. 5Correlation between clinical scale scores and depression prediction scores.Scatter plots and corresponding density distributions illustrate the relationship between depression prediction scores (derived from the proposed model, with higher scores indicating greater likelihood of depression) and four clinical scales: a HAMD (Hamilton Depression Rating Scale), b HAMA (Hamilton Anxiety Rating Scale), c SDS (Self-Rating Depression Scale), and d PHQ-9 (Patient Health Questionnaire-9). The leftmost column displays linear regression plots for all participants, where blue and red points represent depressed and healthy individuals, respectively. Data are presented as regression lines (mean fitted values from ordinary least squares regression) with shaded areas representing standard error. Correlations were assessed using two-sided Spearman correlation tests; 95% confidence intervals for correlation coefficients were calculated using bootstrap resampling (n = 1000). Exact p values are shown in each panel. The middle columns present joint density plots for the entire cohort and for male participants, while the rightmost column shows the data for female participants. Marginal histograms depict the score distribution. Spearman’s correlation coefficient (r), p-value, and 95% confidence interval are reported within each scatter plot. Source data are provided as a Source Data file. Correlation between clinical scale scores and depression prediction scores. Scatter plots and corresponding density distributions illustrate the relationship between depression prediction scores (derived from the proposed model, with higher scores indicating greater likelihood of depression) and four clinical scales: a HAMD (Hamilton Depression Rating Scale), b HAMA (Hamilton Anxiety Rating Scale), c SDS (Self-Rating Depression Scale), and d PHQ-9 (Patient Health Questionnaire-9). The leftmost column displays linear regression plots for all participants, where blue and red points represent depressed and healthy individuals, respectively. Data are presented as regression lines (mean fitted values from ordinary least squares regression) with shaded areas representing standard error. Correlations were assessed using two-sided Spearman correlation tests; 95% confidence intervals for correlation coefficients were calculated using bootstrap resampling (n = 1000). Exact p values are shown in each panel. The middle columns present joint density plots for the entire cohort and for male participants, while the rightmost column shows the data for female participants. Marginal histograms depict the score distribution. Spearman’s correlation coefficient (r), p-value, and 95% confidence interval are reported within each scatter plot. Source data are provided as a Source Data file. Discussion Given low public awareness of mental illnesses and the associated stigma, individuals experiencing mental health symptoms frequently face delays in diagnosis and often first seek care in general medical settings. This pattern contributes to increased healthcare costs46–48 prior to receiving appropriate psychiatric ealuation49–53. In our external validation cohort, the prevalence of MDD aligned closely with rates reported in previous epidemiological studies, supporting the representativeness of our sample54–56. These real-world challenges highlight the pressing need for developing accessible, objective tools for depression screening, which would promote early identification and timely intervention. On the one hand, SHAP analysis of our traditional model identified key features across spectral, prosodic, voicing, and temporal domains, consistent with previous findings31,32,57–67. These acoustic patterns point to several underlying mechanisms in depressed speech. Specifically, spectral features indicated reduced articulation and lower energy levels in individuals with depression, suggesting a constricted vocal tract and diminished speech activity, which are commonly associated with psychomotor retardation19,68–71. The prosodic feature reflected more monotone or irregular speech patterns, likely resulting from altered vocal cord tension or increased pauses58,72,73. Voicing feature irregularities corresponded with slowed cognitive and motor processing, while temporal features revealed slower speech rates, further underscoring the role of psychomotor retardation. On the other hand, the superior performance of Whisper-Emb Model can likely be attributed to its extensive pretraining on diverse speech data, which enhances its robustness and generalizability across populations74–76. In contrast, the pronounced limitations of the CatBoost-openSMILE Model in cross-population performance, particularly in sensitivity, underscore the need for task-specific optimization or regularization strategies in traditional feature-based approaches. Building on these findings, we developed several AI models to augment clinical practice by offering efficient and objective screening tools. These tools are designed to assist healthcare professionals in identifying patients who may require further psychiatric evaluation. Powered by deep learning, the resulting screening applications are non-invasive, require no waiting time, demand minimal technical expertise, and remain cost-effective while achieving high accuracy for preliminary assessment. Furthermore, by examining how various experimental factors and model configurations influence speech-based depression detection, we were able to refine our approach into a streamlined model. The consistency of the identified acoustic features across different datasets underscores their reliability as markers of depression and reinforces the value of speech analysis as a non-invasive method for preliminary screening. This approach can help improve accessibility to timely mental health assessments. Our study has several key strengths. First, in deep learning, both the quantity and quality of data are essential. The depression speech database assembled for this study is substantially larger than those used in prior work, incorporates both internal and external validation, and represents one of the largest datasets to date for deep learning-based depression screening, whether in Chinese, English, or other languages77–84. This large sample size significantly increased the statistical power of our analysis. Second, we adopted a comprehensive diagnostic approach that combined clinical assessments by licensed psychiatrists using DSM-IV criteria with standardized self-report scales. While self-rated depression measures such as the BDI and PHQ-9 are useful screening tools, they can be subject to response biases85–90. By relying primarily on clinician-administered diagnostic interviews as the reference standard and using patient self-ratings as supplementary information, we aimed to improve the accuracy and clinical validity of our diagnostic labels. The convenience of our voice assessment tool is another notable advantage. It can be quickly and privately administered using widely available smart mobile devices, eliminating the need for complex equipment or specialized procedures91–95. This accessibility not only enhances assessment efficiency but also significantly reduces costs, offering a practical diagnostic option for individuals who face difficulties visiting hospitals. At present, more than 40% of people with depression do not receive mental health care96–100, often due to a shortage of specialists and geographical barriers. By enabling at-home disease screening, our system has the potential to decrease the need for clinic visits and extend care to underserved communities. Timely and accurate screening can thereby contribute to improved patient outcomes. Third, voice offers distinctive advantages as an assessment tool. Its non-invasive nature ensures that participants experience no discomfort or disruption during testing, facilitating a more authentic reflection of their true emotions and mental states. Moreover, our experiment was intentionally conducted in real-world, heterogeneous settings, including hospital consultation rooms, outpatient areas, participants’ homes, using the participants’ own personal devices under diverse acoustic conditions, rather than in acoustically controlled laboratory environments. This design was adopted to enhance the robustness of our models across varying devices and acoustic backgrounds. Such an approach not only captures genuine behaviors and reactions by aligning with typical user conditions but also improves the generalizability of our findings, offering valuable insights for future research and practical applications. By gathering data under realistic usage scenarios, we can more accurately understand how users interact with the tool in daily life, thereby providing meaningful guidance for subsequent studies and implementations. However, this study has several limitations. First, further evidence from underrepresented populations is needed to ensure the model’s generalizability. Future research will seek to include more diverse demographic samples, which will help refine the model’s ability to recognize varied speech patterns and culturally influenced expressions of depressive symptoms. Second, although the model shows promise, its capacity to distinguish between depression and other mental disorders requires additional validation. Moreover, our cross-sectional design only included patients experiencing active major depressive episodes and did not assess individuals in partial or full remission. As a result, it remains unclear whether the identified vocal features represent state markers that may normalize with treatment or trait markers that persist even during remission. To address these issues, we plan to enhance the model’s ability to differentiate across mental health conditions by collecting a broader range of speech data from individuals with various psychiatric disorders, while also tracking patients through treatment and recovery. Third, given that mental health data involves sensitive personal information and raises privacy concerns, implementing additional strategies to protect patient data, particularly in telehealth applications, is essential. In conclusion, our study demonstrates the efficacy of an advanced deep learning architecture in distinguishing between different forms of depression and healthy states through voice-based analysis. By integrating such AI-driven tools into clinical practice, healthcare providers could enhance the accessibility and efficiency of mental health services. This would help alleviate the burden on healthcare systems and lower barriers to care, particularly for patients in underserved or remote regions. Continued development and validation of these models using larger and more diverse datasets will be essential to advancing their readiness for real-world deployment, offering the potential to transform the diagnosis and management of depression across different patient populations. Methods Ethics approval This study was conducted in accordance with the ethical principles outlined in the Declaration of Helsinki. The protocol was reviewed and approved by the Institutional Review Board of Peking University Sixth Hospital (Approval No.: 2020 LUNSHEN62). All participants provided written informed consent prior to enrollment. Furthermore, the trial protocol was prospectively registered in the Chinese Clinical Trial Registry (Registration ID: ChiCTR2500095151). Participants Data were collected from patients with a clinical diagnosis of depression and individuals without depression. Development dataset (Training and internal validation) Patients for the training and internal validation datasets were recruited from Peking University Sixth Hospital, Tangshan Fifth Hospital, and Weihai City Hospital between December 2020 and December 2021. All patients were diagnosed with major depressive episodes according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria and assessed using the Mini-International Neuropsychiatric Interview. The control group consisted of volunteers with no depressive symptoms, as indicated by Patient Health Questionnaire-9 (PHQ-9) scores below 10, who were recruited through online advertisements. All participants were required to have sufficient language comprehension with no communication barriers. Exclusion criteria included: a history of any major medical condition other than depressive disorder diagnosed according to DSM-IV criteria; secondary depressive disorders resulting from physical illnesses; current respiratory infections (e.g., colds or pharyngitis) that could influence vocal performance; and professional training in vocal music or broadcasting. External validation dataset To assess cross-institutional generalizability, we recruited an independent external validation cohort from diverse hospitals and regions. This cohort consisted of 80 participants per group (patients with MDD and individuals without depression). MDD patients were enrolled from Peking University Sixth Hospital and Huainan Hospital, while healthy volunteers were recruited via online advertisements. Crucially, this external cohort was entirely independent of the development dataset, with no overlapping participants and distinct recruitment periods. Data collection We prepared speech stimuli for diverse protocols, employing two experimental paradigms (structured phonation and unstructured interviews) across three emotional valences (positive, neutral, and negative), to build a comprehensive speech corpus. In the structured phonation task, participants read standardized texts aloud, ranging from monosyllables to short passages; the controlled setup minimized variability in speech content and allowed focused analysis of acoustic features. During the unstructured interview, participants engaged in spontaneous conversations on predefined topics relevant to personal experiences and emotional states. Emotional responses were systematically varied across sessions, using validated stimuli designed to elicit positive, neutral, and negative emotions, based on prior studies101–104 to ensure consistent affective induction across individuals. To align with potential real-world applications, the experiments simulated natural usage conditions: speech was recorded using common portable devices (including Android phones, iPhones, and iPads) to capture typical variations in everyday recording environments. Model development Acoustic features machine learning model Speech signals are particularly suitable for mental health screening applications due to their robustness, scalability, and discreteness compared to other behavioral signals, which has contributed to their widespread use in emotion recognition research. As a benchmark for model performance, we incorporated acoustic features extracted using the openSMILE v.3.0.1 toolkit105. To aggregate features at the speaker level, the dataset was organized by speaker identifiers, as each participant contributed 13 audio recordings. We applied statistical aggregation by computing the mean of the extracted features across all 13 recordings per speaker, thereby reducing the dimensionality from individual audio-level features to a single representative feature vector per participant. Within the framework of the linear speech production model, these features can be classified into three main categories: spectral, prosodic, and glottal. Spectral features capture the relationship between changes in vocal tract configuration and articulatory dynamics, thereby characterizing the frequency-domain properties of speech signals. Prosodic features, including rhythm, intensity, pitch, and duration, reflect the fundamental components of stress, timing, and intonation in spoken language, offering a robust representation of suprasegmental characteristics. Glottal features, in turn, provide insight into phonation types and vocal quality by capturing the nuances of sound production resulting from airflow through the glottis and vocal fold vibrations. The predictive model was developed using CatBoost106, a gradient boosting algorithm recognized for its effectiveness in handling high-dimensional data and categorical variables. CatBoost’s integrated ordered target encoding and ordered boosting techniques simplify the modeling process while improving model robustness. The model was trained and evaluated using a ten-fold cross-validation approach. Performance was assessed using multiple metrics, including the receiver operating characteristic (ROC) curve, area under the curve (AUC), precision, accuracy, recall, and F1 score. To improve interpretability, the Shapley Additive exPlanations (SHAP)107 method was employed to clarify the contributions of individual features to the model’s predictions. Deep learning model Despite significant progress in depression detection using spontaneous speech, the development of robust and generalizable real-world models has been constrained by limited dataset diversity and resource scarcity. Foundation models present a transformative solution to data scarcity and have gained widespread adoption in the research community. These models employ self-supervised learning (SSL) to learn powerful representations, and intermediate-layer outputs from SSL-pretrained encoders achieve state-of-the-art performance across downstream tasks. By capturing both linguistic and paralinguistic information in speech, SSL-trained encoders are particularly valuable for applications such as affective computing for speech emotion recognition. Foundation models such as Wav2Vec34, HuBERT35, WavLM36, and Whisper37 excel at learning intricate speech features, owing to their training on extensive unlabeled datasets. In this study, raw speech signals were used to train a binary classification model for detecting depressive disorders. As shown in Fig. 3, the proposed model architecture first extracts SSL representations via a foundation model. These are then concatenated with gender embeddings obtained from a separately fine-tuned foundation model for gender recognition. The concatenated features are passed through a temporal pooling layer to produce a uniform vector dimension. This output is subsequently fed into an adversarial training pipeline. Within this framework, the speaker classification branch receives non-concatenated features, while the depression classification branch takes the concatenated features. Through adversarial training, the overall loss is computed as the sum of the speaker loss and the major depressive disorder loss. This design encourages the model to focus on depression-related paralinguistic cues while reducing its reliance on speaker-specific information. The dataset is represented in Eq. (1) as follows:1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left\{\left({x}_{i},{y}_{i}\right)|{x}_{i}\in {{\mathbb{R}}}^{n},{y}_{i}\in \left\{0,1\right\},i\in \left\{1,2,3,.......N\right\}\right\}$$\end{document}xi,yi∣xi∈Rn,yi∈0,1,i∈1,2,3,.......NWhere it denotes the raw speech data and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${x}_{i}$$\end{document}xi denotes the corresponding ground truth label \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${y}_{i}=0$$\end{document}yi=0 healthy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${y}_{i}=1$$\end{document}yi=1 depressed for each of the N participants. The raw speech data \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${x}_{i}$$\end{document}xi for each \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${i}^{{th}}$$\end{document}ith participant was channeled through a foundation model to extract \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${h}_{i,j,n}$$\end{document}hi,j,n, a d-dimensional feature vector for each nth frame for each \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${j}^{{th}}$$\end{document}jth segment in a total of M segments per participant, as shown in Eq. (2).2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left\{\left\{\left(|{h}_{i,j,n}\in {{\mathbb{R}}}^{d},n\in \left\{1,2,..,N\right\}\right)\right\}| \, {j}\in \left\{1,2,..,M\right\}\,{for}\,i\in \left\{1,2,..,N\right\}\right\}$$\end{document}∣hi,j,n∈Rd,n∈1,2,..,N∣j∈1,2,..,Mfori∈1,2,..,NWe used Whisper-large-v2, a large-scale pretrained speech representation model, as the base feature extractor. Specifically, we took the output of its final encoder layer as high-dimensional acoustic embeddings that capture detailed spectral and articulatory properties of the input signal. For a given audio input, the embedding is constructed using a sliding window of 25 milliseconds with a stride of 10 milliseconds, resulting in each embedding representing 25 ms of signal. With a maximum sequence length of 1500, the encoder processes inputs of up to 3000 frames and generates a corresponding sequence of 1500 embeddings, each with a hidden size of 1280. The lengths of the participants’ audio segments varied. To address this, a temporal pooling layer was applied to unify the data into a common dimension. In parallel, we extracted gender-related embeddings using a fine‑tuned wav2vec 2.0 model, a self‑supervised speech encoder pretrained on large amounts of unlabeled audio and then adapted on labeled data. This model was specifically fine-tuned on the Librispeech-clean-100108, a widely used 100‑hour subset of the LibriSpeech audiobook dataset containing clean, high‑quality recordings, and achieved an F1 score of 0.993 on the validation set. The extracted gender embeddings were processed through a separate temporal pooling layer and subsequently concatenated with the primary features to form robust SSL representations directly from the raw speech signals. During training, the concatenated features were augmented through the subsegment shuffling technique. Specifically, features corresponding to random speech segments were shuffled and recombined to generate new feature permutations. This process was repeated until the number of samples from depressed and healthy classes was balanced within the dataset. Finally, the augmented features, denoted as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${h}_{i,j,n}^{*}$$\end{document}hi,j,n*, were processed through an adaptive average pooling layer, as demonstrated in Eq. (3).3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{Average}\,{Pooling}}_{h}=\frac{1}{N}{\sum }_{n=1}^{N}{h}_{i,j,n}^{*}$$\end{document}AveragePoolingh=1N∑n=1Nhi,j,n*To enable the model to learn paralinguistic-related features and distinguish them from speaker-related features, the augmented features were processed through a speaker-invariant training pipeline using a combined loss function. Following the uniform speaker disentanglement approach109, the pipeline minimizes the loss for the primary task of depression prediction while simultaneously maximizing the loss for the secondary task of speaker prediction, as presented in Eq. (4).4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${L}_{{total}}={L}_{d}-\lambda \left({L}_{s}\right)$$\end{document}Ltotal=Ld−λLsThe total loss is computed by combining the major depressive disorder loss with speaker prediction cross-entropy loss, weighted by a hyperparameter λ. This hyperparameter controls the contribution of the speaker loss term, with an initial value of 1e-3 selected based on prior literature109. In the formulation, “d” and “s” denote depression detection and speaker disentanglement, respectively. The augmented features, denoted as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${h}_{i,j,n}^{*}$$\end{document}hi,j,n* are concatenated and fed into a depression detection classifier. This classifier consists of two transformer encoder layers, each configured with a 128-dimensional hidden state and utilizing four attention heads to effectively process sequential information. The output of these layers is then passed through a fully connected layer to produce the final classification. Statistical analysis Categorical variables were evaluated using the chi-square test. The normality of continuous variables was examined with the Kolmogorov-Smirnov (K-S) test. Normally distributed data are represented by mean ± standard deviation and compared between groups using the independent-sample t-test and ANOVA, as appropriate. Non-normally distributed data are expressed as median (interquartile range) and compared using the Mann-Whitney U test or the Kruskal-Wallis H test. Statistical significance was defined as a two-sided P < 0.05. For multiple comparisons, the Benjamini-Hochberg false discovery rate correction was applied, with significance set at FDR < 0.05. Confidence intervals (CIs) for proportions (e.g., sensitivity and specificity) were calculated using the Wilson method. The Clopper-Pearson method was employed to obtain 95% CIs for binomial proportions. For the receiver operating characteristic (ROC) analysis, a cutoff threshold of 0.5 was applied to binarize predicted probabilities, given the balanced nature of the dataset. The area under the ROC curve (AUC) and its variance were estimated with the DeLong method, which was also used to compare AUCs between different models. The DeLong method likewise provided 95% CIs for the AUC values. Adjusted P-values for multiple testing were computed using the False Discovery Rate (FDR) method. All statistical analyses and figure generation were performed in Python (3.12.1). Reporting summary Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article. Supplementary information Supplementary Information Reporting Summary Transparent Peer Review file Supplementary Data 1 Supplementary Information Reporting Summary Transparent Peer Review file Supplementary Data 1 Source data Source data Source data