Auxiliary identification of depression patients using interpretable machine learning models based on heart rate variability: a retrospective study Authors: Min Yang, Huiqin Zhang, Minglan Yu, Yunxuan Xu, Bo Xiang, Xiaopeng Yao Published: 2024 ABSTRACT ObjectiveDepression has emerged as a global public health concern with high incidence and disability rates, which are timely imperative to identify and intervene in clinical practice. The objective of this study was to explore the association between heart rate variability (HRV) and depression, with the aim of establishing and validating machine learning models for the auxiliary diagnosis of depression.MethodsThe data of 465 outpatients from the Affiliated Hospital of Southwest Medical University were selected for the study. The study population was then randomly divided into training and test sets in a 7:3 ratio. Logistic regression (LR), support vector machine (SVM), random forest (RF) and eXtreme gradient boosting (XGBoost) algorithm models were used to construct risk prediction models in the training set, and the model performance was verified in the test set. The four models were evaluated by the area under the receiver operating characteristic curve (ROC), calibration curve and the decision curve analysis (DCA). Furthermore, we employed the SHapley Additive exPlanations (SHAP) method to illustrate the effects of the features attributed to the model.ResultsThere were 237 people in the depressed group and 228 in the non-depressed group. In the training set (n = 325) and test set (n = 140), the area under of the curve(AUC) values of the XGBoost model are 0.92 [95% confidence interval (CI) 0.888,0.95] and 0.82 (95% CI 0.754,0.892)] respectively, which are higher than the other three models. The XGBoost model has excellent predictive efficacy and clinical utility. The SHAP method was ranked according to the importance of the degree of influence on the model, with age, heart rate, Standard deviation of the NN intervals (SDNN), two nonlinear parameters of HRV and sex considered to be the top 6 predictors.ConclusionWe provided a feasibility study of HRV as a potential biomarker for depression. The proposed model based on HRV provides clinicians with a quantitative auxiliary diagnostic tool, which is assist to improving the accuracy and efficiency of depression diagnosis, and can also be utilized for the monitoring and prevention of depression.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12888-024-06384-w. Introduction Depression is a common mental disorder [1], which is mainly manifested as the patient’s continuous sadness or inability to experience happiness, accompanied by daily living and social functioning defects [2]. According to the 2019 Global Burden of Diseases (GBD) Study, depression ranks among the top 10 global disability-adjusted life years (DALYs) [3]. For nearly three decades, depression has been the leading cause of non-fatal health loss [4–6]. Depression affects individuals of all age groups from adolescence to old age [7, 8], and the incidence of depression is approximately twice as common in female as in male [9, 10]. Different from some diseases with gold diagnostic standards [11–13] depression is usually diagnosed through clinical interviews [8, 14, 15], and whose clinical manifestations are diverse [16, 17]. Currently, the traditional diagnosis of depression usually adopts clinical symptoms and depression scale for qualitative diagnosis, and cannot be quantitatively evaluated, which may lead to missed diagnosis or misdiagnosis. In recent years, with the rapid development of artificial intelligence technology, researchers have constantly explored quantitative diagnosis models for depression based on markers such as magnetic resonance imaging (MRI), electroencephalography (EEG), and serotonin levels [18–20]. These models can avoid the subjectivity of traditional diagnostic methods to some extent. However, these machine learning models based on imaging and biological indicators still have problems such as high cost, invasiveness and complexity, so it is urgent to find a biomarker with real-time monitoring, non-invasiveness and low cost. The heart rate variability (HRV) is caused by the dynamic interaction between sympathetic and parasympathetic nervous system activity [21–23]. HRV can be obtained from the R-R interval minimal variability of electrocardiograph (ECG) signals or pulse signals, which are quantitative predictor to evaluate the regulation ability of the autonomic nervous system (ANS) [24, 25]. Currently, several researchers had used HRV for the auxiliary diagnosis of certain mental diseases, cardiovascular diseases, endocrine and other related diseases [26–29]. Some epidemiological and clinical studies indicate that individuals with depression might be in a constant state of stress, which could lead to imbalance of the autonomic nervous system. Specifically, the parasympathetic activity might be weakened, while the sympathetic activity might be enhanced. This imbalance will affect HRV and reduce it [30, 31]. Although previous studies have compared some HRV indicators between depressed people and healthy people [32–34], few studies have comprehensively compared the differences in the time domain, frequency domain and nonlinear parameters of HRV between depressed people and healthy people [35, 36], and established a depression diagnosis model based on heart rate variability [37, 38]. In this study, we aimed to investigate the correlation between HRV and depression, determine the predictive factors of the depression model, and explore the feasibility of HRV as a potential biomarker for depression. At the same time, we established clinical prediction modesl based on HRV to assist clinicians in more accurately identifying depression patients, thereby enabling timely optimization of diagnostic and treatment strategies. Methods Study design This study strictly followed the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement [39], details are shown in Supplemental Table 1. The workflow of this study is illustrated in Fig. 1, which includes three main parts. Firstly, the participants were divided into two groups, depressed and non-depressed, based on specific inclusion and exclusion criteria. Subsequently, we selected clinical characteristic variables and HRV characteristic variables from case reports. We developed prediction models using four machine learning algorithms in the training set, and then evaluated their performance in the test set. Finally, we used the SHAP method to explain the best prediction model. Fig. 1The workflow for data processing and construction of four predictive models The workflow for data processing and construction of four predictive models Data source and study population This retrospective study was approved by the Institutional Review Board of the Southwestern Medical University Hospital (No. KY2020221), and the requirement for written informed consent was waived. This study protocol complies with the principles of the Declaration of Helsinki. The initial data for this study came from Department of Psychiatry, the Affiliated Hospital of Southwest Medical University. We collected outpatient who underwent depression examination between November 2021 and March 2022. The inclusion criteria were as follows: (i) participants were 14 years of age and older; (ii) participants were required to complete both the Depression Assessment Scale and HRV tests. The exclusion criteria were as follows: (i) patients who did not follow the requirements of the HRV test; (ii) patients with irregular heart rhythm, arterial fibrillation or atrioventricular conduction block; (iii) pregnant women. A total of 621 outpatients who underwent depression examination were enrolled in the study. We excluded 156 patients, such as lack of complete HRV reports (n = 142), suffering from cardiovascular diseases(n = 11), and pregnant women(n = 3). Finally, a total of 465 patients were enrolled in the study from November 2021 to March 2022. All patients were randomly divided into a training set (n = 325) and a test set (n = 140), with a ratio of approximately 7:3. A flowchart of the patients’ collection is shown in Fig. 2. Fig. 2The flowchart of patient cohort selection The flowchart of patient cohort selection Determination of depression and HRV measurement In this study, the Hamilton Depression (HAMD) Scale integrated clinical symptoms were used to diagnose depression [40], which provided clinicians with diagnostic decisions. There were a total of 24 items in the HAMD depression scale, which mainly included 8 parts: anxiety/somatization, weight, cognitive disorders, day-night changes, block, sleep disorders, feelings of hopelessness, and systemic symptoms [41]. Most projects are rated on a 5-point scale (0 to 4), while a few projects are rated on a 3-point scale (0 to 2) [42]. This scale has high reliability and validity, cronbach’s alpha is 0.88 [43], and has been widely used in clinical practice [44–46]. The total crude score of HAMD Scale H < 8 is no depression, 8–20 is mild depression, 21–34 is moderate depression, and > 35 is severe depression.In this study, the diagnosis of depression patients required scale scores above moderate depression level and clinical evaluation. The HRV was obtained from VitalScan, a professional commercial device (https://www.creditpharma.com/, Sichuan Credit Pharmaceutical CO.LTD.), in a quiet environment. During the whole HRV measurement process, patients tried to avoid speaking, smoking, taking caffeine, body movement or disturbance, and remain quiet. A short plethysmography (PPG1) was placed on the subject’s left index finger (nail facing up), and the patient’s information was collected on the operating software, and then the HRV was measured for 5 min. After the measurement was completed, the HRV report of all patients was printed and saved. Variable selection A total of 17 characteristic variables were selected from the medical case reports of outpatients with depression, respectively, including 2 demographic characteristic variables (sex, age), 2 clinical characteristic variables (body mass index (BMI), heart rate) and 13 HRV characteristic variables, the details are shown in Table 1. The HRV characteristic variables include time domain variables, frequency domain variables, and nonlinear variables. We use elastic network method for variables selection. Elastic net regression is a combination of ridge regression and Lasso (Least Absolute Shrinkage and Selection Operator) regression machine learning algorithm with Regression advantage, which achieves variables selection and model stability by introducing two regularization parameters, is suitable for processing features with multicollinearity [47]. Ultimately, a total of 6 characteristic variables were used to develop prediction models (Fig. 3). Fig. 3The flowchart of patient cohort selection. (A)The elastic network coefficient profiles of the variables. Different color lines represent the coefficients corresponding to each variable; (B) Six variables features selected by elastic network The flowchart of patient cohort selection. (A)The elastic network coefficient profiles of the variables. Different color lines represent the coefficients corresponding to each variable; (B) Six variables features selected by elastic network Statistical analysis All statistical analysis and calculations were performed using R (v. 3.6.1, R Foundation for Statistical Computing), Python (version 3.11.7) and SPSS 26.0. Continuous variables are expressed as medians and interquartile ranges, and categorical variables are expressed as frequency (percentage). Continuous variables between groups were compared using Mann-Whitney U test, and categorical variables between groups were compared using Chi-square test. All P values were two-tailed, and P < 0.05 was considered statistically significant. Model establishment and validation Four machine learning algorithms, including logistic regression (LR), support vector machine (SVM), random forest (RF) and extreme gradient boosting (XGBoost),were respectively used to develop depression prediction models in the training set. These four models are all supervised machine learning algorithms. LR makes predictions by learning the relationship between data features and target variables, and the output result can be expressed as probability [48]. SVM maps vectors into a higher-dimensional space and is able to handle the interaction of nonlinear features [49]. RF combines different decision trees, and the algorithm is generally stable [50]. The XGBoost model uses the gradient enhancement framework, which is also an integrated method based on decision trees, and has the characteristics of high efficiency and flexibility [51]. These four supervised learning methods are widely used in the diagnosis and prediction of classification problems. To reduce the bias in the prediction outputs and the possibility of overfitting due to insufficient sample size, a combination method of bootstrap validation and 10-fold cross-validation is used to determine the best parameters by using a grid search [52, 53]. Within each fold of the 10-fold cross-validation, we employed resampling techniques on the original training set to generate bootstrap training sets. Subsequently, we trained the machine learning model on these bootstrap sets and evaluated its predictive performance on a separate test set. This resampling and model validation process was meticulously repeated 1000 times to ensure thorough validation of the model. The prediction performance of these model was evaluated by calculating the area of receiver operating characteristics (ROC) curve, calibration curve, sensitivity (SEN), specificity (SPE), positive predictive value (PPV), negative predictive value (NPV), F1 score respectively and calibration curve. The Decision Curve Analysis (DCA) of all models was performed to quantify the net benefit of patients under different threshold probabilities in the cohort to assess the clinical value of the prediction models in our study. Model interpretability with SHapley Additive exPlanation (SHAP) The interpretability of prediction models is crucial in clinical practice. We used the SHAP method to accurately calculate the contribution and influence of each characteristic variable to the final prediction, which visually explained the results of the best prediction model to assist clinical decision-making [54, 55]. The SHAP transforms the characteristic variables space into the target variables space, where the SHAP value demonstrated the degree to which each predictor contributes positively or negatively to the target variables. Furthermore, we analyzed the effects of individual characteristics on the model output using SHAP dependency plots. Results Baseline characteristics A total of 465 outpatients were included in this study, with 237 in the depression group and 228 in the non-depression group. The specific baseline characteristics of all patients are shown in Table 1. The median age was 20 (17,36) years, with 60.6% females and 39.4% males. The median heart rate in the depressed group was 85, higher than 76.09 in the non-depressed group. There were significant differences in sex, age, heart rate and HRV characteristics ( SDNN, SDNN5,pNN50, rmsSD, SDSD, SD1, SD2, VLF, LF, HF and TP ) between the two groups ( P<0.05). Table 1Characteristics between depression and non-depression groupsVariablesTotal (n = 465)Non-depressed (n = 228)Depression (n = 237)Statistic P Age, M (Q₁, Q₃)20.00 (17.00, 36.00)20.00 (18.00, 46.25)18.00 (16.00, 33.00)Z=-4.86 < 0.001 Sex (N, %)χ2 = 19.526 < 0.001 Male183(39.4)113(24.3)70(15.1)Female282(60.6)115(24.7)165(35.9)BMI, M (Q₁, Q₃)20.80 (18.70, 23.40)21.05 (19.00, 24.05)20.80 (18.50, 23.00)Z=-1.520.127Heart rate, M (Q₁, Q₃)80.73 (71.85, 90.70)76.09 (69.35, 85.02)85.00 (76.77, 93.37)Z=-6.59 < 0.001 Time Domain SDNN, M (Q₁, Q₃)43.38 (30.65, 56.77)48.80 (35.31, 58.84)38.12 (25.60, 50.94)Z=-5.50 < 0.001 SDNN5, M (Q₁, Q₃)35.42 (23.84, 44.69)38.56 (29.02, 47.38)29.97 (20.58, 41.34)Z=-5.50 < 0.001 Pnn50, M (Q₁, Q₃)5.53 (0.94, 19.61)8.76 (2.40, 22.02)3.74 (0.42, 15.93)Z=-4.14 < 0.001 Rmssd, M (Q₁, Q₃)28.18 (18.08, 40.11)30.81 (21.82, 41.24)25.25 (16.27, 37.61)Z=-3.74 < 0.001 SDSD, M (Q₁, Q₃)28.14 (18.05, 40.00)30.81 (21.82, 41.24)25.09 (16.25, 37.48)Z=-3.80 < 0.001 Nonlinearity SD1, M (Q₁, Q₃)19.93 (12.78, 28.36)21.79 (15.43, 29.16)17.86 (11.50, 26.59)Z=-3.74 < 0.001 SD2, M (Q₁, Q₃)57.74 (40.55, 73.02)63.93 (47.14, 77.33)50.53 (33.97, 66.28)Z=-5.60 < 0.001 SD1/SD2, M (Q₁, Q₃)0.35 (0.28, 0.44)0.35 (0.27, 0.42)0.36 (0.28, 0.45)Z=-1.280.202 Frequency Domain VLF, M (Q₁, Q₃)611.21 (295.49, 1120.78)763.11 (363.02, 1305.06)469.42 (240.08, 863.80)Z=-4.87 < 0.001 LF, M (Q₁, Q₃)358.56 (132.99, 761.70)474.07 (156.57, 860.63)272.85 (116.31, 579.02)Z=-3.67 < 0.001 HF, M (Q₁, Q₃)216.56 (85.69, 511.42)246.06 (102.15, 561.87)203.68 (63.78, 411.36)Z=-2.49 0.013 TP, M (Q₁, Q₃)1778.04 (842.05, 2994.88)2257.11 (1061.62, 3322.61)1396.91 (609.18, 2525.24)Z=-4.79 < 0.001 LF/HF, M (Q₁, Q₃)1.57 (0.86, 3.06)1.59 (0.91, 3.06)1.57 (0.75, 3.06)Z=-0.400.690Z: Mann-Whitney test, χ2: Chi-square test, M: Median, Q₁: 1st Quartile, Q₃: 3st Quartile. BMI: body mass index, SDNN: Standard deviation of the NN intervals, SDNN5:Mean of the standard deviation of the 5-min NN, pNN50:Percentage of successive NN intervals that differ by more than 50 ms, rmsSD: Root mean square of successive differences, SDSD: Standard deviation between adjacent NN intervals, SD1: Standard deviation of point plot to the transverse diameter, SD2: Standard deviation of the distances of the points from the major axis, VLF: Very low frequency spectrum, LF: Low-frequency power, HF: High frequency spectrum, TP: Total power Table 1Characteristics between depression and non-depression groupsVariablesTotal (n = 465)Non-depressed (n = 228)Depression (n = 237)Statistic P Age, M (Q₁, Q₃)20.00 (17.00, 36.00)20.00 (18.00, 46.25)18.00 (16.00, 33.00)Z=-4.86 < 0.001 Sex (N, %)χ2 = 19.526 < 0.001 Male183(39.4)113(24.3)70(15.1)Female282(60.6)115(24.7)165(35.9)BMI, M (Q₁, Q₃)20.80 (18.70, 23.40)21.05 (19.00, 24.05)20.80 (18.50, 23.00)Z=-1.520.127Heart rate, M (Q₁, Q₃)80.73 (71.85, 90.70)76.09 (69.35, 85.02)85.00 (76.77, 93.37)Z=-6.59 < 0.001 Time Domain SDNN, M (Q₁, Q₃)43.38 (30.65, 56.77)48.80 (35.31, 58.84)38.12 (25.60, 50.94)Z=-5.50 < 0.001 SDNN5, M (Q₁, Q₃)35.42 (23.84, 44.69)38.56 (29.02, 47.38)29.97 (20.58, 41.34)Z=-5.50 < 0.001 Pnn50, M (Q₁, Q₃)5.53 (0.94, 19.61)8.76 (2.40, 22.02)3.74 (0.42, 15.93)Z=-4.14 < 0.001 Rmssd, M (Q₁, Q₃)28.18 (18.08, 40.11)30.81 (21.82, 41.24)25.25 (16.27, 37.61)Z=-3.74 < 0.001 SDSD, M (Q₁, Q₃)28.14 (18.05, 40.00)30.81 (21.82, 41.24)25.09 (16.25, 37.48)Z=-3.80 < 0.001 Nonlinearity SD1, M (Q₁, Q₃)19.93 (12.78, 28.36)21.79 (15.43, 29.16)17.86 (11.50, 26.59)Z=-3.74 < 0.001 SD2, M (Q₁, Q₃)57.74 (40.55, 73.02)63.93 (47.14, 77.33)50.53 (33.97, 66.28)Z=-5.60 < 0.001 SD1/SD2, M (Q₁, Q₃)0.35 (0.28, 0.44)0.35 (0.27, 0.42)0.36 (0.28, 0.45)Z=-1.280.202 Frequency Domain VLF, M (Q₁, Q₃)611.21 (295.49, 1120.78)763.11 (363.02, 1305.06)469.42 (240.08, 863.80)Z=-4.87 < 0.001 LF, M (Q₁, Q₃)358.56 (132.99, 761.70)474.07 (156.57, 860.63)272.85 (116.31, 579.02)Z=-3.67 < 0.001 HF, M (Q₁, Q₃)216.56 (85.69, 511.42)246.06 (102.15, 561.87)203.68 (63.78, 411.36)Z=-2.49 0.013 TP, M (Q₁, Q₃)1778.04 (842.05, 2994.88)2257.11 (1061.62, 3322.61)1396.91 (609.18, 2525.24)Z=-4.79 < 0.001 LF/HF, M (Q₁, Q₃)1.57 (0.86, 3.06)1.59 (0.91, 3.06)1.57 (0.75, 3.06)Z=-0.400.690Z: Mann-Whitney test, χ2: Chi-square test, M: Median, Q₁: 1st Quartile, Q₃: 3st Quartile. BMI: body mass index, SDNN: Standard deviation of the NN intervals, SDNN5:Mean of the standard deviation of the 5-min NN, pNN50:Percentage of successive NN intervals that differ by more than 50 ms, rmsSD: Root mean square of successive differences, SDSD: Standard deviation between adjacent NN intervals, SD1: Standard deviation of point plot to the transverse diameter, SD2: Standard deviation of the distances of the points from the major axis, VLF: Very low frequency spectrum, LF: Low-frequency power, HF: High frequency spectrum, TP: Total power Characteristics between depression and non-depression groups Z: Mann-Whitney test, χ2: Chi-square test, M: Median, Q₁: 1st Quartile, Q₃: 3st Quartile. BMI: body mass index, SDNN: Standard deviation of the NN intervals, SDNN5:Mean of the standard deviation of the 5-min NN, pNN50:Percentage of successive NN intervals that differ by more than 50 ms, rmsSD: Root mean square of successive differences, SDSD: Standard deviation between adjacent NN intervals, SD1: Standard deviation of point plot to the transverse diameter, SD2: Standard deviation of the distances of the points from the major axis, VLF: Very low frequency spectrum, LF: Low-frequency power, HF: High frequency spectrum, TP: Total power In addition, visual statistical analysis was employed on the difference of HRV parameters between the depression group and the non-depression group. The time domain parameters (SDNN, SDNN5, pNN50, rmsSD, SDSD), frequency domain parameters (VLF, LF) and nonlinear parameters (SD1, SD2) of the depression group were lower than those of the non-depression group, as shown in Fig. 4. Fig. 4Visualization of differences in HRV parameters between depressed and non-depressed groups. (A)Time domain parameters of HRV between the two groups. (B) Frequency domain parameters of HRV between the two groups. (C)Nonlinear parameters of HRV between the two groups Visualization of differences in HRV parameters between depressed and non-depressed groups. (A)Time domain parameters of HRV between the two groups. (B) Frequency domain parameters of HRV between the two groups. (C)Nonlinear parameters of HRV between the two groups Development and validation of prediction models We developed four machine learning models (LR, RF, SVM and XGBoost) for depression prediction in the training set. The area under the curve (AUC) values of these models were all greater than 0.7 (Fig. 5). Among the four models, XGBoost and RF exhibited the higher AUC (XGBoost:0.9191,95% Confidence Interval (CI): 0.888,0.95; RF:0.9078, 95% CI: 0.875,0.941). The LR model and SVM have lower AUC of 0.754[0.691,0.817] and 0.7599[0.698,0.822] respectively. Fig. 5ROC curves for the machine-learning models to predict depression. (A) ROC curve of the LR model. (B) ROC curve of the SVM model. (C) ROC curve of the RF model. (D) ROC curve of the XGBoost model ROC curves for the machine-learning models to predict depression. (A) ROC curve of the LR model. (B) ROC curve of the SVM model. (C) ROC curve of the RF model. (D) ROC curve of the XGBoost model The results of SPE, SEN, PPV, NPV and F1 score were used to further compare the performance of all models (Table 2). The SPE, SEN, NPV, PPV, F1 score and differentiation of the XGBoost model were better than the other models. The DeLong test (Supplemental Table 2) showed that the performance of XGBoost model was better than the other models (p < 0.05). Therefore, the XGBoost model has the best prediction performance, and we will further explain the XGBoost model in the future later. Table 2Comprehensive performance evaluation indexes of different prediction modelsModelAUC(95% CI)SPE(95% CI)SEN(95% CI)PPV(95% CI)NPV(95% CI)F1 scoreLRTraining set0.754[0.691,0.817]0.6786[0.593,0.764]0.713[0.63,0.796]0.6949[0.611,0.779]0.6972[0.613,0.781]0.7039Test set0.7079[0.606,0.81]0.6765[0.568,0.785]0.68[0.551,0.809]0.6615[0.552,0.771]0.6133[0.501,0.726]0.6476SVMTraining set0.7599[0.698,0.822]0.7[0.63,0.77]0.7739[0.697,0.85]0.6981[0.628,0.768]0.7143[0.632,0.797]0.7092Test set0.7271[0.627,0.827]0.7353[0.633,0.837]0.74[0.618,0.862]0.6949[0.589,0.801]0.6579[0.526,0.789]0.6727RFTraining set0.9078[0.875,0.941]0.7625[0.698,0.827]0.8606[0.808,0.913]0.7889[0.727,0.851]0.8414[0.786,0.897]0.8232Test set0.8096[0.738,0.881]0.7794[0.684,0.875]0.7639[0.666,0.862]0.7857[0.691,0.88]0.7571[0.658,0.856]0.7746XGboostTraining set0.9191[0.888,0.95]0.8[0.739,0.8619]0.8727[0.822,0.924]0.8182[0.759,0.877]0.8591[0.806,0.912] 0.8446 Test set0.8233[0.754,0.892]0.7794[0.684,0.875]0.7917[0.698,0.885]0.7917[0.698,0.885]0.7794[0.684,0.875] 0.7917 CI: Confidence Interval, AUC: Area under of the curve, SPE: Specificity, SEN: Sensitivity, PPV: Positive predictive value, NPV: Negative predictive value, F1 score: balanced, LR: Logistic regression, SVM: support vector machine, RF: Random Forest, XGBoost: Extreme gradient boosting Table 2Comprehensive performance evaluation indexes of different prediction modelsModelAUC(95% CI)SPE(95% CI)SEN(95% CI)PPV(95% CI)NPV(95% CI)F1 scoreLRTraining set0.754[0.691,0.817]0.6786[0.593,0.764]0.713[0.63,0.796]0.6949[0.611,0.779]0.6972[0.613,0.781]0.7039Test set0.7079[0.606,0.81]0.6765[0.568,0.785]0.68[0.551,0.809]0.6615[0.552,0.771]0.6133[0.501,0.726]0.6476SVMTraining set0.7599[0.698,0.822]0.7[0.63,0.77]0.7739[0.697,0.85]0.6981[0.628,0.768]0.7143[0.632,0.797]0.7092Test set0.7271[0.627,0.827]0.7353[0.633,0.837]0.74[0.618,0.862]0.6949[0.589,0.801]0.6579[0.526,0.789]0.6727RFTraining set0.9078[0.875,0.941]0.7625[0.698,0.827]0.8606[0.808,0.913]0.7889[0.727,0.851]0.8414[0.786,0.897]0.8232Test set0.8096[0.738,0.881]0.7794[0.684,0.875]0.7639[0.666,0.862]0.7857[0.691,0.88]0.7571[0.658,0.856]0.7746XGboostTraining set0.9191[0.888,0.95]0.8[0.739,0.8619]0.8727[0.822,0.924]0.8182[0.759,0.877]0.8591[0.806,0.912] 0.8446 Test set0.8233[0.754,0.892]0.7794[0.684,0.875]0.7917[0.698,0.885]0.7917[0.698,0.885]0.7794[0.684,0.875] 0.7917 CI: Confidence Interval, AUC: Area under of the curve, SPE: Specificity, SEN: Sensitivity, PPV: Positive predictive value, NPV: Negative predictive value, F1 score: balanced, LR: Logistic regression, SVM: support vector machine, RF: Random Forest, XGBoost: Extreme gradient boosting Comprehensive performance evaluation indexes of different prediction models CI: Confidence Interval, AUC: Area under of the curve, SPE: Specificity, SEN: Sensitivity, PPV: Positive predictive value, NPV: Negative predictive value, F1 score: balanced, LR: Logistic regression, SVM: support vector machine, RF: Random Forest, XGBoost: Extreme gradient boosting Calibration plots were used to evaluate the calibration level of the prediction model (Fig. 6(A)-(D)). The Bootstrap method was used to repeatedly sample 1000 times and draw calibration curves for internal validation of the prediction model. The calibration curves of the four prediction models were close to ideal curves, indicating that the prediction model had a good fitting effect. The DCA was used to evaluate the net benefit of the four prediction models for clinical decision-making at different diagnostic thresholds. We plotted the decision curves of the four models to compare their clinical net benefit in the test set, respectively. In Fig. 6(E)-(F), the horizontal axis represents the threshold for the probability of developing depression, ranging from 0 to 1, and the vertical axis represents the net benefit rate of patients after intervention. The net benefit of XGBoost was significantly larger than that of the other models in all three groups. Fig. 6The calibration curves and DCA curves for the four models. (A) Calibration curve of LR model. (B) Calibration curve of SVM model. (C) Calibration curve of RF model. (D) Calibration curve of XGBoost model. (E) The DCA curves of four models in the training set. (F) The DCA curves of four models in the testing set. The calibration curves and DCA curves for the four models. (A) Calibration curve of LR model. (B) Calibration curve of SVM model. (C) Calibration curve of RF model. (D) Calibration curve of XGBoost model. (E) The DCA curves of four models in the training set. (F) The DCA curves of four models in the testing set. Explanation of the XGBoost model using the SHAP method Since the predictions of the XGBoost model were generally better than the other three machine learning models, we chose this model to analyze the interpretability of clinical diagnosis. While explanatory methods can reveal the decision-making process of black box models, caution is needed when incorporating population demographic variables such as age and sex, as the reliability of these variables when evaluated using SHAP analysis techniques may not be comparable to traditional statistical analysis. Consequently, we conducted comprehensive univariate and multivariate analyses specifically for these demographic variables (supplementary materials Table 3), addressing gaps not covered by SHAP. Our primary focus was on the interpretability of the remaining four features integral to the model: heart rate, SDNN, SD2, and the ratio SD1/SD2, which reflect distinct aspects of HRV. The SHAP summary plot (Fig. 7(A)) was used to understand the risk factors for depression globally. The horizontal position shows whether the effect of each value is associated with a higher or lower model prediction, with blue indicating a lower value and red indicating a higher value. Positive SHAP values (extending to the right in the graph) indicate that the feature is associated with an increased model prediction, while negative SHAP values (extending to the left in the graph) indicate that the feature is associated with a decreased model prediction. As shown in the figure, lower values of time domain parameter SDNN and nonlinearity parameter SD2 were associated with a significantly increased the predictive outcomes of the model, while higher heart rate values was associated with a significantly increased the predictive outcomes of the model. Nonlinearity parameter SD1 / SD2 influence on model output is not absolute positive correlation and negative correlation. In addition, the SHAP dependency plots have been plotted for more information using the 4 main features that affect the output of the XGBoost prediction model as examples (Fig. 7(B)-(D)). SHAP values for specific features exceed zero, representing an increased outcome of depression prediction model. Fig. 7SHAP diagrams of the XGboost model. (A) The SHAP summary plot of the XGBoost model.The Y-axis shows the name of the feature, and the X-axis shows the shapley value.(B) SHAP dependence plot for the heart rate. (c) SHAP dependence plot for the SDNN; (C) SHAP dependence plot for the SD2. (D) SHAP dependence plot for the SD1/SD2. The x axis represents the values of certain characteristic variable, and the SHAP values of characteristic variables are shown in the y axis SHAP diagrams of the XGboost model. (A) The SHAP summary plot of the XGBoost model.The Y-axis shows the name of the feature, and the X-axis shows the shapley value.(B) SHAP dependence plot for the heart rate. (c) SHAP dependence plot for the SDNN; (C) SHAP dependence plot for the SD2. (D) SHAP dependence plot for the SD1/SD2. The x axis represents the values of certain characteristic variable, and the SHAP values of characteristic variables are shown in the y axis Discussion In this study, our findings demonstrated that the HRV characteristic variables of patients with depression were significantly lower than those of non-depression population, while the heart rate is higher than those of non-depression population. Meanwhile, we developed and validated four depression prediction models using machine learning algorithms for evaluating the risk of depression in the study population. The XGBoost model showed the best performance in terms of discrimination and accuracy. Finally, the SHAP method was utilized to explain the XGBoost model and reveal the importance of characteristic variables for XGBoost prediction result. The top six variables with the greatest impact for the XGBoost model were heart rate, age, SDNN, SD2, sex and SD1/SD2, respectively. The proposed model might assist clinicians to more accurately discriminate patients with depression, so as to improve diagnosis and treatment strategies in time. One of our primary findings indicated that certain characteristic variables of HRV were used as clinical predictors for identifying patients with depression. The autonomic nervous system (ANS) has been shown to identify alterations in emotion processing associated with common psychopathology [56, 57], while HRV can be utilized to assess autonomic nervous system activity [58]. At present, although HRV has not yet become a specific predictor for the diagnosis of depression, as a non-invasive predictor to quantify autonomic nervous activity, which still has significant clinical value disease monitoring and treatment evaluation. Our findings had demonstrated that several characteristic variables of HRV were significantly differences between depressed patients and non-depression population. Previous research has found that depression patients had a significantly lower resting-state HRV for psychologically stressed compared to healthy individuals [59–62]. Our results indicated that time domain variables (SDNN, SDNN5, pNN50, rmsSD, SDSD), frequency domain variables (VLF, LF) and non-linear variables (SD1 and SD2) of HRV were lower in the depression group compared to the non-depression group, which was consistent with the result of previous studies. It is worth noting that most of the previous studies focused on the time domain and frequency domain variables of HRV, ignoring the importance of nonlinear variables of HRV. The non-linear variables have been shown to be robust and reliable, revealing valuable additional information regarding HRV characteristics in different patient populations, which can offer more comprehensive insights compared to linear methods alone [63]. Our study had found that the nonlinear variables (SD1 and SD2) were also significantly differences between the two groups, which further proved the clinical value of nonlinear variables of HRV. Another finding of our study indicate that four machine learning models can be used to as effective tools for identifying patients with depression. Among them, the XGBoost model shows the best prediction performance compared to the other three models. Despite the outstanding advantages and promising applications of machine learning models, we need to construct decision curves to further evaluate the clinical efficacy. The decision curves of the four predictive models show that the XGBoost model exhibits extremely high benefit values over a wide range of threshold levels, and the clinical benefit of the XGBoost model and the RF model were significantly greater than those of the SVM model and the LR model. The results further demonstrate the HRV-based XGBoost prediction model have strong prediction performance and excellent clinical utility. Therefore, when employing the depression scale to identify patients with depression, clinicians can combine machine learning models to assist in decision-making and enhance the accuracy and efficiency of diagnosis. Additionally, this model can also be used for the monitoring and prevention of depression, and timely detection of changes in HRV of patients, so that appropriate intervention measures can be taken to prevent further aggravation of depression. Furthermore, due to the inherent complexity of machine learning models and their opaque decision-making processes, which is challenging for users to comprehend and trust these models in clinical applications. Therefore, we employ the SHAP diagram to visualize the influence of the four features incorporated in the model on the XGBoost model. Heart rate, the time-domain parameter SDNN, the nonlinear parameter SD2, and SD1/SD2 are the four characteristics based on HRV that we primarily discuss. Mysa Saad et al. developed a classification algorithm using heart rate changes during sleep to distinguish depressed patients from healthy controls with an overall classification accuracy of 79.9% [64].Our study included heart rate as a predictive variable in the model and performed the interpretability analysis, with an accuracy rate of 87.27% in the training set and 79.17% in the test set of the optimal model, further verifying the feasibility of using heart rate as a biomarker for depression.Time domain parameters SDNN and nonlinear parameters SD2 and SD1/SD2 are the three HRV that have the greatest influence on XGBoost model. Human autonomic nervous system includes sympathetic nervous system and parasympathetic nervous system, and HRV is usually affected by parasympathetic autonomic nerve activation, including vagus nerve and sympathetic nerve activation [65, 66]. SDNN is influenced by vagus nerve activity [67]. The decrease in SDNN may reflect changes in the regulation of heart rate by the autonomic nervous system, especially the decreased activity of the parasympathetic nerve (vagus nerve) [68]. This change may be related to the pathophysiological mechanism of depression, such as autonomic nervous system dysfunction. Nonlinear parameters of HRV, such as SD2 and SD1/SD2, play an important role in assessing ANS function and its relationship to depression [69, 70]. SD2 is a crucial parameter in nonlinear analysis of heart rate variability, which is usually associated with long-term fluctuations or complexity of heart rate variability. People with depression generally have lower SD2 values, and this reduction is positively correlated with the severity of depression [71]. These results are consistent with our findings. The influence of nonlinear parameters SD1 / SD2 on the model output is not completely positive or negative correlation, but it is also one of the highly significant characteristics for the model. In patients with depression, this ratio is often abnormal, indicating that the stability of ANS is affected [72]. The XGBoost model based on SHAP analysis might assist in the early classification of depression and enhance the clinical usability of the model. It is crucial to emphasize that the evidence is insufficient for an intervention in clinical practice as of now, and further research is requisite to explore the applicability of this SHAP approach. Despite promising findings, our study had several limitations. Firstly, this was a single-center retrospective study, and the results were dependent on the composition of these limited data, which may have biased the results. Therefore, further prospective studies in multicenter populations are needed, and more factors that influence the occurrence of depression can be incorporated to build a comprehensive model combined with heart rate variability. Secondly, although the proposed model was explained using SHAP interpretable framework, there were still many unexplained parts of the machine learning modelling process, which would limit the generalization and application of the model. Finally, our study only focused on the association between depression and HRV characteristic variables, without fully considering the impact of other mental disorders, especially psychiatric comorbidities. Therefore, it’s worth considering to investigate the relationship between different types of mental disorders and HRV characteristic variables in the same way. Conclusions This study provides a reference for the feasibility of heart rate variability as a potential biomarker for depression, especially the nonlinear parameter indicator in heart rate variability. Changes in these parameters may provide new clues to the diagnosis of depression, especially when it is difficult to determine a diagnosis with traditional methods. The XGBoost model based on SHAP provides a quantitative auxiliary diagnostic tool for clinicians, which is conducive to improving the accuracy and efficiency of diagnosis, and can also be used for the monitoring and prevention of depression. Electronic Supplementary Material Below is the link to the electronic supplementary material. Supplementary Material 1 Supplementary Material 1