Predicting sleep disorder trajectories in older patients with chronic pain: an explainable machine learning approach Authors: Xiaoang Zhang, Yaqing Hu, Weichen Liu, Yuping Liao, Andi Zhou, Shushu Chen, Daying Zhang, Jianmei Wei Published: 2026 ABSTRACT AbstractBackgroundSleep disorders exhibit a high prevalence among chronic pain (CP) patients, yet knowledge gaps regarding sleep disorder trajectories in older CP patients may hinder healthy aging. This study aimed to identify sleep disorder trajectories and their predictive factors, and to develop tools for predicting these trajectories using an explainable machine learning (XML) approach.MethodsThis prospective cohort study was conducted in hospitalized older adults (aged ≥ 60 years) with chronic pain. Data on general characteristics, pain level, anxiety, depression, and perceived social support were collected at admission. Sleep disorder was assessed from admission to six months during routine chronic pain management, including pharmacologic treatment, non-pharmacologic interventions, psychosocial support, nursing care, and discharge education. Predictors were selected using Boruta algorithm and LASSO regression, and ten XML models were developed.ResultsAmong 596 patients, the prevalence of sleep disorders in the overall cohort ranged from 42.8% to 52.2% across the four assessment time points. Four heterogeneous sleep disorder trajectories (stable-low, high-declining, moderate-increasing, and high-persistent) and seven predictors were identified. The XML model determined pain level, depression, anxiety, and perceived social support were the most important predictors.ConclusionSleep disorders in older CP patients exhibited a high prevalence from admission to six months, manifesting in four distinct trajectories of deterioration or improvement. These findings may enhance the understanding of the heterogeneous progression of sleep disorders. Prioritizing key factors such as pain level, depression, anxiety, and perceived social support in preventive strategies and risk stratification may inform more targeted interventions to improve sleep health outcomes.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12888-026-08081-2.Keywords: Sleep disorder, Elderly, Chronic pain, Machine learning, Prediction Introduction The global aging population is rapidly increasing, with China experiencing particularly pronounced growth. China not only has a large population base of older adults, but the proportion of older adults in its total population is projected to reach 20% by mid-century [1]. Chronic pain (CP), defined as pain persisting or recurring for more than three months and exceeding normal tissue healing time, is among the most prevalent and debilitating conditions in older adults [2]. The prevalence of CP rises with age, with estimates suggesting that 41%-57% of this population is affected [3]. CP significantly impairs quality of life in older patients, leading to functional limitations, psychological distress, and substantial burdens on healthcare systems [1]. As global aging intensifies, the comprehensive management of CP in this growing population has emerged as a critical public health priority. Sleep disorders are highly prevalent among older patients with CP, characterized by difficulties in initiating or maintaining sleep, excessive daytime sleepiness, or other disruptions in sleep architecture [4]. These disorders are intricately linked to CP through mechanisms involving neuroendocrine dysregulation, inflammatory responses, and alterations in central pain modulation pathways [5]. Psychosocial factors, including depression, anxiety, and limited social support, can further exacerbate sleep disorders by amplifying pain perception, increasing emotional distress, and impairing coping capacity [6]. In turn, sleep disorders exacerbate pain sensitivity, reduce physical functioning, impair cognitive performance, and diminish overall well-being. Compared to the general older adult population, individuals with CP experience a significantly higher incidence of sleep disorders, with prevalence rates reaching up to 75% [4]. Given that sleep disorders represent a potentially modifiable risk factor [7], comprehensive assessment of sleep disorder risk and early preventive interventions are critical for healthcare professionals to enhance CP management and improve quality of life in this population. Healthcare professionals and researchers have actively assessed sleep disorders in clinical practice and studies. Existing research have documented varying degrees of sleep disorders among patients with chronic pain conditions such as knee osteoarthritis, fibromyalgia, and breast cancer, and have explored their associations with pain, anxiety, depression, and fatigue [8–11]. However, a key limitation is that, despite the dynamic nature of sleep disorders, characterized by fluctuations over time including persistence, improvement, or deterioration [12], most studies rely on cross-sectional designs or single time-point assessments, primarily focusing on prevalence estimates or associations with selected factors. Although a few longitudinal studies, particularly in breast cancer populations, have identified heterogeneous sleep trajectories (e.g., low, moderate, and high levels) [12], these investigations remain limited to specific disease groups and fail to fully capture the temporal heterogeneity of sleep disorders in older adults with CP. Consequently, elucidating the trajectories of sleep disorders and identifying high-risk individuals early are critical for developing proactive and targeted clinical interventions. More importantly, understanding the predictors of sleep disorder trajectories is crucial for the early identification of individuals at high risk of persistent or severe sleep disorders [13]. Previous cross-sectional studies have explored factors associated with sleep disorders. Biological factors, such as pain level and duration, have been consistently linked to sleep disorders, while psychosocial and behavioral factors, including anxiety, depression, social support, and tea or coffee consumption, have also been increasingly explored for their influence on sleep patterns [14, 15]. However, it remains unclear whether these factors can reliably predict high-risk sleep disorder trajectories over time. Moreover, existing studies often rely on traditional logistic regression models with relatively limited variable sets, primarily emphasizing biological and psychological factors while underrepresenting behavioral and social domains [15]. This narrow focus restricts a comprehensive understanding of the complex and nonlinear interactions underlying sleep disorder trajectories. In contrast, the present study incorporates a broader range of variables, including pain level and duration, anxiety and depression, lifestyle factors such as tea or coffee consumption and smartphone use, and perceived social support, enabling a more comprehensive biopsychosocial assessment. Machine learning (ML), as a branch of artificial intelligence, has demonstrated remarkable success in predicting a wide range of medical outcomes [16]. Explainable machine learning (XML) models, particularly those leveraging SHapley Additive exPlanations (SHAP), further provide healthcare professionals with an interpretable framework to understand model outputs [17]. By visually representing the relationships between predictors and trajectories through intuitive visualizations, SHAP enables healthcare professionals to discern the magnitude and direction of each predictor’s contribution to sleep disorder trajectories, facilitating the development of more precise clinical strategies. While models have been developed to predict outcomes such as frailty and cognitive decline in older patients with CP [17, 18], the value of XML in predicting sleep disorder trajectories remains underexplored. This study aims to identify distinct subgroups of sleep disorder trajectories and their predictors among older patients with CP, and to develop ten XML models to predict these trajectories. The findings may enhance healthcare professionals’ understanding of the risk of worsening sleep disorders and help inform risk stratification and more targeted, proactive sleep management strategies, with potential to improve patient outcomes. Methods Study design This prospective cohort study adhered to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines [19]. The study comprised three stages. First, eligible older patients with CP were recruited, and baseline data on sleep disorders, general characteristics, pain level, anxiety, depression, and social support were collected at admission, with follow-up sleep assessments at 1, 3, and 6 months post-admission. Second, Growth Mixture Model (GMM) was applied to identify distinct sleep disorder trajectory subgroups, and key predictors were selected using the Boruta algorithm and LASSO regression. Finally, ten XML models were developed to predict trajectory subgroups, with SHAP employed to interpret the contributions and directions of the predictors on sleep disorder trajectories. Ethics approval This study received ethical approval from the Medical Ethics Committee of the First Affiliated Hospital of Nanchang University (IIT2024722). All procedures were conducted in accordance with the ethical standards outlined in the Declaration of Helsinki and Ethics institutional guidelines. Written informed consent was obtained from all participants prior to enrollment, ensuring the protection of their rights, privacy, and confidentiality throughout the research process. Study participants and sample size Older adults with CP were recruited from the pain ward of a tertiary hospital in Nanchang, China, between August 2024 and March 2025. Participants had various types of chronic pain conditions (e.g., musculoskeletal and neuropathic pain) and received an integrative biopsychosocial pain management in accordance with hospital clinical protocols, including pharmacologic treatment, non-pharmacologic interventions (e.g., physical therapy and rehabilitation), psychosocial support, nursing care, and discharge education. Sleep hygiene education (e.g., guidance on sleep routines and environmental optimization) was routinely incorporated during hospitalization and follow-up. The duration of hospitalization followed routine clinical practice. Inclusion criteria were as follows: (1) age 60 years or older; (2) meeting the International Association for the Study of Pain (IASP) diagnostic criteria for CP [20], defined as pain persisting for at least three months, occurring daily or near-daily, with a Visual Analog Scale score ≥ 3; (3) ability to engage in effective communication; and (4) willingness and capacity to complete a 6-month follow-up questionnaire survey. Exclusion criteria included: (1) severe psychiatric or neurological disorders, such as dementia; (2) presence of malignant tumors or receipt of palliative care; (3) severe cardiac, cerebral, pulmonary, renal, or hepatic dysfunction; and (4) severe immune-related diseases, such as acquired immunodeficiency syndrome or leukemia. Sample size was estimated using the events per variable (EPV) ≥ 10 method [21]. Accounting for 20 candidate variables and an anticipated 20% dropout rate, a minimum of 250 patients was required. Given that the training set represented 70% of the total sample for the XML model, the minimum total sample size was estimated at 357. To further enhance the reliability of results and optimize model performance, we ultimately recruited 637 patients. Outcome variable The primary outcome variable in this study was sleep disorders, assessed using the Pittsburgh Sleep Quality Index (PSQI) at four time points: hospital admission (T0), and 1 (T1), 3 (T2), 6 (T3) months post-admission [22]. These follow-up intervals were selected to capture changes in sleep disorder trajectories during inpatient analgesic treatment and nursing, as well as post-discharge home and community care, thereby identifying potential intervention windows [23]. The PSQI is a widely validated self-report questionnaire comprising 19 items across seven components: subjective sleep quality, sleep latency, sleep duration, habitual sleep efficiency, sleep disturbances, use of sleep medication, and daytime dysfunction. The global score ranges from 0 to 21, with higher scores indicating poorer sleep quality. A total score exceeding 7 indicates significant sleep disorders. To ensure the reliability of PSQI assessments, a pilot evaluation was conducted prior to formal data collection, involving two assessors evaluating 30 patients. The resulting Kappa coefficients ranged from 0.86 to 0.92, demonstrating excellent inter-rater consistency. Candidate variables Based on prior research, clinical relevance, and expert panel discussions, candidate variables were selected. These variables encompass physical, psychological, and social dimensions associated with sleep disorders, ensuring a comprehensive exploration of factors influencing sleep disorder trajectories. The selected variables are routinely assessed or easily obtainable in clinical settings, facilitating their evaluation and the potential translation of findings without imposing additional burden in clinical practice. General characteristics were collected using a self-report questionnaire and included age, gender, body mass index (BMI), education level, employment status, marital status, smoking status, alcohol consumption, tea or coffee consumption, living arrangement, monthly household income, smartphone use, pain location, pain duration, pain characteristics, and number of pain sites. Pain level was assessed using the Visual Analogue Scale (VAS), a 10-cm horizontal line ranging from 0 (“no pain”) to 10 (“worst imaginable pain”). Participants were asked to indicate the point that best represented their average pain level over the past week, with higher scores reflecting greater pain level [24]. Symptoms of anxiety and depression were evaluated using the Hospital Anxiety and Depression Scale (HADS), which comprises 14 items divided equally between the anxiety (HADS-A) and depression (HADS-D) subscales [25]. Each item is rated on a 4-point Likert scale (0–3), with subscale scores ranging from 0 to 21. Higher scores indicate greater symptom severity, and a score of 8 or higher on either subscale suggests the presence of clinically relevant anxiety or depressive symptoms. Perceived social support was measured using the Perceived Social Support Scale (PSSS), which consists of 12 items assessing support from family, friends, and significant others [26]. Each item is rated on a 7-point Likert scale (1= “very strongly disagree” to 7= “very strongly agree”), with higher total scores indicating stronger perceived social support. Data collection PSQI and candidate variable data were collected at baseline, defined as within 48 h of hospital admission, when patients were undergoing initial clinical evaluation and treatment. Follow-up PSQI assessments were conducted at 1 month, 3 months, and 6 months post-admission, covering both inpatient treatment and post-discharge recovery periods. A stringent quality control protocol was implemented throughout the data collection process. Prior to commencement, all research personnel underwent standardized training, utilizing uniform scripts and guidelines to minimize interviewer bias. At baseline, data were collected through face-to-face interviews conducted in a private hospital ward setting to ensure response accuracy. Follow-up assessments were conducted by telephone, with scripted reminders sent 48 h in advance and interviews scheduled at convenient times. Interviewers provided only neutral prompts to guide questionnaire completion, and questionnaires were checked on-site for completeness and logical consistency. Any missing or ambiguous responses were clarified directly with participants. Small tokens of appreciation were provided to participants after each data collection session. All data were entered using a double-entry method, and logical checks were performed regularly to identify inconsistencies such as identical responses across items. A total of 637 patients were initially recruited, of which 596 provided valid data. 41 dropped out due to invalid questionnaires (n = 19), loss to follow-up (n = 14), loss of interest (n = 5), or transfer due to health deterioration (n = 3). Identification of sleep disorder trajectories Distinct sleep disorder trajectory subgroups were identified using GMM. GMM offers flexibility by allowing individual variation in intercepts, slopes, and residuals within each latent class, making it suitable for modeling complex, non-linear changes and inter-individual variability in sleep disorders over time [27]. In contrast, group-based trajectory modeling assumes homogeneity of growth parameters within classes, whereas linear mixed-effects models assume a single trajectory with random effects, potentially oversimplifying heterogeneity [28]. The optimal number of trajectory subgroups was determined based on statistical fit and clinical interpretability. Lower values of the Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and Sample-Size Adjusted BIC (SABIC) indicated superior model fit, while entropy values approaching 1.0 reflected clearer class separation. The Lo-Mendell-Rubin likelihood ratio test (LMR-LRT) and bootstrap likelihood ratio test (BLRT) were further used to compare models with k versus k-1 classes to confirm model adequacy. To further evaluate the clinical relevance of the identified trajectory groups, the Barthel Index (BI) [29], a widely used measure of functional independence in activities of daily living, was assessed at 6 months post-admission (T3) and compared across trajectory groups. Variable selection Predictors of sleep disorder trajectories were selected within training set using the Boruta algorithm and LASSO regression. Boruta, a random forest-based wrapper method, conducts all-relevant feature selection by comparing original variables against permuted shadow features, reducing the risk of omitting informative predictors [30]. LASSO regression, a penalized regression technique, enhances interpretability by shrinking less relevant coefficients to zero, preventing overfitting [31]. Combining Boruta and LASSO enables healthcare professionals to identify variables with complex nonlinear associations while simplifying the model structure, ensuring both the clinical relevance and statistical robustness of the selected predictors for sleep disorder trajectory prediction. XML models development The dataset was randomly split into training (70%) and testing (30%) sets for model training and validation, respectively. To ensure data comparability, min-max normalization was applied to standardize feature scales. Ten XML models were developed to predict sleep disorder trajectories [31], as they represent diverse algorithmic approaches, each excelling in interpretability, predictive power, and capturing non-linear relationships: Logistic Regression (LR), Support Vector Machine (SVM), Adaptive Boosting (AdaBoost), K-Nearest Neighbors (KNN), Random Forest (RF), Light Gradient Boosting Machine (LightGBM), Gradient Boosting Decision Tree (GBDT), Extreme Gradient Boosting (XGBoost), Stacking, and Gaussian Naive Bayes (GNB). LR provides interpretability through linear coefficients; SVM effectively handles high-dimensional data; AdaBoost enhances weak learners through iterative weighting; KNN is intuitive and nonparametric; RF reduces variance via ensemble averaging; LightGBM and GBDT improve computational efficiency and accuracy on large datasets; XGBoost enhances generalization by regularization; Stacking integrates multiple learners for higher predictive performance; and GNB offers simplicity and robustness under conditional independence assumptions. Hyperparameters were optimized on the training set via grid search and manual fine-tuning [17]. The predictive performance was evaluated on the training set using accuracy, precision, recall, F1 score, specificity, Receiver Operating Characteristic (ROC) curves with Area Under the Curve (AUC), calibration curves, Brier score, and Decision Curve Analysis (DCA) to identify the optimal model [32]. The testing set is used for the final evaluation of the optimal model to assess its generalization ability. Finally, SHAP was used to interpret the optimal model [17]. Statistical analysis Statistical analyses were conducted using SPSS 25.0 and Python 3.12. Continuous variables were tested for normality and are presented as mean ± standard deviation (SD) for normally distributed data or median (interquartile range) for non-normally distributed data; group comparisons were performed using the t-test or Mann-Whitney U test. Categorical variables were summarized as frequencies and percentages, and comparisons between groups were conducted using the chi-square test. P < 0.05 was considered statistically significant. Results Baseline characteristics of participants A total of 596 participants were included in this study. The mean age was (71.4 ± 6.0) years, and 60.4% were female. Nearly half of participants had a normal BMI (46.6%), and the majority had completed primary (34.4%) or junior secondary education (30.4%). At admission, the mean PSQI score was (10.2 ± 3.1), the mean VAS score was (7.2 ± 1.4), and the mean HADS-A and HADS-D scores were (9.5 ± 3.5) and (7.9 ± 3.0), respectively. The mean PSSS score was (47.4 ± 5.1). Detailed baseline characteristics are presented in Table 1. Table 1Baseline characteristics of participants in different trajectories (N = 596)CharacteristicTotalSleep disorder trajectoriesLSHDMIHP P Age (years old) Mean (SD)71.44 ± 6.01471.64 ± 5.94372.27 ± 6.26671.13 ± 5.97470.80 ± 5.7980.198Gender Male Female236 (39.6%)360 (60.4%)61 (40.2%)90 (59.8%)50 (37.6%)83 (62.4%)58 (41.4%)82 (58.6%)67 (39.0%)105 (61.0%)0.490BMI Underweight Normal Overweight or obese87 (14.6%)278 (46.6%)231 (38.8%)25 (16.6%)81 (53.6%)45 (29.8%)24 (18.0%)65 (48.9%)44 (33.1%)18 (12.8%)63 (45.0%)59 (42.1%)20 (11.6%)69 (40.1%)83 (48.3%)0.022Educational level Primary school or below Junior high school High school University or above205 (34.4%)181 (30.4%)119 (20.0%)91 (15.2%)61 (40.4%)42 (27.8%)26 (17.2%)22 (14.6%)39 (29.3%)33 (24.8%)37 (27.8%)24 (18.1%)53 (37.9%)45 (32.1%)22 (15.7%)20 (14.3%)52 (30.2%)61 (35.5%)34 (19.8%)25 (14.5%)0.121Employment status Unemployment/Retirement Employed400 (67.1%)196 (32.9%)100 (66.2%)51 (33.8%)91 (68.4%)42 (31.6%)92 (65.7%)48 (34.3%)117 (68.0%)55 (32.0%)0.797Marital status Unmarried/widowed/divorced Married111 (18.6%)485 (81.4%)29 (19.2%)122 (80.8%)30 (22.6%)103 (77.4%)24 (17.1%)116 (82.9%)28 (16.3%)144 (83.7%)0.529Smoking No Yes325 (54.5%)271 (45.5%)98 (64.9%)53 (35.1%)77 (57.9%)56 (42.1%)71 (50.7%)69 (49.3%)79 (45.9%)93 (54.1%)0.004Alcohol consumption No Responsible drinking Alcohol abuse320 (53.7%)215 (36.1%)61 (10.2%)84 (55.6%)57 (37.8%)10 (6.6%)71 (53.4%)49 (36.8%)13 (9.8%)84 (60.0%)38 (27.1%)18 (12.9%)81 (47.1%)71 (41.3%)20 (11.6%)0.113Tea or coffee consumption None Occasional Frequent286 (48.0%)204 (34.2%)106 (17.8%)104 (68.9%)35 (23.2%)12 (7.9%)77 (57.9%)38 (28.6%)18 (13.5%)61 (43.6%)51 (36.4%)28 (20.0%)44 (25.6%)80 (46.5%)48 (27.9%)< 0.001Living arrangements Live alone Live with family180 (30.2%)416 (69.8%)47 (31.1%)104 (68.9%)45 (33.8%)88 (66.2%)39 (27.9%)101 (72.1%)49 (28.5%)123 (71.5%)0.683Monthly household income (CNY) < 3000 3000–5000 5000–10,000 > 10,000132 (22.1%)267 (44.8%)128 (21.5%)69 (11.6%)35 (23.2%)65 (43.1%)31 20.5%)20 (13.2%)26 (19.5%)61 (45.9%)28 (21.1%)18 (13.5%)37 (26.4%)64 (45.7%)30 (21.4%)9 (6.4%)34 (19.8%)77 (44.8%)39 (22.7%)22 (12.7%)0.673Smartphone use None Proficient292 (49.0%)304 (51.0%)95 (62.9%)56 (37.1%)68 (51.1%)65 (48.9%)64 (45.7%)76 (54.3%)65 (37.8%)107 (62.2%)< 0.001Pain location Lower back Neck/shoulder Limbs Trunk Multiple sites (number of pain sites ≥ 2)72 (12.1%)49 (8.2%)42 (7.1%)21 (3.5%)412 (69.1%)22 (14.6%)14 (9.3%)11 (7.3%)7 (4.6%)97 (64.2%)20 (15.0%)12 (9.0%)9 (6.8%)6 (4.5%)86 (64.7%)14 (10.0%)11 (7.9%)6 (4.2%)4 (2.9%)105 (75.0%)16 (9.3%)12 (7.0%)16 (9.3%)4 (2.3%)124 (72.1%)0.583Pain duration (month) 3–6 7–11 ≥ 12167 (28.0%)228 (38.3%)201 (33.7%)63 (41.7%)52 (34.5%)36 (23.8%)42 (31.6%)52 (39.1%)39 (29.3%)36 (25.7%)51 (36.4%)53 (37.9%)26 (15.1%)73 (42.4%)73 (42.4%)< 0.001Pain characteristics Stabbing pain Cutting pain Burning pain Bloating pain211 (35.4%)135 (22.7%)149 (25.0%)101 (16.9%)57 (37.7%)33 (21.9%)30 (19.9%)31 (20.5%)50 (37.6%)27 (20.3%)37 (27.8%)19 (14.3%)49 (35.0%)33 (23.6%)37 (26.4%)21 (15.0%)55 (32.0%)42 (24.4%)45 (26.2%)30 (17.4%)0.749Number of pain sites 1–2 3–4 ≥ 5279 (46.8%)216 (36.3%)101 (16.9%)78 (51.7%)53 (35.1%)20 (13.2%)67 (50.4%)44 (33.1%)22 (16.5%)65 (46.4%)54 (38.6%)21 (15.0%)69 (40.1%)65 (37.8%)38 (22.1%)0.399Pain level Mean (SD)7.22 ± 1.4435.92 ± 1.6597.72 ± 1.3146.67 ± 1.3918.42 ± 1.320< 0.001Anxiety Mean (SD)9.52 ± 3.5366.96 ± 3.4298.46 ± 3.63510.33 ± 3.26212.26 ± 3.813< 0.001Depression Mean (SD)7.93 ± 3.0415.44 ± 3.0327.03 ± 3.0188.94 ± 2.78010.27 ± 3.238< 0.001Perceived social support Mean (SD)47.42 ± 5.13850.65 ± 4.92948.69 ± 4.84146.16 ± 4.98043.95 ± 5.547< 0.001SD: standard deviation. BMI: body mass index. LS: low-stable. HD: high-declining. MI: moderate-increasing. HP: high-persistent Baseline characteristics of participants in different trajectories (N = 596) Age (years old) Mean (SD) Gender Male Female 236 (39.6%) 360 (60.4%) 61 (40.2%) 90 (59.8%) 50 (37.6%) 83 (62.4%) 58 (41.4%) 82 (58.6%) 67 (39.0%) 105 (61.0%) BMI Underweight Normal Overweight or obese 87 (14.6%) 278 (46.6%) 231 (38.8%) 25 (16.6%) 81 (53.6%) 45 (29.8%) 24 (18.0%) 65 (48.9%) 44 (33.1%) 18 (12.8%) 63 (45.0%) 59 (42.1%) 20 (11.6%) 69 (40.1%) 83 (48.3%) Educational level Primary school or below Junior high school High school University or above 205 (34.4%) 181 (30.4%) 119 (20.0%) 91 (15.2%) 61 (40.4%) 42 (27.8%) 26 (17.2%) 22 (14.6%) 39 (29.3%) 33 (24.8%) 37 (27.8%) 24 (18.1%) 53 (37.9%) 45 (32.1%) 22 (15.7%) 20 (14.3%) 52 (30.2%) 61 (35.5%) 34 (19.8%) 25 (14.5%) Employment status Unemployment/Retirement Employed 400 (67.1%) 196 (32.9%) 100 (66.2%) 51 (33.8%) 91 (68.4%) 42 (31.6%) 92 (65.7%) 48 (34.3%) 117 (68.0%) 55 (32.0%) Marital status Unmarried/widowed/divorced Married 111 (18.6%) 485 (81.4%) 29 (19.2%) 122 (80.8%) 30 (22.6%) 103 (77.4%) 24 (17.1%) 116 (82.9%) 28 (16.3%) 144 (83.7%) Smoking No Yes 325 (54.5%) 271 (45.5%) 98 (64.9%) 53 (35.1%) 77 (57.9%) 56 (42.1%) 71 (50.7%) 69 (49.3%) 79 (45.9%) 93 (54.1%) Alcohol consumption No Responsible drinking Alcohol abuse 320 (53.7%) 215 (36.1%) 61 (10.2%) 84 (55.6%) 57 (37.8%) 10 (6.6%) 71 (53.4%) 49 (36.8%) 13 (9.8%) 84 (60.0%) 38 (27.1%) 18 (12.9%) 81 (47.1%) 71 (41.3%) 20 (11.6%) Tea or coffee consumption None Occasional Frequent 286 (48.0%) 204 (34.2%) 106 (17.8%) 104 (68.9%) 35 (23.2%) 12 (7.9%) 77 (57.9%) 38 (28.6%) 18 (13.5%) 61 (43.6%) 51 (36.4%) 28 (20.0%) 44 (25.6%) 80 (46.5%) 48 (27.9%) Living arrangements Live alone Live with family 180 (30.2%) 416 (69.8%) 47 (31.1%) 104 (68.9%) 45 (33.8%) 88 (66.2%) 39 (27.9%) 101 (72.1%) 49 (28.5%) 123 (71.5%) Monthly household income (CNY) < 3000 3000–5000 5000–10,000 > 10,000 132 (22.1%) 267 (44.8%) 128 (21.5%) 69 (11.6%) 35 (23.2%) 65 (43.1%) 31 20.5%) 20 (13.2%) 26 (19.5%) 61 (45.9%) 28 (21.1%) 18 (13.5%) 37 (26.4%) 64 (45.7%) 30 (21.4%) 9 (6.4%) 34 (19.8%) 77 (44.8%) 39 (22.7%) 22 (12.7%) Smartphone use None Proficient 292 (49.0%) 304 (51.0%) 95 (62.9%) 56 (37.1%) 68 (51.1%) 65 (48.9%) 64 (45.7%) 76 (54.3%) 65 (37.8%) 107 (62.2%) Pain location Lower back Neck/shoulder Limbs Trunk Multiple sites (number of pain sites ≥ 2) 72 (12.1%) 49 (8.2%) 42 (7.1%) 21 (3.5%) 412 (69.1%) 22 (14.6%) 14 (9.3%) 11 (7.3%) 7 (4.6%) 97 (64.2%) 20 (15.0%) 12 (9.0%) 9 (6.8%) 6 (4.5%) 86 (64.7%) 14 (10.0%) 11 (7.9%) 6 (4.2%) 4 (2.9%) 105 (75.0%) 16 (9.3%) 12 (7.0%) 16 (9.3%) 4 (2.3%) 124 (72.1%) Pain duration (month) 3–6 7–11 ≥ 12 167 (28.0%) 228 (38.3%) 201 (33.7%) 63 (41.7%) 52 (34.5%) 36 (23.8%) 42 (31.6%) 52 (39.1%) 39 (29.3%) 36 (25.7%) 51 (36.4%) 53 (37.9%) 26 (15.1%) 73 (42.4%) 73 (42.4%) Pain characteristics Stabbing pain Cutting pain Burning pain Bloating pain 211 (35.4%) 135 (22.7%) 149 (25.0%) 101 (16.9%) 57 (37.7%) 33 (21.9%) 30 (19.9%) 31 (20.5%) 50 (37.6%) 27 (20.3%) 37 (27.8%) 19 (14.3%) 49 (35.0%) 33 (23.6%) 37 (26.4%) 21 (15.0%) 55 (32.0%) 42 (24.4%) 45 (26.2%) 30 (17.4%) Number of pain sites 1–2 3–4 ≥ 5 279 (46.8%) 216 (36.3%) 101 (16.9%) 78 (51.7%) 53 (35.1%) 20 (13.2%) 67 (50.4%) 44 (33.1%) 22 (16.5%) 65 (46.4%) 54 (38.6%) 21 (15.0%) 69 (40.1%) 65 (37.8%) 38 (22.1%) Pain level Mean (SD) Anxiety Mean (SD) Depression Mean (SD) Perceived social support Mean (SD) SD: standard deviation. BMI: body mass index. LS: low-stable. HD: high-declining. MI: moderate-increasing. HP: high-persistent Sleep disorder prevalence and trajectories The prevalence of sleep disorders in the overall cohort varied across the four time points (T0-T3), ranging from 42.8% to 52.2%. GMM was fitted to data with varying numbers of trajectory subgroups, indicating that a 4-class model provided the optimal fit (Table 2). The Average Posterior Probabilities (AvePP) and the Odds of Correct Classification (OCC) for the selected 4-class model are presented in Table 3. The identified trajectories, depicted in Fig. 1A, were labeled as: “Low-stable (LS)” (25.3%), characterized by consistently low PSQI scores throughout follow-up; “High-declining (HD)” (22.3%), with high baseline scores that gradually improved; “Moderate-increasing (MI)” (23.5%), exhibiting progressively increasing scores from a moderate baseline; and “High-persistent (HP)” (28.9%), with persistently elevated PSQI scores. Subgroup-specific sleep disorder prevalence ranges across the four time points were 7.3%-14.8% (LS), 19.3%-66.4% (HD), 45.2%-72.1% (MI), and 89.9%-95.3% (HP) (Fig. 1B). Table 2Result of GMM fittingClassesLog-likelihoodAICBICSABICEntropyLMRBLRT1-2754.0175508.0335569.4965544.344---2-2640.8055281.6095408.9265355.4460.727< 0.001< 0.0013-2565.9335131.8665325.0375243.2290.8890.003< 0.0014-2442.9014885.8025144.8265034.6910.9230.021< 0.0015-2564.3915128.7815253.6595175.1970.9140.1890.076-2640.5695281.1385371.8695325.0800.9090.3240.21 Result of GMM fitting Table 3AvePP and OCC for the 4-class modelClassProportionAvePPOCCLow-stable0.2530.91215.814High-declining0.2230.86210.925Moderate-increasing0.2350.8449.302High-persistent0.2890.93118.643AvePP: the average posterior probabilities. OCC: the odds of correct classification AvePP and OCC for the 4-class model AvePP: the average posterior probabilities. OCC: the odds of correct classification Trajectories and prevalence of sleep disorders at T0, T1, T2, and T3. (A) Trajectories of the four sleep disorder subgroups. (B) Prevalence of sleep disorders across the four subgroups Compared to the LS group, participants in the HD, MI, and HP groups had a higher proportion of overweight individuals, greater smoking prevalence, more frequent tea or coffee consumption, higher smartphone usage, longer pain duration, greater pain level, elevated anxiety and depression scores, and reduced social support, with all differences statistically significant (P < 0.05, Table 1). To explore the clinical relevance of the HP and HD trajectories, functional outcomes at 6 months post-admission (T3) were compared using BI. The HP group had significantly lower BI scores than the HD group (65.4 ± 12.8 vs. 80.1 ± 10.5; t = 10.758, P < 0.001), indicating poorer functional status and supporting the potential clinical distinctiveness of these trajectories. Selected predictors The Boruta algorithm identified eight key variables based on feature importance parameters, including pain level, depression, social support, anxiety, pain duration, tea or coffee consumption, smartphone use, and BMI (Fig. 2A). Similarly, LASSO regression, using optimal penalization to minimize overfitting, selected depression, pain level, social support, anxiety, pain duration, smartphone use, tea or coffee consumption, and smoking as significant predictors (Fig. 2B). Predictors consistently identified by both methods, including tea or coffee consumption, smartphone use, pain duration, pain level, anxiety, depression, and perceived social support, were ultimately retained for XML model construction The Venn diagram illustrating predictor selection and the four-class patient profiles based on the identified predictors are presented in Supplementary Material. Results of predictor selection and patient profiles. (A) Predictors selected by the Boruta algorithm. (B) Predictors selected by LASSO regression Model evaluation and comparison After developing and training ten XML models, we calculated and compared their performance metrics. Overall model accuracy was evaluated at the multiclass level by considering predictions across all trajectory categories. To provide a more detailed assessment of classification performance, a one-vs-rest framework was employed to derive trajectory-specific metrics, including precision, recall, F1 score, specificity, and ROC curves with corresponding AUC values. These class-level performance indices were then aggregated using a macro-averaging strategy to summarize overall discriminative performance. Model calibration was examined using a unified calibration curve generated by pooling predicted probabilities from all trajectory classes. For this analysis, predicted probabilities were stratified into deciles, and the average predicted risk within each decile was compared against the empirically observed outcome frequency to evaluate agreement between predictions and observed data. DCA was conducted by collapsing the original multiclass outcome into a clinically relevant binary classification, contrasting the HP trajectory with the combined group of all remaining trajectories. Performance comparisons of the XML models are detailed in Supplementary Material. The RF model demonstrated the best overall performance, achieving an accuracy of 0.83, precision of 0.82, recall of 0.79, F1 score of 0.79, and specificity of 0.94. ROC curves further confirmed RF’s superior ability to discriminate between sleep disorder trajectory subgroups, with an AUC of 0.87 (Fig. 3A). The calibration curve demonstrated that the RF model’s predicted probabilities for sleep disorder trajectories closely aligned with observed outcomes, indicating strong model reliability (Brier score: 0.018) (Fig. 3B). Moreover, DCA revealed that RF provided the highest net benefit across a wide range of threshold probabilities, supporting its clinical utility for risk stratification and decision-making in clinical practice (Fig. 3C). Performance of the RF model on the testing set was largely concordant with the results obtained from internal validation. The RF model was prioritized for SHAP interpretation due to its superior overall performance. ROC, calibration, and decision curve analysis of ten XML models. (A) ROC curves. (B) Calibration curves. (C) DCA for clinical net benefit Model interpretation Using the LS group as the reference, SHAP values were calculated for each predictor relative to the HD, MI, and HP trajectory subgroups. Figure 4A indicates that pain level, depression, and perceived social support were the three most important predictors for the HD trajectory. Interestingly, pain level, depression, and anxiety emerged as the most influential predictors for both the MI trajectory (Fig. 4B) and the HP trajectory (Fig. 4C). The feature importance plots for each trajectory are presented in Supplementary Material. SHAP summary plots for three sleep disorder trajectories relative to SL group reference. (A) SHAP summary plot for HD group. (B) SHAP summary plot for MI group. (C) SHAP summary plot for HP group The waterfall plot in Fig. 5A illustrates an individual patient with a depression score of 7, a pain level score of 8, and an anxiety score of 6, for whom occasional tea or coffee consumption and a pain duration of 7–11 months collectively contribute to classification into the HD trajectory. Waterfall plots in Fig. 5B and C similarly depict individual-level risk profiles for patients in the other trajectory groups. SHAP dependence plots revealed non-linear relationships between key predictors and the risk of the HP trajectory (Fig. 6). SHAP values became positive with occasional or frequent tea or coffee consumption, proficient smartphone use, pain duration exceeding approximately 7 months, pain levels above approximately 7, and anxiety and depression scores exceeding approximately 6, respectively, indicating an increased likelihood of classification into the HP group. In contrast, higher perceived social support (above approximately 45) was associated with a reduced risk. Sankey diagrams in Supplementary Material visualize the relative contribution patterns of predictors across the LS, HD, MI, and HP trajectories. SHAP waterfall plots for three sleep disorder Trajectories relative to SL reference Group. (A) Waterfall plot for a patient with HD trajectory. (B) Waterfall plot for a patient with MI trajectory. (C) Waterfall plot for a patient with HP trajectory SHAP dependence plots for the HP Trajectory Discussion To our knowledge, this is the first study to apply XML models to predict sleep disorder trajectories in older patients with CP. We identified four distinct sleep disorder trajectories: LS, HD, MI, and HP, and determined that tea or coffee consumption, smartphone use, pain duration, pain level, anxiety, depression, and perceived social support were predictors of these trajectories. Through the XML framework, pain level, depression, anxiety, and perceived social support emerged as the most important predictors. These findings enhance understanding of the heterogeneity of sleep disorder trajectories and may facilitate early identification of high-risk individuals who may benefit from intensified monitoring and targeted interventions. In this study, the prevalence of sleep disorders in the overall cohort of older patients with CP ranged from 42.8% to 52.2% across the four time points (T0-T3), which was lower than the pooled prevalence among CP populations in a meta-analysis (75.3%) [4], suggesting relatively better sleep hygiene. This may be attributed to the integrative biopsychosocial approach to pain management adopted by our clinical team, which emphasizes comprehensive management strategies and incorporates individualized sleep hygiene education throughout hospitalization and post-discharge follow-up. However, a cross-sectional survey of breast cancer patients reported a lower prevalence of 30.6% [10], suggesting variability across populations. Notably, the LS subgroup exhibited the most favorable outcomes, with prevalence ranging from 7.3% to 14.8%, although it comprised only 25.3% of the cohort. Overall, greater attention to sleep health and more comprehensive intervention strategies remains warranted for older adults with CP. Recognizing that a one-size-fits-all approach to sleep hygiene may not effectively enhance long-term outcomes, we identified four heterogeneous sleep disorder trajectories, namely LS, HD, MI, and HP, to support risk stratification and targeted interventions. The LS trajectory, characterized by consistently low sleep disorder, represents the most favorable outcome. A similar stable pattern was reported by Yun et al. in multiple sclerosis outpatients, with median PSQI scores remaining around 6 over one year [33]. However, 54.2% of that cohort had sleep disorders at admission, compared with only 14.8% in our LS group. The HD group displayed high initial sleep disorder upon admission but progressive improvement during hospitalization and post-discharge, likely attributable to effective treatment responses and sustained family or peer support, which enhanced motivation for adopting robust sleep hygiene practices [5]. In contrast, the MI group, despite moderate initial sleep disorder, experienced progressive deterioration in sleep quality throughout treatment. This trajectory may stem from heightened catastrophizing and increased psychological distress, leading to hyperarousal [34]. Therefore, greater attention to patients with moderate-to-severe baseline sleep disturbance may be important to prevent deterioration and improve sleep outcomes [2]. The HP group maintained persistently high sleep disorder despite slight improvement. Similarly, Fontes et al. reported a persistent high-level sleep disorder trajectory among breast cancer patients over three years following diagnosis, with gradual improvement observed within the first year [12]. This sustained high-risk pattern may partly reflect differences in clinical course and treatment exposure. Specifically, patients in the HP group may have greater disease severity, higher psychological burden, and poorer treatment response, often requiring longer hospitalization, which may contribute to sustained sleep disorders. Accordingly, patients in the HP group may require closer monitoring and more intensive sleep-focused interventions, including sleep hygiene support and specialist referral when necessary [5]. We identified factors associated with sleep disorder trajectories to facilitate early identification of patients at risk of worsening sleep patterns. Consistent with previous research on factors associated with sleep quality in patients with chronic widespread pain [15], our findings confirm pain level, depression, and anxiety as key factors of sleep disorder trajectories. According to the Cognitive Model of Insomnia, chronic pain may increase nocturnal arousal and discomfort, while anxiety and depression perpetuate insomnia through maladaptive cognitions such as worry and catastrophizing, reinforcing hyperarousal and a self-perpetuating cycle of pain, emotional distress, and sleep dysfunction [35]. SHAP analysis further quantified these effects, demonstrating that pain level, depression, and anxiety were the most influential factors associated with worsening sleep disorder. Notably, perceived social support emerged as a key factor, ranking among the top three contributors associated with improved sleep trajectories. This finding further underscores its critical role and extends previous research demonstrating an association between social support and sleep quality in older adults with CP [36]. From the perspective of the biopsychosocial model, perceived social support may protect sleep by buffering stress, enhancing emotional regulation, and promoting adaptive coping with pain. Support from family or peers may also encourage adherence to healthy sleep practices, thereby disrupting the cycle between chronic pain, psychological distress, and sleep disorder [6]. Overall, these findings underscore the relevance of a biopsychosocial approach, suggesting that integrating pain and psychological assessment, strengthening social support, and providing individualized sleep education may help maintain healthy sleep patterns [5]. Husak et al. emphasized the association between longer pain duration and worsening sleep disorders [34], a finding corroborated by our study. Prolonged pain may disrupt sleep through persistent nociceptive stimulation, dysregulation of the hypothalamic-pituitary-adrenal axis, and increased psychological distress, all contributing to hyperarousal and impaired sleep continuity [5]. Accordingly, for patients presenting with prolonged pain at admission, closer sleep monitoring and proactive management to prevent further chronicity of pain may be important for improving sleep quality [34]. Regarding dietary factors, our study revealed that frequent tea or coffee consumption was associated with worsening sleep disorders, even after accounting for somatic and psychological factors like pain, anxiety, and depression. Chen et al. also reported an association between frequent coffee consumption and poor sleep quality among healthcare workers experiencing musculoskeletal pain [37]. Caffeine, as a central nervous system stimulant, interferes with adenosine signaling, delays sleep onset, and reduces total sleep time [37]. Therefore, patient education aimed at limiting caffeine intake, particularly in the afternoon and evening, may be an important consideration for supporting improved sleep hygiene [37]. In addition, our study found that smartphone use was associated with worsening sleep disorders among older adults, a pattern more commonly reported in adolescents [38]. This relationship may be bidirectional: smartphone use may disrupt circadian rhythms through blue-light exposure and induce psychological overstimulation, while sleep difficulties may in turn increase nighttime smartphone use [39]. Therefore, providing guidance on appropriate smartphone use, such as reducing screen time before bedtime and establishing device-free routines in the evening, may help improve sleep quality [40]. To enhance the understanding of the complex relationships between factors and sleep disorder trajectories, we developed XML models using routinely assessed or easily obtainable clinical variables. The optimal RF model achieved an AUC of 0.87, surpassing the performance reported by Forte et al. (0.74). and Öz et al. (0.70) in studies of older adults and patients with chronic obstructive pulmonary disease, respectively [7, 16]. This superior performance may be related to the ensemble nature of the RF model, which can capture complex, non-linear relationships while reducing overfitting [17]. Importantly, our emphasis lies in the interpretability of the model. The XML framework, combined with SHAP analyses, may help healthcare professionals better understand how individual variables contribute to different sleep trajectories. SHAP summary, dependence, and Sankey plots may assist in identifying the overall contribution of each variable to the worsening or alleviation of sleep disorder trends, thereby providing potential guidance for developing comprehensive prevention strategies [31]. Building on this, SHAP waterfall plots at the individual level may support more refined risk stratification, enabling healthcare professionals to identify potential sleep trajectories and the most relevant factors for each patient. Accordingly, these tools may offer additional insights into how factors such as pain, depression, and anxiety are associated with sleep disorders in clinical practice [32]. A key methodological consideration in this study is that predictors and the initial sleep assessment were collected at the same baseline time point, which may limit the predictive lead time. However, the outcome was defined as sleep disorder trajectories derived from repeated follow-up assessments, reflecting longitudinal patterns. Therefore, these multidimensional baseline features may still provide useful indications of individual prognosis and potentially aid in identifying patients likely to follow high-risk or improving trajectories at an early stage, before such patterns fully manifest. Nevertheless, partial overlap between predictors and baseline symptoms may affect the model’s practical predictive utility. Future studies should incorporate earlier or time-lagged predictors to strengthen causal inference and improve prospective prediction of trajectory transitions. This study yields significant implications for clinical practice and research. First, by investigating the high prevalence of sleep disorders in older CP patients and identifying four heterogeneous sleep disorder trajectories along with their predictive factors, our findings may contribute to a better understanding of sleep disorder progression and risk stratification. Attention to key factors such as pain intensity, depression, anxiety, and perceived social support, together with integrated management approaches combining sleep hygiene, pain management, psychological care, and social support, may help inform more comprehensive health strategies [2, 6]. In addition, this study offers potential insights into the application of artificial intelligence (AI) in sleep assessment. Following adequate external validation, these models could be considered as supportive tools for sleep screening, which may help inform the selection of targeted management strategies, such as sleep hygiene education and multimodal pain management, to address the complexity of sleep disorders in this population [8]. Future research should further explore the use of multimodal data to advance AI applications and improve understanding of the dynamic progression of sleep disorders [41]. Limitation This study has several limitations that warrant consideration. First, the relatively short follow-up period may have constrained our ability to capture the long-term dynamics of sleep disorder trajectories. Second, the sampling from a single hospital may limit the generalizability of findings across diverse healthcare settings or populations. Additionally, while the predictive factors used were readily obtainable, variables such as pain level, depression, and anxiety relied primarily on self-reported measures, potentially introducing recall bias. Furthermore, the assessment of sleep disorders was based predominantly on questionnaire-based scales, lacking objective measures such as polysomnography, which may compromise evaluation accuracy. Future research should prioritize incorporating multi-center sampling and extending longitudinal follow-up, while integrating objective sleep monitoring tools with self-reported data to enhance measurement precision. Conclusion This study showed that, in the overall cohort of older adults with CP, the prevalence of sleep disorders ranged from 42.8% to 52.2% across the four assessment time points. Four distinct sleep disorder trajectories were identified: LS, HD, MI, and HP. Tea or coffee consumption, smartphone use, pain duration, pain level, anxiety, depression, and perceived social support emerged as significant predictors of these trajectories, with pain level, depression, anxiety, and perceived social support being the most influential. These findings highlight the potential importance of considering biopsychosocial factors in understanding sleep disorder trajectories and may help inform more refined risk stratification and individualized management strategies in this population. Supplementary Information Below is the link to the electronic supplementary material. Supplementary Material 1 Author contributions X.Z. and J.W. conceived and designed the study; X.Z., Y.H., Y.L., S.C, and A.Z. participated in data collection. X.Z. and W.L. analyze data; X.Z. and Y.H. wrote the manuscript; D.Z., and J.W. revised the manuscript for intellectual content. All authors have read and approved the submitted manuscript. All authors agree to be accountable for all aspects of the work. Data availability The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. Declarations Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Supplementary Materials Supplementary Material 1 Data Availability Statement The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.