Development and evaluation of a machine learning based risk prediction model for osteoporosis

Lin Zhou, Qi Zhang, Haijing Wei, Jie Liu, Yongchang Yang

Abstract

AbstractObjectiveOsteoporosis is a systemic bone disease that can lead to decreased bone strength and increased susceptibility to fractures. This study aims to evaluate the level of cytokines and other common blood biomarkers, and establish a machine learning-based risk prediction model for osteoporosis patients.MethodsThe clinical data of patients with osteoporosis who were hospitalized in the Seventh Medical Center of the Chinese People’s Liberation Army General Hospital from January 2023 to September 2024 were retrospectively collected. 16 cytokines and other potential predictive variables were analyzed to determine the independent risk factors. Eleven machine learning models were established, and the performance of the models was evaluated using the ROC curve, the PR curve and the confusion matrix, and the optimal model was selected. SHAP to provide insights into the model’s predictions and construct a visual representation of the prediction model.ResultsA total of 187 patients were included, and 16 characteristics such as IL-8 were identified as risk characteristics. By comparing 11 different models, it was found that the Gradient Boost was the best model, with an AUC value of 0.91, an accuracy rate of 0.81, an F1 score of 0.80, and a relatively balanced specificity and sensitivity. And the SHAP value was used to reveal the direction and strength of each feature in predicting osteoporosis.ConclusionThis study successfully developed a risk prediction model based on 16 common variables, including IL-8, providing critical evidence for the early identification and prevention of osteoporosis patients, with potential clinical significance.Keywords: biomarkers, cytokine, machine learning, osteoporosis, prediction model

Introduction

Osteoporosis is a systemic bone disease characterized by a declined in bone mineral density (BMD) and deteriorated of bone microarchitecture, which lead to decreased bone strength and increased susceptibility to fractures (). As a major global public health issue, its impact on people aged 50 and above is particularly severe, with a lifetime risk of fragility fractures of 50% for women and 20% for men (; ). With the continuous acceleration of global population aging, osteoporosis has become an increasingly serious threat to public health, and its related socio-economic burden is expected to increase significantly in the coming decades (). At present, the majority of osteoporosis patients have not received standardized diagnostic assessments and evidence-based treatment interventions, which in turn has led to a sustained increase in global fragility fracture rates.

In clinical practice, dual energy X-ray absorptiometry (DXA) is the gold standard for evaluate BMD and confirming the diagnosis of osteoporosis (; ). Although DXA technology has reduced radiation exposure to safe levels, the persistent anxiety of patients towards radiation fear cannot be ignored. In addition, the reliance on professional equipment poses a significant obstacle to its widespread adoption, especially in resource limited primary healthcare settings (; ). In contrast, detection technology based on blood biomarkers has superior clinical applicability and significant advantages over traditional methods, especially in improving patient compliance and enhancing medical accessibility.

The hematological indicators of osteoporosis mainly evaluate bone metabolism, endocrine function, and other potential pathogenic factors. Furthermore, deviations in routine hematological parameters such as whole blood cell count and liver and kidney function tests may be related to the pathogenesis or progression of osteoporosis (; ). Recently, a few researches shown that cytokines were related to the immune inflammatory responses in vivo through complex regulatory networks. In the pathological mechanism of osteoporosis, cytokines play a pivotal role in the dynamic balance between osteoclasts and osteoblasts (; ).

At present, various osteoporosis risk assessment tools have been developed worldwide. among which, FRAX ® The fracture risk assessment tool is a typical example (; ). This tool integrates clinical factors and BMD data to accurately estimate the probability of an individual experiencing severe osteoporotic fractures and hip fractures within 10 years (; ). However, the tool still relies on BMD data and has not yet incorporated the predictive value of blood biomarkers such as osteocalcin.

Our study conducted a comprehensive comparative analysis of multidimensional hematological features between patients with osteoporosis and matched healthy controls, with a focus on exploring the relationship between BMD measurements and hematological indicators. By integrating various hematological parameters across cellular, metabolic, and endocrine dimensions, we use advanced machine learning techniques to identify key biomarker patterns and construct an intelligent diagnostic framework for osteoporosis, providing support for clinical decision. Furthermore, by comprehensively analyzing these biomarkers, our study aims to provide effective preventive strategies for high-risk populations.

Materials and methods

Clinical data

This research enrolled participants who underwent DXA for osteoporosis diagnosis. Osteoporosis participants met the following inclusion criteria: (1) DXA detection bone density T-value ≤ −2.5 at lumbar spine or femoral neck; (2) A history of fragility fractures (; ). The participants with any of the following conditions were excluded from this research: (1) A history of malignant tumors, cerebrovascular accidents, acute myocardial infarction, or other serious acute/chronic diseases; (2) Concurrent diagnosis of inflammatory disease or autoimmune diseases, etc. (3) Incomplete medical record information that precluded adequate study assessment (; ). According to the inclusion criteria, 359 participants (including osteoporosis patients and sex-and age-matched healthy controls during the same period) were recruited from the seventh Medical Center Chinese PLA General Hospital from January 2023 to September 2024. And 187 eligible participants met all standards and were involved in the ultimate analysis. This research was approved by the Ethics Committee of our institution (S2025-076-01).

Based on collecting and summarizing existing literature and clinical experience, we identified the predictor variables for this study. These variables included demographic and anthropometric parameters (age, age at menopause, height, weight, body mass index (BMI)). Additionally, 98 chief laboratory tests indicators, including blood cell counts, C-reactive protein, bilirubin, lipid profile, cytokines, and other relevant biomarkers, were retrospectively collected for all participants.

All serum samples were obtained via centrifugation after blood collection and stored at −80 °C. Serum concentrations of cytokines, including IFNα, IFNγ, IL-10, IL-17F, IL-1, IL-18, IL-22, IL-2R, IL-2, IL-17A, IL-8, IL-4, were quantified using chemiluminescence assay. The reagent kit was purchased from Orienter Bioengineering Co., Ltd (Sichuan, China), and experiments were conductedaccording to the manufacturer’s instructions.

Statistical analysis

Normally distributed data was represented as mean ± standard deviation, and intergroup comparisons were performed by t-tests. Non-normally distributed data was represented as median and analyzed by Kruskal–Wallis H test. Missing values for each variable were count using “VIM” package in R and imputed via the “zoo” package in R. Using the createDataPartition function in the R “caret” package to split data into training and validation cohort in a 7:3 ratio. The Boruta algorithm was applied to the cohort to perform preliminary feature selection and identify key variables.

Logistic regression, AdaBoost, Decision tree, EXtreme Gradient Boosting (XGBoost), Artificial Neural Network (ANN), LightGBM, GradientBoost, K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Random forest and Naïve bayes were used to screen risk factors for osteoporosis, and a novel predictive model was built according to the identified factors.

During the model construction process, L2 regularization is used to constrain the model weights and limit the excessive amplification of feature weights to reduce the risk of overfitting; At the same time, the model’s generalization performance will be objectively evaluated through an independent test set (30% samples) to avoid the problem of performance overestimation caused by relying solely on the training set for evaluation. Accuracy, positive recall, negative recall, positive precision, negative precision, F1 score, Receiver Operating Characteristic (ROC) curve and the area under the ROC curve (AUC) were calculated to evaluate the predictive performance of the model. This research used SPSS 26.0 software (SPSS Inc., United States of America) and R software (version 4.5.1, United States of America).

Results

Firstly, 359 individuals were involved in this study. Among them, 123 were excluded because of incomplete medical record information, and 49 were excluded for meeting the predefined exclusion criteria. Ultimately, 187 participants were finally enrolled, including 125 osteoporosis patients and 62 healthy controls during the same period. The flowchart was shown in . The demographic and clinical characteristics of all participants are summarized in .

Comparison of cytokine levels between the two groups revealed that the levels of IL-8 and IL-18 were significantly decrease in the osteoporosis group (). These results suggest that abnormalities in various cytokines may be contribute to the pathogenesis of osteoporosis.

Notably, the osteoporosis group exhibited significantly lower age of menopause and BMI compared to healthy group (p < 0.001). A total of 96 potential predictive variables were included in the initial analysis. Meanwhile, missing values within each variable across all included participants were quantified, and missing data were imputed with the mean value for numerical vatiables (). According to univariate analysis, 24 of these features reveled significant differences between the two groups (p < 0.05) (), specifically: BGP, CTx, IL-8, PINP, D-DIMER, AFU, CHE, TG, CHO, ALP, UA, INR, PTA, PT, RBC, P-LCR, BASO%, EO#, HGB, PDW, MPV, PCT, BAP, LDL.

illustrates the correlation analysis of these features among participants, with several features showning high correlation (). Only one representative feature was retained from each set of highly correlated variables to avoid multicollinearity. Boruta visually displays the importance of each feature, as shown in . Green boxplots indicate that the corresponding variables are confirmed as important predictors, while red boxplots denote rejected variables and yellow boxplots represent tentative variables. Decision curve analysis (DCA) was applied to analysis the clinical applicability, the 16 features model and 19 features model achieved equivalent net benefits, which were superior to those of 12 features model (). Above all, 16 risk features, including BGP, CTx, IL-8, PINP, D-DIMER, AFU, TG, CHO, ALP, NSBUA, INR, RBC, P-LCR, PCT and BAP, were ultimately selected to construct predictive model. The expression differences of the 16 risk characteristics mentioned above are shown in .

After data preprocessing, all data was randomly divided into training and validation cohort in a 7:3 ratio. In this study, eleven distinct machine learning algorithms were employed to establish the predictive model, including Logistic regression, AdaBoost, Decision tree, XGBoost, ANN, LightGBM, GradientBoost, KNN, SVM, Random forest and Naïve bayes. The performance of these models, which evaluated by accuracy, precision, recall, and F1 score, was reported for both the training and validation cohort in . The accuracy and AUC of multiple models in the training set of this study reached 1.0, indicating the presence of overfitting. The main reason is that the total sample size of this study was only 187 cases, and the modeling included 16 predictive features, resulting in a relatively high proportion of features compared to the sample size. However, the significant decrease in various indicators of the test set proves that the model still has stable predictive ability in independent samples and has clinical reference value. And the performance of each model is compared in multiple dimensions via the radar map to further improve visual interpretability ().

Based on comprehensive evaluation of ROC curves and PR curves, six models demonstrated high predictive performance: Gradient boost, Random forest, LightGBM, XGBoost, Logistic regression, KNN ().

Moreover, the Confusion Matrix for the six models are presented in . As a fundamental visualization tool for demonstrating model performance in machine learning, confusion matrices explicitly illustrate the classification results of each model, specifically the number of true positives, true negatives, false positives and false negatives, which providing a direct reference for evaluating the models.

For the SHapley Additive exPlanations (SHAP) value (), each line shows the SHAP values for the features assigned to all participants. The x-axis represents the SHAP value, which revealed the contribution of a certain feature to the prediction result. Its absolute value reflects the intensity of features on the model’s predictive outcome. Moreover, the SHAP values of five samples were calculated by Gradient Boost model (), which intuitively illustrates how the model generates predictions based on these features and visualizes the interpretive logic of SHAP. A nomogram was also constructed to quantify the relative importance of these features in predicting osteoporosis ().

Discussion

As an escalating threat to public health, osteoporosis relies on DXA for diagnosis which significantly limited the feasibility for population screening. To address this barrier, several studies have explored the risk factors for osteoporosis and developed predictive models for its early identification. For instance, Kunze et al. conducted a machine learning model to predict 10-year risk of menopause-related osteoporosis, incorporating 14 baseline risk factors including clinical characteristics, laboratory results and imaging data (). Park et al. developed a XGBoost -based prediction model, which included variables such as genetic factor, breastfeeding history, age, residential area, serum albumin and so on (). FRAX, recommended by WHO, is currently the mainstream fracture risk assessment tool worldwide. It only relies on simple clinical information such as age, body mass, past fragility fractures, and hormone use to calculate the 10-year fracture probability. Despite the emergence of novel and well-performing methods for bone health assessment, disparities in study quality remain prevalent. On the other hand, many models require an excessive number of input features, such as imaging result, which are not readily accessible in routine clinical practice (; ; ). These gaps highlight the urgent need in the development, validation, and reporting of AI driven osteoporosis prediction models. Compared with these, the machine learning model constructed in this study integrates 16 blood testing indicators such as IL-8, PINP, CTx, etc., without relying on DXA equipment, and is more suitable for primary screening of large-scale populations at the grassroots level.

Recently, various studies have shown that cytokines participate in precise regulation of immune inflammatory responses in the body through complex signaling networks. In the pathological progression of osteoporosis, some cytokines, especially those involved in the dynamic balance between osteoclasts and osteoblasts, play a key role in bone remodeling (; ). These cytokines mainly include members of the tumor necrosis factor family, interleukin family, and growth factors, which together regulate the differentiation, activation, and function of osteoclasts and osteoblasts. Of particular importance, the RANKL/RANK/OPG signaling axis serves as the core regulatory hub of the cytokine network and signaling pathway, playing an irreplaceable role in maintaining bone metabolism balance (; ). To further investigate the role of cytokines in osteoporosis, we selected 80 representative samples (40 osteoporosis patients and 40 healthy controls) for comparative analysis of cytokine levels. Consistent with the above mechanism insights, our results showed significant differences (p < 0.05) in IL-8 and IL-18 levels between the two groups. These findings confirm that IL-8 and IL-18 are closely related to the pathogenesis of osteoporosis and support their practicality as potential risk factors for predicting osteoporosis. The production and secretion of interleukin-8 (IL-8) are tightly regulated by MAPK signaling pathway. Its primary biological function is to mediates the chemotaxis and activate of neutrophils and recruits T cells and eosinophils to the site of inflammation, thereby amplifying the local inflammatory response (; ; ). In clinical practice, as an inflammatory factor, IL-8 can directly act on monocytes/osteoclast precursor cells; On the other hand, IL-8 participates in stimulating osteoblasts to further secrete RANKL, reduce OPG, increase the RANKL/OPG ratio, thereby promoting osteoclast differentiation, enhancing bone resorption, disrupting the dynamic balance of osteoblasts and osteoclasts, and exacerbating bone loss. IL-18 acts as a downstream effector of the NLRP3 inflammasome caspase 1 pyroptosis pathway. Activated NLRP3 inflammasome increases IL-18 levels, drives inflammatory bone damage, and participates in the pathological process of osteoporosis.

In addition to cytokines, characteristics and laboratory parameters were also incorporated to identify key factors distinguishing osteoporosis patients from healthy controls. Initially, we employed various analytical approaches and validation strategies to develop and evaluate a prediction model for osteoporosis. This model was ultimately built using 16 variables, including BGP, CTx, IL-8, PINP, D-DIMER, AFU, TG, CHO, ALP, NSBUA, INR, RBC, P-LCR, PCT, BAP. Based on these identified key factors, we employed 11 machine learning algorithms to build predictive models and evaluated their performance in predicting the risk of osteoporosis. Among these models, Gradient Boost showed the best performance with an AUC value of 0.91, accuracy of 0.81, F1 score of 0.80, and a relatively balanced specificity and sensitivity. Subsequently, the SHAP value indicated the magnitude of which features influence the prediction. A larger positive value indicates the feature increase the probability of predicting osteoporosis, while a large negative value indicates the feature substantially reduce this prediction probability. As illustrated in , the SHAP value visualization for five typical samples intuitively demonstrates the direction and strength of each feature contribution to the osteoporosis prediction for individual samples. For each sample, the model first sets a base value. Then, by summing the positive and negative contributions of all features, the final prediction value is obtained. Compare the final value with the base value to indicate whether the participate was a high risk of osteoporosis or low. We also developed a nomogram that illustrates the importance of each independent risk factor in the predictive model.

Consistent with the finding of previous research, biochemical markers of bone metabolism are widely used in clinical practice for diagnosis, monitoring, and risk stratification of osteoporosis. Among them, BGP is a unique protein synthesized by odentinocytes and osteoblasts (; ). As a major non-collagenous protein secreted by these cells, BGP is currently recognized as a biomarker of the bone formation rate. Notably, this study confirmed that BGP could predict osteoporosis fracture risk, rendering it high valuable for the early prevention of fractures (; ). In addition, ALP and PINP are also considered as well established biomarkers of bone formation: ALP plays an important role in bone mineralization, while PINP, a byproduct of type I collagen synthesis, directly mirrors the rate of new bone formation (; ). In contrast, CTx is a key biomarker of bone resorption as it is released during the degradation of mature type I collagen by osteoclasts, specifically indicating osteoclast activity and bone resorption rate (; ). Consistent with these known functions, our study identified BGP, PINP, ALP, and CTx as important predictive variables, further verifying their clinical practicality in the assessment of osteoporosis.

In addition to bone metabolism biomarkers, previous research reports have shown a positive correlation between bone mass stage and PT, INR, and D-dimer in adults over 50 years old in China (). It is worth noting that D-dimer is significantly negatively correlated with BMD in the Ward’s triangle, which refers to the radiographic area located at the intersection of primary compression, secondary compression, and primary tension trabeculae in the femoral neck (; ). The underlying link between coagulation abnormalities and reduced BMD may be attributed to impaired bone microarchitecture: for instance, in patients with diabetes and vascular complications, compromised nutrient delivery to bone tissue and disrupted bone remodeling processes have been shown to accelerate the deterioration of trabecular structure, that could also contribute to the observed correlation between D-dimer and Ward’s triangle BMD in the elderly population. Consistent with these findings, our study also identified coagulation-related indicators as a key predictive variable for osteoporosis, further supporting the potential crosstalk between the coagulation system and bone metabolism.

Dyslipidemia are key driving factors for oxidative stress and inflammation. It has a direct adverse effect on bone health by shifting the balance of bone marrow stromal cells towards adipogenic differentiation and away from osteogenic differentiation. Bone marrow obesity can also secrete pro-inflammatory adipokines, further inhibiting bone formation, promoting osteoclast mediated bone resorption, and ultimately weakening the overall bone structure (; ). The harmful effects of UA on bone health are mainly attributed to two key processes: first, intracellular oxidative stress generated during UA degradation, and second, UA induced accumulation of pro-inflammatory factors. These processes impair bone homeostasis by inhibiting osteoblast mediated bone formation while promoting osteoclast driven bone resorption (; ). In addition, existing studies have shown that UA can inhibit the expression of 1α-hydroxylase in renal proximal tubular cells. Due to the fact that 1α - hydroxylase is a key enzyme in activating vitamin D, its reduction further disrupts the normal bone remodeling process, ultimately increasing the risk of osteoporotic fractures (; ).

According to reports, inflammatory markers from whole blood, such as neutrophil lymphocyte monocyte count and PDW, PCT, ESR, and CRP levels, are negatively correlated with BMD values (; ). Specifically, as BMD values and t-scores decrease, the level of neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio, monocyte-to-lymphocyte ratio are consistently elevated. This negative correlation is consistent with the recognized link between inflammation and impaired bone homeostasis. The elevation of neutrophils and monocytes reflects a pro-inflammatory systemic state, which can promote the differentiation and activity of osteoclasts. In contrast, lymphocytes play an anti-inflammatory role and support bone formation. Therefore, the relative decrease in lymphocytes further exacerbates the inflammatory imbalance, accelerates bone loss, and reduces BMD (; ).

Although the potential mechanisms linking these factors to osteoporosis have not been fully elucidated, the strong clinical relevance provides potential ideas for the development of predictive models, enhancing the clinical value of the conclusions. It is important to emphasize that the predictive model we have developed is designed to provide an auxiliary tool for clinical decision-making. This research still has some limitations that need to be acknowledged. Firstly, this study utilized retrospective data without conducting subgroup analysis stratified by age and menopausal status. In addition, the lack of smoking, exercise, and calcium intake makes it difficult for us to evaluate the predictive ability of the model in different subgroups and limits the representativeness of the study population, which may introduce potential biases. Therefore, there was no synchronous calculation of FRAX risk values in this queue to complete the horizontal efficacy comparison. Secondly, this study was conducted in a single institution with a limited sample size, which resulted in a certain degree of overfitting in the training set model. In the future, multi center large-scale research is needed to verify the reliability of model performance. Thirdly, although we have identified significant associations between coagulation function, lipids, inflammation, and osteoporosis, the underlying molecular and cellular mechanisms driving these associations are still not fully elucidated. Therefore, it is needed to develop further studies to analyze the causal relationships and mechanistic connections between these factors and the pathogenesis of osteoporosis.

Conclusion

In this study, a machine learning based osteoporosis prediction model was successfully developed, which demonstrated robust performance in clinical applications. By integrating 16 accessible predictive factors, the model avoids the reliance on initial risk assessment of DXA, which helps to implement osteoporosis screening more widely and improves the universality of early risk identification for high-risk populations. Over the long term, this enhanced screening efficiency is expected to provide timely intervention for high-risk populations of osteoporosis, ultimately helping to reduce the incidence of osteoporotic fractures and improve the skeletal health of the population.

Funding Statement

The author(s) declared that financial support was received for this work and/or its publication. This work was supported by the Wu Jieping Medical Foundation (320.6750.18173).

Footnotes

Ethics statement

The studies involving humans were approved by the ethics committee of Chinese PLA General Hospital. The studies were conducted in accordance with the local legislation and institutional requirements. The human samples used in this study were acquired from a by-product of routine care or industry. Written informed consent for participation was not required from the participants or the participant’s legal guardians/next of kin in accordance with the national legislation and institutional requirements.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.