Development of machine learning models to predict risk of hospitalisation and 90-day readmission among patients with cardiovascular risk factors using community health survey data Authors: Arinze Nkemdirim Okere, Tianfeng Li, Md Mohaimenul Islam, Askal A Ali, Sarah G Buxbaum, Vakaramoko Diaby Published: 2025 ABSTRACT AbstractObjectivesThis study aimed to develop and validate machine learning (ML) models to predict all-cause hospital admissions and 90-day readmissions using structured, patient-reported survey data.MethodsA cross-sectional survey was conducted between 3 July 2021 and 18 December 2022, among US adults aged ≥18 years with at least one cardiovascular risk factor. Participants were recruited through social media, community pharmacies and outpatient clinics. The final sample included 1318 participants. Primary outcomes were any all-cause hospitalisation and readmission within 90 days. Eight supervised ML models were trained using an 80:20 train–test split and 10-fold cross-validation. Model performance was evaluated using area under the receiver operating characteristic curve (AUROC), precision, recall, F1 score and calibration metrics. SHapley Additive exPlanations (SHAP) values identified key predictors.ResultsAmong 1318 participants, 35.0% reported at least one hospitalisation and 10.4% reported a 90-day readmission. The Extra Trees (ET) model demonstrated the best performance across both outcomes. For hospitalisation, ET achieved an AUROC of 0.93, precision of 0.83 and recall of 0.87. For readmission, AUROC was 0.99 with precision of 0.95 and recall of 0.96. SHAP analysis identified heart disease, medication burden, race/ethnicity, employment and insurance status as the most influential predictors.DiscussionPatient-reported data reflecting behavioural, social and clinical factors can predict hospitalisations with high accuracy, complementing traditional EHR-based models.ConclusionsIntegrating such patient-reported and behavioural data into electronic health records could enable earlier identification of high-risk individuals and support targeted, preventive interventions to improve healthcare outcomes.Keywords: Artificial intelligence, Machine Learning, Health Services Research WHAT IS ALREADY KNOWN ON THIS TOPIC Existing risk prediction tools for hospitalisation predominantly rely on retrospective clinical data, rendering them reactive and focused on identifying risk only after adverse events have occurred. Existing risk prediction tools for hospitalisation predominantly rely on retrospective clinical data, rendering them reactive and focused on identifying risk only after adverse events have occurred. WHAT THIS STUDY ADDS This study demonstrates that patients at risk of hospitalisation can be proactively identified using self-reported outcomes that capture their knowledge, perceptions and attitudes towards medication use. This study demonstrates that patients at risk of hospitalisation can be proactively identified using self-reported outcomes that capture their knowledge, perceptions and attitudes towards medication use. HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY Understanding how patients’ perceived medication knowledge and attitudes shape health outcomes provides insight into healthcare utilisation. Integrating such patient-reported and behavioural data into electronic health records can enable earlier risk detection and guide targeted, patient-centred interventions. Understanding how patients’ perceived medication knowledge and attitudes shape health outcomes provides insight into healthcare utilisation. Integrating such patient-reported and behavioural data into electronic health records can enable earlier risk detection and guide targeted, patient-centred interventions. Introduction Cardiovascular disease is a leading cause of preventable hospitalisations and readmissions in the USA, creating significant clinical and economic burdens.1,3 Many admissions are ambulatory care–sensitive and could be avoided with timely, effective outpatient management.4 5 Reducing these admissions is both a public health priority and an economic necessity. Current prevention strategies primarily target patients with complex health needs by improving access to primary and preventive care.6 However, these approaches are often reactive, identifying risk only during clinical encounters and missing opportunities for earlier intervention. Existing risk prediction tools rely heavily on retrospective clinical data such as diagnostic codes and prior hospitalisations.7,9 Although informative, these models often miss early signals embedded in behavioural, social and patient-reported factors. Patient-reported outcomes—such as perceptions of health, medication use and social context—offer valuable insights but remain underused.10 11 Furthermore, traditional models often fail to capture the complex, nonlinear interactions among these diverse risk domains.12 13 Therefore, the objective of our study was to develop and evaluate machine learning (ML) models to predict all-cause 90-day hospital admission and readmissions from patient-reported clinical or social-shortened behavioural factors. Methods Study design We conducted a cross-sectional survey of US adults between 3 July 2021 and 18 December 2022. This study followed the methodological and ethical standards of the American Association for Public Opinion Research, the Strengthening the Reporting of Observational Studies in Epidemiology guidelines for cross-sectional studies and the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis reporting guideline. Study setting and participants Participants were recruited through community outreach, including social media, local pharmacies and outpatient clinics. After verbal consent, surveys were completed electronically via Qualtrics or on paper during home or clinic visits. Participation was voluntary and anonymous, with race and ethnicity self-reported following standard demographic data collection practices.14 Inclusion and exclusion criteria We included participants who were 18 years of age or older and had at least one established cardiovascular risk factor, such as hypertension, hyperlipidaemia or type 2 diabetes mellitus. Participants were excluded if they had a known diagnosis of cognitive impairment or neurological disorders that could compromise their ability to understand or respond accurately to survey items. Additionally, individuals with incomplete survey responses were excluded from the analysis to maintain data integrity. Survey development and predictor variables During survey development, we selected clinically relevant variables based on two criteria: (1) evidence from the literature and (2) patient-reported data commonly captured in electronic health records (EHRs). This approach ensured alignment with real-world clinical practice. The selected variables were grouped into five conceptual domains: (1) social determinants of health, (2) health conditions, (3) medication burden, (4) medication literacy and (5) medication self-efficacy (see online supplemental eTable 1 for details). Study outcomes The study focused on two primary self-reported outcomes: (1) all-cause hospitalisations within 90 days and (2) hospital readmissions within 90 days following a prior discharge, as reported by the survey respondent. Data preprocessing We implemented a structured preprocessing pipeline to ensure data integrity and readiness for ML model development (see figure 1 for an overview of the analytical workflow, including data preprocessing, model development, performance evaluation and model explanation). The schematic summarises the entire ML pipeline from raw survey data to model interpretability. The pipeline followed a validated framework from prior studies,15 addressing common data challenges such as missingness, inconsistent formatting and variable encoding.16 Flow chart showing the pipeline used to compare machine learning models. SHAP, SHapley Additive exPlanations. Additionally, a comprehensive list of variables, along with their operational definitions and coding structures, is provided in online supplemental #Asupplemental #A (Feature Description). Online supplemental eFigures 1 and 2 present the distribution of both the categorical and continuous variables for prediction. We began by accessing data quality to evaluate internal consistency, detect outliers and examine the extent and distribution of missing values. The proportion of missing data ranged from 0% to 11.4%, which was within acceptable thresholds for multivariable analysis.17 18 To address missingness and reduce potential bias, we applied multivariate imputation by chained equations, an iterative method that estimates missing values using relationships among observed variables.19 20 Categorical variables were transformed using one-hot encoding to preserve their non-ordinal structure and ensure compatibility with ML algorithms. For multicategorical variables (gender, insurance and race), one category was dropped during one-hot encoding to serve as the reference and avoid collinearity. For example, for gender, ‘non-binary’ was dropped, and ‘female’ and ‘male’ were encoded as binary variables. Continuous variables were standardised where appropriate to enhance scale comparability and improve model convergence. This preprocessing strategy ensured the dataset was appropriately structured for training and evaluation of the predictive models. Visual summaries of missing data patterns, imputation diagnostics and distributions of original versus imputed values are presented in online supplemental eFigures 3-5. Train/test split and resampling The dataset was randomly divided into training (80%) and test (20%) partitions. To address class imbalance, the Synthetic Minority Over-sampling Technique (SMOTE) was applied exclusively to the training partition, while the test set was retained in its original class distribution to ensure unbiased model evaluation. After resampling, the training data achieved balanced class representation (online supplemental eTable 2). All random operations—including data splitting, sampling and model initialisation—were performed using fixed random seeds for reproducibility. Due to the imbalance of outcome classes, SMOTE was used to synthetically augment the minority class by interpolating between existing minority samples. Implementation details and post-resampling distributions are summarised in online supplemental eTable 2. Feature selection and model development To identify the most informative and non-redundant predictors, we employed a multimethod feature selection strategy integrating filter, wrapper and embedded techniques (for details Online supplemental #B).21 This integrated approach optimised predictive accuracy while reducing overfitting and multicollinearity (online supplemental eFigure 6). Eight supervised ML classifiers were trained and compared: Logistic Regression (LR), Support Vector Classifier (SVC), Gaussian Naïve Bayes (GNB), Random Forest (RF), Extra Trees (ET), Gradient Boosting (GB), XGBoost and AdaBoost. We developed these models using the predictor variables identified earlier and applied 10-fold cross-validation within the training set to improve generalisability and reduce overfitting. Hyperparameter tuning was conducted using both grid search and randomised search strategies within a 10-fold cross-validation framework applied to the training data. Evaluation on the independent test set followed identical preprocessing procedures as the training data, except that SMOTE was not applied to preserve the original class distribution. A comprehensive summary of the optimised hyperparameter settings and model performance is provided in online supplemental eTable 3. This tuning process identified the most effective parameter configurations, including regularisation strength for LR; kernel type and penalty parameter (C) for the SVC and number of trees, learning rate and maximum tree depth for ensemble models. Detailed configurations and corresponding performance metrics are presented in onlinesupplemental eTable 3 eFigure 7. Model evaluation Model discrimination was evaluated using area under the receiver operating characteristic curve (AUROC), precision, recall, F1 score and false positive rate, each reported with 95% CIs. Calibration performance was additionally assessed using the Brier score and Expected Calibration Error (ECE), with calibration curves plotted across ten bins to evaluate agreement between predicted and observed probabilities. Lower Brier and ECE values indicate stronger calibration and better probabilistic reliability, with ideal models showing near-diagonal calibration curves. Confusion matrices for the final models are shown in online supplemental eFigure 8. Model explainability To enhance model interpretability, we applied SHapley Additive exPlanations (SHAP), a model-agnostic, game-theoretic approach that quantifies the contribution of each input feature to the prediction of both outcomes. Additionally, we constructed SHAP dependence plots to assess how changes in a single feature’s value affected predicted risk while accounting for potential interactions with other variables. Software and Packages Analyses were conducted using Python V.3.12.2, along with standard scientific libraries including scikit-learn, imbalanced-learn, XGBoost, pandas, NumPy, SHAP and matplotlib/seaborn. Results Baseline characteristics of patients We included 1318 adult participants in the final analysis. Of these, 461 (35.0%) reported at least one hospital admission, and 137 (10.4%) reported a readmission within 90 days of discharge. The sample was 54.0% female (n=712) and 38.3% male (n=505). The most represented age groups were 41–50 years (n=272; 20.6%) and 51–60 years (n=270; 20.5%). A total of 322 participants (24.4%) held a bachelor’s degree. Racial distribution included 518 (39.3%) who identified as White and 450 (34.1%) as Black or African American. Most participants were married (n=865; 65.6%) and employed at the time of the survey (n=999; 75.8%) (see online supplemental eTable 4 and eTable 5 for more details). Model performance and calibration for predicting hospital admission and readmission Online supplemental eTable 6 summarises the predictive performance of eight ML models for two key outcomes: all-cause hospital admission and 90-day readmission. Prediction of all-cause hospital admission Among the models evaluated, the ET model demonstrated the highest and most balanced performance in predicting hospital admissions. During cross-validation, ET achieved an area under the curve (AUC) of 0.93 (95% CI 0.92 to 0.94), precision of 0.83 (95% CI 0.81 to 0.86), recall of 0.87 (95% CI 0.84 to 0.90) and F1 score of 0.85 (95% CI 0.83 to 0.87). The model demonstrated strong calibration, with a Brier score of 0.11 (95% CI 0.10 to 0.12) and an ECE of 0.07 (95% CI 0.05 to 0.09). On the independent test set, ET maintained robust generalisability with an AUC of 0.85 (95% CI 0.81 to 0.90), precision of 0.79 (95% CI 0.71 to 0.87), recall of 0.73 (95% CI 0.65 to 0.81) and F1 score of 0.76 (95% CI 0.70 to 0.82). Test calibration metrics were similarly strong (Brier=0.15, 95% CI 0.13 to 0.18; ECE=0.06, 95% CI 0.04 to 0.11). Calibration curves demonstrated close alignment between predicted probabilities and observed outcomes, indicating excellent reliability and minimal overconfidence. In comparison, RF and XGBoost models exhibited comparable discrimination with test AUCs of 0.85 (95% CI 0.81 to 0.90) and 0.84 (95% CI 0.79 to 0.89), respectively, but slightly higher Brier scores (RF=0.16; XGBoost=0.16) and ECE values (RF=0.07; XGBoost=0.08). GB model achieved strong calibration (ECE=0.04) but a marginally lower test AUC of 0.85 (95% CI 0.81 to 0.90). SVC achieved a slightly higher AUC (0.86 (95% CI 0.82 to 0.91)) but demonstrated moderate calibration (Brier=0.15; ECE=0.06). GNB showed the weakest calibration (ECE=0.16) and higher Brier score (0.20), indicating systematic overconfidence. Prediction of 90-day readmission The ET model exhibited the most favourable performance in predicting 90-day hospital readmissions. During cross-validation, ET achieved an AUC of 0.99 (95% CI 0.99 to 1.00), precision of 0.95 (95% CI 0.93 to 0.96), recall of 0.96 (95% CI 0.94 to 0.97) and an F1 score of 0.95 (95% CI 0.94 to 0.96). The model also demonstrated excellent calibration, with a Brier score of 0.04 (95% CI 0.04 to 0.05) and ECE of 0.05 (95% CI 0.05 to 0.06), underscoring its accuracy and stability across folds. On the independent test set, ET maintained robust generalisability with an AUC of 0.86 (95% CI 0.79 to 0.91), precision of 0.57 (95% CI 0.41 to 0.74), recall of 0.43 (95% CI 0.28 to 0.57) and F1 score of 0.49 (95% CI 0.35 to 0.62). The Brier score (0.10; 95% CI 0.08 to 0.12) and ECE (0.07; 95% CI 0.04 to 0.11) indicated stable calibration without evidence of overfitting. Calibration curves further confirmed well-aligned predicted probabilities and observed readmission risks across deciles. RF and XGBoost models followed closely, each achieving comparable cross-validation AUCs (0.99; 95% CI 0.99 to 1.00) with slightly higher Brier scores (95% CI 0.04 to 0.05) and ECE values (95% CI 0.06 to 0.07). Their test AUCs (RF=0.85; XGBoost=0.84) and calibration metrics were marginally lower than ET but still indicated strong discrimination and specificity. GB model demonstrated slightly reduced yet acceptable performance (AUC=0.85; 95% CI 0.78 to 0.91; F1=0.62; Brier=0.11; ECE=0.07), maintaining overall reliability with low overfitting risk (onlinesupplemental eTable 6 eFigures 8 ). Receiver operating characteristic analysis To assess the discriminative power of the classifiers, an ROC analysis using the micro-average AUC approach was conducted to evaluate model performance in predicting (A) hospital admission and (B) 90-day readmission. Figure 2 presents the AUROC curves for all eight ML models across both training and testing datasets. AUROC curves for eight machine learning models used to predict a) hospital admission, and b) 90-day readmission. ET, Extra Trees; GB, Gradient Boosting; LR, Logistic Regression; RF, Random Forest; ROC, receiver operating characteristic curve; SVC, Support Vector Classifier. For hospital admission (figure 2A, training phase), the ET model achieved the highest AUROC of 0.929, followed closely by the RF model (AUROC=0.922) and XGBoost (AUROC=0.911). In contrast, GNB exhibited the lowest discriminative ability (AUROC=0.806) for predicting hospital admission. During the testing phase, ET and RF maintained strong generalisability with AUROCs of 0.854 and 0.853, respectively, though SVC achieved a marginally higher AUROC (0.863). For 90-day hospital readmission (figure 2B, training phase), model performance was uniformly higher across classifiers. The ET, RF and XGBoost models achieved AUROCs of 0.993, 0.993 and 0.989, respectively. Other models—including GB and SVC—also performed well (AUROCs=0.983 and 0.965, respectively). On the independent test set, both ET and RF exhibited the strongest generalisability, with AUROCs of 0.856 and 0.851, respectively, indicating excellent discrimination and calibration stability. Model interpretability SHAP feature ranking and summary plots analysis Figure 3 presents SHAP-based feature importance and summary plots for the ML models predicting hospital admission. Heart disease was the strongest predictor, followed by home medication burden, female sex and blood pressure. Other influential factors included race (White), chronic conditions (eg, diabetes) and medication self-efficacy. Socioeconomic indicators such as education, job status and access to healthcare (eg, Medicare/Medicaid status, primary care access) also contributed meaningfully to predictions. Positive SHAP values for heart disease, medication burden and blood pressure indicated higher hospitalisation risk, while private insurance and medication confidence showed protective effects. Interpretability of the SHAP-based model for hospital admission prediction among patients with cardiovascular disease, showing (A) the mean SHAP feature importance across all predictors and (B) a SHAP summary plot demonstrating the direction and magnitude of individual feature contributions to the model’s prediction of hospital admission risk. SHAP, SHapley Additive exPlanations. Figure 4 illustrates SHAP-based interpretations for 90-day readmission predictions. Similar to admission models, heart disease, home medication burden, race (White) and job status were leading predictors. Additional contributors included private insurance, sex and medication self-efficacy. Social determinants of health (eg, insurance type, employment, education) and clinical indicators (eg, blood pressure, cholesterol) were strongly associated with readmission risk. High SHAP values reflected elevated readmission probability, whereas private insurance and greater medication confidence were linked to reduced risk. Interpretability of the SHAP-based model for hospital readmission prediction among patients with cardiovascular disease, showing (A) the mean SHAP feature importance across all predictors and (B) a SHAP summary plot demonstrating the direction and magnitude of individual feature contributions to the model’s prediction of hospital readmission risk. SHAP, SHapley Additive exPlanations. Model interpretability and feature dependence SHAP double and single feature dependence analyses (see online supplemental eFigures 10–12) revealed that hospitalisation and 90-day readmission risks were shaped by the combined and independent effects of clinical, behavioural and social factors. Higher medication literacy and self-efficacy consistently exhibited negative SHAP values, indicating reduced risk, especially when paired with better understanding of medication use, side effects and interactions. Conversely, elevated HbA1c, heart disease and greater medication burden showed positive SHAP contributions, signifying higher risk. Discussion This study is the first to develop and validate ML models using only patient-reported data to predict all-cause hospital admissions and 90-day readmissions. Among eight tested algorithms, the ET model showed the highest predictive accuracy. Findings highlight the critical role of social determinants of health (SDOH), which proved as predictive—if not more so—than clinical factors. Comparison with prior studies Earlier studies on healthcare utilisation mostly relied on clinical data from EHRs, such as vital signs and lab results.22,24 Although helpful, those models often missed the impact of social and behavioural factors. Our study adds to this literature by showing that patient-reported data, including education, job status and insurance type, improve prediction. As these factors have been shown to correlate with delays in care, worse outcomes or unnecessary higher healthcare utilisation.25 26 In addition, our findings showed that variables such as medication confidence and insurance coverage were among the strongest predictors of both hospital admission and readmission. This is consistent with prior studies demonstrating that low medication literacy and poor confidence in managing medications are associated with reduced adherence, increased adverse events and a higher risk of hospitalisation.27,29 Clinical and policy implications Our findings emphasise the importance of integrating SDOH into routine care. Embedding standardised social risk screening, enhancing health education and adopting equity-focused payment models can improve outcomes and reduce disparities. The developed ML models effectively identify patients at high risk of hospitalisation using readily available data, enabling targeted interventions such as early follow-up, medication review and care coordination. Incorporating these predictive tools into EHRs allows real-time risk stratification, supports timely and personalised care, and strengthens clinical decision-making to reduce preventable hospitalisations and advance health equity. Strengths and limitations A major strength of this study is the inclusion of both clinical and non-clinical factors—such as medication literacy, burden and self-efficacy—which improved model performance and interpretability. The Medication Self-Efficacy, Burden and Literacy Instrument (also known as Okere–Renier Survey) has been previously psychometrically validated and demonstrated strong reliability and construct validity.30 In addition, the inclusion of participants from multiple US regions strengthens the relevance of the findings to community-based populations with cardiovascular risk factors. This study has several limitations. All data were self-reported, which may introduce recall bias and limit accuracy. The cross-sectional design prevents causal conclusions, and we could not distinguish between planned and unplanned hospitalisations. Continuous variables were categorised, possibly masking subtle differences. Medication adherence was not directly measured, and participants were recruited through convenience sampling, limiting generalisability. Lastly, while the models were internally validated, external and prospective validation are needed to confirm their accuracy and real-world use. Despite these limitations, the SHAP analyses revealed that hospitalisation and readmission risks are jointly driven by medication behaviours, disease control and social context. Higher medication literacy and self-efficacy were linked to lower risk, whereas poor glycaemic control, multimorbidity and high medication burden increased it. These findings suggest that behavioural and educational factors are as influential as clinical ones. Integrating the multidimensional predictive score from our ML model—derived from self-reported behavioural, clinical and social data—into EHRs could enable early identification of high-risk patients. Embedding such tools into clinical workflows may enhance pharmacist-led interventions and improve chronic disease outcomes. Conclusions The ET model outperformed other algorithms in predicting hospitalisations using patient self-reported data on perceived knowledge, attitudes and behaviours. Such patient-reported behavioural, social and clinical factors can complement EHR-based models to enable early, targeted interventions for high-risk individuals. Supplementary material Footnotes Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors. Provenance and peer review: Not commissioned; externally peer reviewed. Patient consent for publication: Not applicable. Data availability free text: The datasets used and analysed during the current study are available from the corresponding author on reasonable request. The underlying code for this study and training/validation datasets is not publicly available but may be made available to qualified researchers on reasonable request from the corresponding author. Ethics approval: Ethical approval was obtained from Florida A&M University Institutional Review Board (Project Title: 1370827-4, Reference #: 001-19). Data availability statement Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as supplementary information. References Associated Data Supplementary Materials Data Availability Statement Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as supplementary information.