Predicting early and late neonatal mortality using machine learning models in Oman Authors: Sulaiman Salim Al Mashrafi, Laleh Tafakori, Mali Abdollahian Published: 2025 ABSTRACT BackgroundNeonatal mortality is a major issue in global health and is included in the Sustainable Development Goals (SDGs). Early neonatal deaths account for 47% of under-five mortality. Developing a dependable model to predict early neonatal mortality and recognise its related risk factors is essential for child survival and enhancing children’s health outcomes. We utilised various machine learning models to predict early and late neonatal mortality using a comprehensive secondary dataset from Oman.MethodsTen different machine learning (ML) models were used to predict early and late neonatal mortality in three distinct setups: using the original local dataset, applying the data-driven approach represented by Synthetic Minority Over-Sampling Technique (SMOTE) to address the imbalanced distribution, and implementing an algorithm-driven approach via cost-sensitive classification. A total of 2,940 de-identified local records on newborn deaths were categorised into early deaths (0–6 days) and late deaths (7–27 days) for model training and testing using a 10-fold cross-validation. Various calibration and discrimination metrics were utilised to assess the models’ performance due to the issue of an imbalanced dataset.ResultsThe analysis revealed that 71.6% of the deaths occurred during the early neonatal period (0–6 days). Logistic Regression (LR), Linear Discriminant Analysis (LDA), and Random Forest (RF) were the top performers across the three scenarios, with AUC-PR (Area Under Precision and Recall Curves) above 0.85 and an exemplary Brier score. However, RF Brier score was more stable across the three setups, especially with SMOTE (Brier = 0.1864), compared to the Brier score of LDA (0.2211) and LR (0.2164) indicating an effective calibration. The APGAR (Appearance, Pulse, Grimace, Activity, and Respiration) score at 5 min was identified as the most significant predictor of early and late neonatal mortality.ConclusionThis study is one of the first to train and evaluate multiple ML algorithms under three different scenarios to predict early and late neonatal mortality and to identify associated risk factors using real data from Oman. The results indicate that RF, LDA and LR performed the best based on their discrimination and calibration performance. The findings have the potential to inform clinical decision-making and prompt timely interventions to enhance survival rates.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12889-025-25796-1. Introduction The first month of the child’s life is critical for the newborn’s survival. The neonatal period is defined as the first 28 days of a newborn’s life and is the most vulnerable time for a child’s survival [1]. A significant number of newborns do not survive beyond this vulnerable period. Approximately, 2.3 million neonatal deaths were reported worldwide in 2022 [2]. Therefore, neonatal mortality is a critical concern in public health and for decision-makers [3]. Target 3.2 in Goal 3 of the Sustainable Development Goals (SDGs) states that, “By 2030, end preventable deaths of newborns and children under five years of age, with all countries aiming to reduce neonatal mortality to at least as low as 12 per 1,000 live births and under-5 mortality to at least as low as 25 per 1,000 live births” [4]. Understanding the predictors and risk factors associated with neonatal mortality is crucial. This empowers healthcare professionals and decision-makers to ensure a healthier future for all babies through targeted interventions and strategies. Notably, demographic factors such as maternal age, gravidity, and gender are significant contributors to neonatal deaths [5, 6]. For instance, a study including a sample of 43,547 pregnant women conducted in Nepal demonstrated that the odds of neonatal mortality among mothers aged less than 20 years are higher compared to mothers aged 20 years or above (Adjusted Odds Ratio, AOR = 1.76, CI 95% 1.17–2.59) [7]. A systematic review has reported various health and clinical indicators as the top significant contributors to neonatal mortality, encompassing birth weight, APGAR score, gestational age, temperature, maternal blood pressure, and heart rate [5]. Furthermore, maternal body mass index (BMI) has been found to be significantly associated with neonatal mortality [3, 8]. Other commonly and frequently reported causes of early neonatal deaths include birth asphyxia, respiratory distress syndrome, grand multigravida mothers, and inability to exclusively breastfeed [9]. Several studies have employed ML algorithms to predict neonatal mortality. In a systematic review of literature, Mangold et al. [5] included 11 studies that predicted neonatal mortality using ML models. The authors found that logistic regression, random forest, and neural network were the most frequently utilised ML algorithms, with features ranging from 3 to 66 in all studies. The most featured variables involved in the studies include baby weight, gestational age, APGAR score, and multiple births. The average applied models was 4 per study, with a minimum of one model and a maximum of 52 models per study. Most studies (73%) reported the area under the curve (AUC) as a metric for evaluating model performance, with a mean AUC ranging from 58.3% to 97%. The authors argued that ML algorithms can accurately predict the risk of neonatal mortality. Predictive models of neonatal mortality using machine learning are notably scarce in the Arab world. To date, no studies have been found from Arab countries that predict neonatal mortality using machine learning algorithms. A study from Yemen applied the Artificial Neural Network (ANN) to forecast the Infant Mortality Rate (IMR) [10]. Infant mortality refers to the death of children in their first year of life. The study utilised yearly data on IMR for Yemen from 1960 to 2020 and projected the IMR from 2021 to 2030. The authors stated that the model is stable, and the forecasted IMR will likely remain around 43 per 100,000 live births. Similarly, a study forecasted Oman’s IMR utilising the ANNs to be 9 per 100,000 live births from 2021 to 2030 [11]. This section has identified the key predictors and causes of neonatal mortality. While several studies have applied ML algorithms to predict neonatal mortality, only one study specifically focused on early neonatal mortality among low-birth-weight preterm newborns. Moreover, to the best of our knowledge, no studies have addressed this topic in the Arab region, including Oman. These gaps highlight the need for further research in this area, particularly within Oman and other Arab countries. This study contributes by serving as a foundation for future research on neonatal mortality prediction in general, and early neonatal mortality in particular. Significance of the study This study is one of the first to classify neonatal mortality into early and late in Oman and Arab region. Utilising a large representative sample size (n = 2,940) enhances the generalisability of the results. We have employed ten different machine learning models across three distinct setups to comprehensively evaluate their performance. Additionally, this study integrates features of both the newborn and the mother. Early neonatal mortality is a global health concern, accounting for nearly half of under-five mortality. Therefore, addressing this health issue using innovative solutions, including artificial intelligence is vital for child survival. Our work is a stepping stone for future research work in this field. Differentiating early neonatal deaths (0–6 days) and late neonatal deaths (7–27 days) is crucial for early interventions to reduce the burden of early neonatal mortality. The causes and Key Performance Indicators (KPIs) related to these two outcomes can differ significantly. Understanding the time of death is paramount to accessing obstetric care and the availability of a trained health professional and other resources. A systematic review revealed that about 57% of neonatal deaths occurred in the first three days of life, which necessitates scaling up evidence-based interventions [12]. Newborns at risk of early mortality should receive intensive and immediate postnatal monitoring. Machine learning models can explore hidden patterns in large datasets. Therefore, developing dependable predictive models using machine learning can speed up diagnosis and enable timely interventions. Objectives: To identify the most significant KPIs of early and late neonatal mortality.To develop predictive models using ML algorithms to classify neonatal mortality into early and late. To identify the most significant KPIs of early and late neonatal mortality. To identify the most significant KPIs of early and late neonatal mortality. To develop predictive models using ML algorithms to classify neonatal mortality into early and late. To develop predictive models using ML algorithms to classify neonatal mortality into early and late. Methods Data description This project employs a quantitative research design utilising a secondary dataset pertaining to neonatal mortality in Oman. The data were extracted from the health information management system (ALSHIFA) at the national level and span the first nine years of the Sustainable Development Goals (SDGs) from 2016 to 2024. The retrospective cohort study encompasses information related to both the newborns and their mothers, as presented in Table 1 of the supplementary_file_1. The inclusion criterion was defined as the retrieval of all age-related deaths of newborns aged 0 to 27 days, in accordance with the World Health Organisation definition of neonatal mortality. Data preprocessing A total of 2,940 neonatal deaths were retrieved from 2016 to 2024 across the 11 governorates in Oman, encompassing a total of 17 features. The target variable was categorised into two main outcomes: Early neonatal deaths (Occurring from 0 to 6 days) and late neonatal deaths (occurring from 7 to 27 days). Figure 1 in the supplementary_file_1 illustrates the distribution of the sample according to some essential features. We utilised the random forest technique to handle the missing values in our dataset. The proportion of missing values for the variables with missingness is illustrated in Table 2 of the supplementary_file_1. The proportion of missingness is less than 5% for each variable with missing values. This technique is more efficient compared to the traditional mean or median imputation because it takes into account other characteristics of each record. Furthermore, random forest is capable of imputing numerical and categorical missing values. The imputation was employed without the target variable to avoid target leakage. Summary statistics, corresponding tests, and visualisations were computed and reported. All analyses were conducted using R (version 4.4.2), with a two-sided significance level of 0.05. Imbalanced dataset In this study, the outcome exhibits an imbalanced nature with a predominance class of early deaths (n = 2,104, 71.6%) compared to late deaths, which account for (n = 836, 28.4%). The imbalanced dataset is a common challenge in the real-world, including the medical field and fraud detection (e.g [12]).,, where the classes are distributed unevenly [12]. Figure 1 depicts the distribution of the two classes. Fig. 1The distribution of early and late neonatal deaths The distribution of early and late neonatal deaths To address the imbalanced distribution, several techniques and methods have been proposed, including data-driven approaches and algorithm-driven approaches [13, 14]. Consequently, we explored three different scenarios to train our models. The first approach is to utilise the original dataset without any adjustments to handle the imbalance issue. This approach was proposed because we are interested in predicting the majority class (Early deaths). It will serve as a baseline for the models’ performance compared to the other two proposed approaches. The second approach is to employ data-driven methods. In this approach, the training set is modified via different techniques, including under-sampling and over-sampling [15]. The former approach shrinks the majority class, whereas over-sampling increases the minority class [16]. Each approach has its advantages and drawbacks, which are well-documented in the existing literature. Our study will employ an over-sampling method named Synthetic Minority Over-Sampling Technique (SMOTE). It has been reported that in a noisy data environment, SMOTE exhibits higher robustness compared to the under-sampling techniques [17]. Moreover, the mean of a particular class is not affected by SMOTE, but it can reduce the data variability [18]. Lastly, we will employ an algorithm-driven approach. This approach focuses on how the modules behave rather than modifying the dataset’s structure by augmenting the size of the minority class or reducing the number of the majority [19]. Cost-sensitive classification via the (mlr) package in R using ClassifTask was employed to incorporate class-dependent misclassification costs [20]. This method is reported to support both rebalancing and threshold adjustment. Machine learning classifiers For each scenario, we have employed and compared 10 machine learning models to classify neonatal mortality into early and late deaths. The utilised models for classification are “Quadratic Discriminant Analysis (QDA), Linear Discriminant Analysis (LDA), Logistic Regression (LR), K-Nearest Neighbors (KNN), Artificial Neural Network (ANN), Random Forest (RF), Extreme Gradient Boosting (XGBoost), Decision Tree (DT), Support Vector Machine (SVM), and Naive Bayes (NB)”. A brief description of each classifier is included in the supplementary_file_1. The dataset was randomly partitioned into training (80%) and test (20%)0.10-fold cross-validation was employed for each model. We also employed the preprocess function in R for scaling and centering the numeric features. The transformation was applied to the train data alone. This is followed by applying the calculated parameters from the train set to normalise both the train and test sets to ensure that data leakage is avoided while the target variable is treated as categorical. We used caret’s trainControl() to configure resampling, and specified preProcess in train() to ensure each fold was pre-processed. All categorical variables were converted into factors, each with a reference level to ensure consistency across all splits. The positive class is the early class, as our objective is to detect newborns dying at early stages of life. The subsequent section briefly describes the machine learning models employed in this study. Hyperparameter tuning Hyperparameter tuning for each machine learning model was carried out using 10-fold cross-validation to ensure robust model evaluation and AUC-PR across the three scenarios. The caret package in R was employed for this purpose. Models with tunable parameters were optimised via grid search. Models without tunable hyperparameters, including LDA and QDA were directly evaluated using the same cross-validation setup. The final performance comparisons are based on the best-tuned models under each strategy. A threshold of 0.5 was set for binary classification. A comprehensive description of hyperparameter tuning is provided in the supplementary_file_1. Model evaluation In this work, we will utilise several validation metrics to assess model performance, including accuracy, sensitivity (recall), precision, F1-score, the area under the Receiver Operating characteristic Curve (ROC), AUC curve, the AUC-PR (the precision-recall curve), and Brier score. Given the presence of an imbalanced dataset, accuracy is not the best metric, as it is misleading [19]. Therefore, adopting the above-mentioned measures would be more appropriate than relying solely on accuracy [21]. The evaluation metrics utilised in the study are listed in the attached supplementary_file_1. Results This section presents the key findings of the study. The results are structured into two main subsections. The first subsection examines the characteristics of the features used in the analysis through descriptive statistics. The second subsection reports the results and comparative performance of the machine learning models applied in the study. Summary statistics This retrospective cohort study encompasses a total sample size of 2,940 neonatal deaths in Oman, spanning the years 2016 to 2024. As detailed in the methods section, the target variable was classified into a binary variable: Early neonatal deaths (accounting for 2,104 deaths, 71.6%) and late neonatal deaths (accounting for 836 deaths, 28.4%). Table 1 presents the distribution of the target variable alongside each categorical feature included in the study together with the p-value corresponding to the test of significant differences between early and late neonatal deaths. Table 1The summary statistics of the categorical features for early and late neonatal mortality (NM) and test of their differencesFeatureLevelsEarly NMn = 2,104Late NMn = 836Totaln = 2,940P-value of the test for the differencesGenderMale1,1414201,5610.0798Female9634161,3790.0762Delivery modeNormal1,3734551,828< 0.01*Caesarean7313811,112< 0.01*Employment statusEmployed6222388600.7644Un-employed1,4825982,0800.8171Mother educationIlliterate6030900.6665Primary & Middle school226923180.8460Secondary1,2164841,7000.8517Advanced6022308320.8157*p-value < 0.05 indicates significant differences Table 1The summary statistics of the categorical features for early and late neonatal mortality (NM) and test of their differencesFeatureLevelsEarly NMn = 2,104Late NMn = 836Totaln = 2,940P-value of the test for the differencesGenderMale1,1414201,5610.0798Female9634161,3790.0762Delivery modeNormal1,3734551,828< 0.01*Caesarean7313811,112< 0.01*Employment statusEmployed6222388600.7644Un-employed1,4825982,0800.8171Mother educationIlliterate6030900.6665Primary & Middle school226923180.8460Secondary1,2164841,7000.8517Advanced6022308320.8157*p-value < 0.05 indicates significant differences The summary statistics of the categorical features for early and late neonatal mortality (NM) and test of their differences *p-value < 0.05 indicates significant differences The results in Table 1 show a statistically significant difference between the mode of delivery for the early and late mortality. The results also suggest no statistically significant difference between the gender, employment status, and maternal education level (p-value > 0.05). Table 3 in the supplementary_file_1 provides insights into the characteristics of the numerical features included in our study. The mean birth weight of the total dataset was approximately 1,625 g, and the proportion of newborns with low birth weight (< 2500 g) was 78.4%. On the other hand, 50% of the newborns had a 5-minute APGAR score of 6 (median = 6). The mean maternal age was about 31 years, and approximately 22.5% of them were in their primigravida, while nearly 32% were a grand multigravida (pregnant 5 times or more). The mean gestational age at birth was 31.2 weeks, indicating a very high prematurity prevalence (born before 37 weeks of pregnancy) of about 75%. The maternal BMI was recorded at 30.6 kg/m², and the prevalence of maternal overweight and obesity was 28.6% and 50.4%, respectively. Table 2 presents the characteristics of the numerical features for early and late neonatal deaths with their statistical differences. The p-value measures the difference between the early and the late neonatal mortality for each numerical variable. Among the 12 numerical features, four variables have significant differences between the two classes of the target variable (p-value < 0.05), namely, child length, APGAR5 score, mother age, and the gestational age at birth. Notably, a highly significant difference (p-value < 0.001) was observed in the APGAR score at 5 min between early (mean = 5.0) and late neonatal deaths (mean = 7.1), as presented in Table 2. Table 2The summary statistics including minimum, maximum and standard deviation for numerical features of early and late mortality and test of their mean differencesVariableEarly neonatal deathsLate neonatal deathsP-value of mean differencesMinMaxSDMeanMinMaxSDMeanBIRTH WEIGHT230.05010.0929.51618.6410.04600.0955.41640.90.566CHILD LENGTH19.057.07.638.620.060.08.039.60.003*APGAR50.010.02.95.00.010.02.27.1< 0.001*Head circumference12.054.05.227.914.052.04.928.30.093MOTHER AGE17.049.06.231.218.048.06.631.80.033*GRAVIDA1.017.02.53.71.016.02.63.70.917Gestational age at birth5.044.06.031.019.043.05.931.50.036*Mother diastolic blood pressure21.0160.016.774.643.0167.018.175.40.292Mother systolic blood pressure45.0180.019.0113.053.0195.019.4114.40.079Mother temperature33.739.30.436.735.039.50.436.70.522MOTHER PULSE36.0146.012.186.052.0155.012.786.10.915BMI_calculated14.597.37.530.514.669.47.030.90.246*p-value < 0.05 indicates significant differences Table 2The summary statistics including minimum, maximum and standard deviation for numerical features of early and late mortality and test of their mean differencesVariableEarly neonatal deathsLate neonatal deathsP-value of mean differencesMinMaxSDMeanMinMaxSDMeanBIRTH WEIGHT230.05010.0929.51618.6410.04600.0955.41640.90.566CHILD LENGTH19.057.07.638.620.060.08.039.60.003*APGAR50.010.02.95.00.010.02.27.1< 0.001*Head circumference12.054.05.227.914.052.04.928.30.093MOTHER AGE17.049.06.231.218.048.06.631.80.033*GRAVIDA1.017.02.53.71.016.02.63.70.917Gestational age at birth5.044.06.031.019.043.05.931.50.036*Mother diastolic blood pressure21.0160.016.774.643.0167.018.175.40.292Mother systolic blood pressure45.0180.019.0113.053.0195.019.4114.40.079Mother temperature33.739.30.436.735.039.50.436.70.522MOTHER PULSE36.0146.012.186.052.0155.012.786.10.915BMI_calculated14.597.37.530.514.669.47.030.90.246*p-value < 0.05 indicates significant differences The summary statistics including minimum, maximum and standard deviation for numerical features of early and late mortality and test of their mean differences *p-value < 0.05 indicates significant differences Model performance and evaluation This section outlines the results of the trained machine learning models across the three scenarios employed in our study, along with the evaluation metrics utilised for model performance. The first approach employed in this research is via utilising the original dataset. Table 3 provides a summary of the performance of the various ML algorithms with the original dataset.Table 3Original dataset models’ performanceMethodAccuracyRecall (Sensitivity)PrecisionF1 ScoreAUC (area under the curve)AUC-PR(precision-recall AUC)95% CI of AUC-PRBrier scoreQDA0.67800.82140.75160.78500.66190.83430.7916–0.87350.2134LDA0.74960.94520.76200.84380.72290.86500.8276–0.89790.1778LR0.73760.95710.74720.83920.72570.86400.8256–0.89830.1779KNN0.70190.89760.74070.81160.66860.83450.7964–0.87030.1939ANN0.73250.88100.77570.82500.71260.86160.8250–0.89400.1828RF0.72400.93100.74620.82840.70550.86520.8307–0.89430.1836XGBoost0.71550.99050.71850.83280.70190.86670.8334–0.89540.2111DT0.66270.77860.75690.76760.65170.80310.7562–0.85030.2323SVM0.71550.97380.72390.83050.66490.83390.7921–0.87080.1921NB0.72740.87380.77430.82100.70620.85940.8218–0.89300.1866Where: QDA Quadratic Discriminant Analysis, LDA Linear Discriminant Analysis, LR Logistic Regression, KNN K Nearest Neighbors, ANN Artificial Neural Network, RF Random Forest, XGBoost Extreme Gradient Boosting, DT Decision Tree, SVM Support Vector Machine, NB Naive Bayes Table 3Original dataset models’ performanceMethodAccuracyRecall (Sensitivity)PrecisionF1 ScoreAUC (area under the curve)AUC-PR(precision-recall AUC)95% CI of AUC-PRBrier scoreQDA0.67800.82140.75160.78500.66190.83430.7916–0.87350.2134LDA0.74960.94520.76200.84380.72290.86500.8276–0.89790.1778LR0.73760.95710.74720.83920.72570.86400.8256–0.89830.1779KNN0.70190.89760.74070.81160.66860.83450.7964–0.87030.1939ANN0.73250.88100.77570.82500.71260.86160.8250–0.89400.1828RF0.72400.93100.74620.82840.70550.86520.8307–0.89430.1836XGBoost0.71550.99050.71850.83280.70190.86670.8334–0.89540.2111DT0.66270.77860.75690.76760.65170.80310.7562–0.85030.2323SVM0.71550.97380.72390.83050.66490.83390.7921–0.87080.1921NB0.72740.87380.77430.82100.70620.85940.8218–0.89300.1866Where: QDA Quadratic Discriminant Analysis, LDA Linear Discriminant Analysis, LR Logistic Regression, KNN K Nearest Neighbors, ANN Artificial Neural Network, RF Random Forest, XGBoost Extreme Gradient Boosting, DT Decision Tree, SVM Support Vector Machine, NB Naive Bayes Original dataset models’ performance Where: QDA Quadratic Discriminant Analysis, LDA Linear Discriminant Analysis, LR Logistic Regression, KNN K Nearest Neighbors, ANN Artificial Neural Network, RF Random Forest, XGBoost Extreme Gradient Boosting, DT Decision Tree, SVM Support Vector Machine, NB Naive Bayes Across the 10 models, AUC-PR values clustered between 0.8031 and 0.8667 with overlapping 95% CIs, indicating no statistically significant differences in discrimination for the positive class. Figure 2 illustrates the comparison of the area under the precision-recall curve (AUC-PR) of the ten models. The Delong test for ROC curves differences, demonstrated that there is no statistically significant difference between ROC curves of LDA and LR (P-value = 0.606). The same conclusion is drawn for all other models, except for QDA where the p-value was 0.0003. All the results of the pairwise Delong tests can be found on the supplementary file named (delong_tests_Original data). Fig. 2Comparison of the model AUC-PR curves using the original dataset Comparison of the model AUC-PR curves using the original dataset Considering probability accuracy, LDA and LR achieved the lowest Brier scores (0.1778 and 0.1779, respectively). LDA also yielded the highest F1-score (0.8438) with high recall (0.9452) and good precision (0.7620). Although Extreme Gradient Boosting (XGBoost) attained the highest recall (0.9905), its calibration was poorer (Brier = 0.2111) with lower precision (0.7185). The RF achieved a high AUC-PR, but its other metrics are lower than those of LDA and LR. In addition, its Brier score is higher compared to Brier scores of LDA and LR as presented in Table 3. We therefore select LDA as the primary model (LR as a close alternative), and we report XGBoost as a high-sensitivity option for settings prioritising maximal case detection. Figure 3 presents the calibration plots (mean predicted probability vs. observed event proportion) of the ten machine learning models for the first scenario. Fig. 3Calibration plots using the original dataset Calibration plots using the original dataset In the second approach, we employed the SMOTE to address the class imbalance in the dataset. More details of the dataset characteristics after SMOTE is presented in the supplementary_file_1. The results of the 10 machine learning algorithms with SMOTE are presented in Table 7 in the supplementary_file_1. Notably, RF maintained an AUC-PR of (0.8692) post-balancing the data which is comparable to LR and LDA. However, RF reported the lowest Brier score (0.1864) implying that the model is well calibrated compared to other models, including LDA and LR. As a result, RF is outperforming other models in the second scenario which employs SMOTE based on the highest F-score, highest AUC-PR and the lowest Brier score. Delong test results after SMOTE are presented in the supplementary file named (delong_tests_SMOTE). The calibration plots of the 10 machine learning models after SMOTE are presented in the supplementary file. The performance of the 10 machine learning models was evaluated in the third scenario, which is the employment of an algorithm-driven approach via cost-sensitive classification. The performance metric is presented in Table 8 of the supplementary_file_1. The LDA performed better compared to other algorithms, with the highest metrics indicating its power in this probabilistic prediction. On the other hand, robust results were displayed by RF, but lower calibration power as per its Brier score (0.1832) compared to LDA (0.1778). The Delong pairwise tests are presented in the supplementary file named (delong_tests_Cost_Sensitive) for more details. The feature importance plot from the RF (using original data) illustrates that APGAR 5 is the most important feature, followed by birth weight (Fig. 4). The other feature importances are ranked from top to bottom in the same figure. Variable importance for other models, including LR, DT, ANN, and XGBoost are presented in the supplementary files. All the models consistently reported the five-minute APGAR score as the top important feature in this classification problem. Fig. 4Feature importance based on RF model using original data Feature importance based on RF model using original data Discussion In this retrospective cohort study, we utilised a nationwide sample comprising 2,940 neonatal deaths in Oman from 2016 to 2024, spanning the first nine years of the SDGs to predict early and late neonatal mortality. To the best of our knowledge, this is one of the first studies to predict neonatal mortality differentiated by early and late deaths. A review of the literature revealed only one prior study addressing early mortality among very low birth weight preterm infants [22]. By monitoring the significant factors associated with early mortality death, our findings have the potential to inform clinical decision-making and support timely interventions aimed at improving survival rates and quality of life. Overall, the LR, LDA and RF demonstrated better performance in the three approaches in terms of their AUC-PR and Brier scores. Although, the performance didn’t improve significantly compared to the original dataset scenario, it suggests that future work in Oman should focus on collecting more relevant features rather than rebalancing the existing dataset. Our results are consistent with the findings of a prior study that predicted early deaths among very low birth weight preterm infants [22]. In that study, the authors reported that ANN and LR outperformed other models, with AUC scores of 0.87 and 0.86, respectively. While certain machine learning algorithms, such as RF, are well-suited for capturing non-linear and complex relationships, LR demonstrated superior predictive performance for neonatal mortality in neonatal intensive care units outperforming ANNs, Convolutional Neural Networks (CNNs), RF, and XGBoost in terms of AUC [23]. Various features have been incorporated into our research; however, the APGAR score at 5 min emerged as the most important variable in our models. Other studies also indicated that a low APGAR score at 5 min is highly associated with neonatal deaths [23, 24]. Consequently, health practitioners should be cautious about the accurate reporting of this score and ensure it is fully reported for each newborn. Future research may employ ML algorithms to predict APGAR scores in Omani newborns with a nationally representative sample, along with features collected during pregnancy and after delivery. Additionally, our findings revealed a high prevalence of low birth weight among the neonatal mortality cases in our sample (78.4%), which is approximately 6 times higher than the 2023 prevalence of low birth weight in Oman., according to the annual statistics by the Omani Ministry of Health [25]. Hence, it is recommended that future research employ ML algorithms to predict the risk of low birth weight. Several strengths are worth acknowledging in our study. We included a representative large sample size (n = 2,940). To the best of our knowledge, this is the first large-scale study to predict early and late neonatal deaths in Oman. Furthermore, we employed three different scenarios for model training and compared the evaluation metrics to select the best performing model. The employment of 10 ML techniques provides comprehensive insights into their predictive ability for this classification problem. We included various recommended features of the mother and the baby. Our study has the following limitations. The features in our study were collected at the time of birth. Subsequent work should incorporate features during pregnancy and characteristics collected from ultrasound that are essential for maternal outcomes, such as small for gestational [26], fetal growth, fundal height [27], chest circumference [28], and congenital anomalies [29, 30]. Therefore, more attention should be directed to the data collected during antenatal care. Currently, information collected from ultrasound during antenatal visits is not adequately documented within the health information system (ALSHIFA). Additionally, our study didn’t account for maternal comorbidities, such as diabetes or heart disease, which are crucial for birth outcomes. Moreover, as we utilise a dataset from Oman, our study’s results may be limited to the Omani population and the Omani healthcare system. The applications of the models to other healthcare settings depend on the availability of resources and the variables collected by the health information system in each country. Conclusion Neonatal mortality is a major global health issue. The neonatal period is defined as the first 28 days of a newborn’s life and is the most vulnerable time for child survival. The early neonatal death refers to death between 0 and 6 days, and late to death between 7 and 27 days. A sample of 2,940 de-identified neonatal death data was utilised to train and test ten machine learning techniques across three experimental setups to predict early and late neonatal mortality. To perform a more nuanced model evaluation, we used 10-fold cross-validation and utilised comprehensive metrics, including AUC, AUC-PR, and Brier score, given the issue of an imbalanced dataset in our study. The results of our study demonstrated that early neonatal deaths accounted for about 71.6% of the neonatal deaths compared to 28.4% of late neonatal mortality. Across the three scenarios, logistic regression, linear discriminant analysis and random forest exhibited better performance based on their corresponding Calibration and discrimination measures. However, random forest showed a more stable calibration across the three scenarios compared to the other two models (Brier scores of 0.1836, 0,1864, and 0.1832). The APGAR score at 5 min underscores its high importance in the early neonatal mortality classification problem, highlighting the importance of accurately reporting this measure. Other variables identified as highly significant include birth weight, mother’s BMI, and child length. Monitoring the variables associated with early neonatal death would enable health practitioners to effectively plan and implement early interventions to identify those at risk of early death. Consequently, achieving indicator 3.2.2 in goal 3 of the Sustainable Development Goals (SDG 3). Future work should incorporate hybrid machine learning algorithms with more features collected during pregnancy and the postnatal period to enhance the robustness and prediction accuracy of the model. Supplementary Information Supplementary Material 1 Supplementary Material 1