Development and internal validation of a machine learning prediction model for postoperative hemorrhage following modified radical mastectomy for breast cancer Authors: Weiyuan Chen, Xiaoxu Li, Xiaoman Zhang, Wenqi Shen Published: 2026 ABSTRACT AbstractBackgroundPostoperative hemorrhage is a clinically meaningful complication of modified radical mastectomy (MRM), but existing risk-prediction models rely on logistic regression (LR), rarely provide interpretable individualized risk attribution, and have not been specifically developed or internally validated for MRM. We aimed to develop and internally validate machine learning models for predicting clinically significant postoperative hemorrhage after MRM.MethodsWe retrospectively analyzed consecutive patients who underwent MRM at the Affiliated Hospital of Xuzhou Medical University between January 2022 and October 2024. Patients operated between January 2022 and December 2023 constituted the development cohort; patients operated between January and October 2024 constituted a strictly held-out temporal validation cohort. The primary endpoint was clinically significant hemorrhage, defined as unplanned reoperation for bleeding, transfusion for bleeding, or postoperative wound drainage exceeding 500 mL within any 24-hour period, ascertained over a 30-day postoperative follow-up window. Five algorithms—LR, decision tree (DT), random forest (RF), support vector machine (SVM), and extreme gradient boosting (XGBoost)—were trained on least absolute shrinkage and selection operator (LASSO)-selected predictors and evaluated by area under the curve (AUC), calibration metrics, decision curve analysis (DCA), and 1,000-iteration optimism-corrected bootstrap. Shapley additive explanations (SHAP) were used for model interpretation.ResultsA total of 1,842 patients were analyzed, of whom 74 developed clinically significant postoperative hemorrhage. The XGBoost model achieved the best performance [temporal validation AUC 0.827, 95% confidence interval (CI): 0.734–0.920; sensitivity 0.737 and specificity 0.814 at the Youden-derived threshold; Brier score 0.036], outperforming RF (AUC 0.819), SVM (0.794), LR (0.783), and DT (0.738). The XGBoost model demonstrated good calibration and the highest decision-curve net benefit across threshold probabilities of 5–30%. SHAP analysis identified tumor size, age, operation time, neoadjuvant chemotherapy, lymph node metastasis, pathological stage, and preoperative anticoagulant use as the seven leading predictors.ConclusionsAn XGBoost-based machine learning model predicted clinically significant postoperative hemorrhage after MRM and provided interpretable individualized risk estimates that may support early perioperative risk stratification and individualized hemostatic management. As a single-center study with internal (temporal) validation only and modest sensitivity, these findings should be regarded as exploratory, and multicenter external validation is required before clinical use.Keywords: Breast cancer, postoperative hemorrhage, machine learning, extreme gradient boosting (XGBoost), risk prediction model Highlight box Key findings • An extreme gradient boosting-based machine learning model predicted clinically significant postoperative hemorrhage after modified radical mastectomy (MRM) with a temporal validation area under the curve of 0.827, a Brier score of 0.036, and the highest decision-curve net benefit across threshold probabilities of 5–30%. • Tumor size, age, and operation time were the leading predictors on Shapley additive explanations (SHAP) analysis. What is known and what is new? • A 2026 meta-analysis of 82,012 patients identified mastectomy, hypertension, and congestive heart failure as confirmed independent risk factors for post-breast cancer surgery bleeding, but no validated, procedure-specific risk-prediction tool exists for MRM. • This is the first interpretable machine learning prediction model developed and internally validated (including temporal validation) specifically for hemorrhage after MRM, providing individualized, SHAP-based risk attribution to support preoperative risk stratification. What is the implication, and what should change now? • Surgeons performing MRM can use the model at preoperative consultation to identify high-risk patients (predicted probability ≥10%) for prolonged perioperative monitoring, intensified intraoperative hemostasis, and multidisciplinary perioperative anticoagulation review. • Multicenter prospective external validation is the planned next step before clinical deployment. Introduction Breast cancer is the most prevalent malignancy in women globally and one of the leading causes of cancer-related mortality (1). Despite advances in early detection and systemic therapy, surgical resection remains a cornerstone of treatment for most patients with operable disease. Modified radical mastectomy (MRM) continues to be a primary surgical option, particularly for patients with locally advanced disease or unfavorable tumor-to-breast ratios. Postoperative hemorrhage occurs in approximately 1.1–3.3% of patients undergoing mastectomy or breast-conserving surgery in contemporary series (2) and may complicate recovery by causing wound hematomas requiring evacuation, increasing transfusion requirements, prolonging hospital stay, and delaying initiation of adjuvant systemic therapy (3,4). In pooled analysis, the risk factors that have reached statistical significance for post-breast cancer surgery bleeding are mastectomy (versus breast-conserving surgery), hypertension, and congestive heart failure, whereas axillary lymph node dissection, antithrombotic medication, age, diabetes, and neoadjuvant chemotherapy have not (2); preoperative antithrombotic use has nonetheless been reported as an independent predictor in a nationwide database study of 477,108 procedures (4). Risk-prediction tools in breast surgery have largely been built for other endpoints—a recent machine learning model, for example, predicted major complications within one year of post-mastectomy breast reconstruction but, like most such tools, was only internally validated (5). Existing risk assessment for hemorrhage has therefore relied largely on unadjusted association analyses, and no validated, procedure-specific prediction tool exists for MRM. Accurate preoperative identification of patients at elevated bleeding risk is therefore clinically important, allowing individualized perioperative planning, including targeted intraoperative hemostatic measures (such as meticulous cautery of at-risk vascular beds and selective use of topical hemostatic agents), perioperative anticoagulation review, and intensified postoperative monitoring. Risk prediction of hemorrhage after MRM has historically relied on logistic regression (LR), which remains a robust and widely used benchmark that often performs competitively in clinical prediction; its standard additive specification may, however, not fully capture non-linear relationships or high-order interactions among clinical variables unless these are explicitly modeled (6). Machine learning algorithms—including random forest (RF), support vector machine (SVM), decision tree (DT), and extreme gradient boosting (XGBoost)—can accommodate such non-linear and high-order structure and have shown improved discrimination or calibration in some clinical prediction tasks, although they do not uniformly outperform well-specified LR (6-8), and have been successfully applied to a range of perioperative and oncologic contexts including postoperative complications in major cancer surgery (7), prediction of vascular surgery outcomes (8), detection of breast cancer lymph node metastasis in patients eligible for neoadjuvant treatment (9), prognostic prediction in breast cancer (10), prediction of breast cancer-related lymphedema (11), and early-warning systems in critical care (12). However, dedicated and well-validated machine learning models for postoperative hemorrhage after MRM remain limited. Therefore, the present study aimed to develop and internally validate machine learning models for predicting clinically significant postoperative hemorrhage after MRM, to identify the key clinical predictors of hemorrhage, and to compare the predictive performance of multiple algorithms to identify the optimal model. We selected LR as the reference standard alongside four representative machine learning algorithms spanning tree-based (DT, RF, and XGBoost) and margin-based (SVM) families, and judged their utility by empirical discrimination and calibration rather than by algorithmic complexity. We present this article in accordance with the TRIPOD reporting checklist (available at https://gs.amegroups.com/article/view/10.21037/gs-2026-0268/rc). Methods Study participants This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the Ethics Committee of Affiliated Hospital of Xuzhou Medical University (No. XYFY2024-KL450-01), and individual consent for this retrospective analysis was waived. This study retrospectively analyzed consecutive patients who underwent MRM in the Department of Thyroid and Breast Surgery, Affiliated Hospital of Xuzhou Medical University, between January 2022 and October 2024. MRM was defined as either the Patey procedure (preserving the pectoralis major muscle and resecting the pectoralis minor muscle, with axillary lymph node dissection) or the Auchincloss procedure (preserving both the pectoralis major and minor muscles, with axillary lymph node dissection). Axillary lymph node dissection was an integral component of both procedures and was therefore performed in all included patients, including those in whom the axillary nodes were subsequently found to be free of metastasis. Eligible patients were assigned to two non-overlapping cohorts according to the date of surgery: patients operated between January 2022 and December 2023 constituted the development cohort, used for feature selection, model training, and hyperparameter tuning; patients operated between January and October 2024 constituted the temporal validation cohort, which was strictly held out during model development and used solely for the final evaluation of model performance. Sample size was not fixed in advance: all eligible consecutive cases within the prespecified window were enrolled rather than accrued to a predetermined target N. The resulting development cohort provided approximately 7.9 events per predictor for the seven variables retained in the final models, modestly below the conventional threshold of 10 events per variable; this constraint is addressed in the Discussion. Inclusion criteria were as follows: (I) histologically confirmed breast cancer; (II) MRM as defined above, confirmed by the operative record; (III) age ≥18 years; and (IV) availability of complete preoperative, intraoperative, and 30-day postoperative clinical data. Exclusion criteria were: (I) hematological or coagulation disorders predisposing to bleeding regardless of surgery; (II) another concurrent active malignancy; (III) cognitive or communication impairment that would compromise data collection; (IV) bilateral synchronous breast cancer surgery during the same admission; (V) incomplete clinical data; and (VI) breast-conserving surgery, simple mastectomy, classical Halsted radical mastectomy, or mastectomy with immediate breast reconstruction. Procedure classification was independently performed by a breast surgeon based on the operative record, blinded to bleeding outcomes; disagreements were resolved by a second senior breast surgeon. The full audit trail of inclusion and exclusion is provided in Table S1 and Figure 1. Outcome ascertainment extended to 30 days after surgery for every included patient and was based on the records held in the hospital information and electronic medical record (EMR) systems; availability of complete 30-day postoperative data was an inclusion requirement, so no included patient was lost to follow-up within this window. To verify the comparability of the two cohorts, the distribution of all candidate predictors was compared between the development and temporal validation cohorts using standardized mean differences (SMDs), with SMD <0.25 prespecified as indicating distributional stability across the time split. Study flow diagram of the modified radical mastectomy cohort. CI, confidence interval; LASSO, least absolute shrinkage and selection operator; SMD, standardized mean difference. Variables and data collection Patient demographic and clinical data were extracted from the hospital information system and EMR system, covering the entire study period from January 2022 to October 2024. Candidate predictors were prespecified based on prior literature on post-mastectomy hemorrhage (2,4) and were extracted from structured fields of the EMR by two trained research assistants using a prespecified data dictionary; a 5% random sample was independently audited by an internal medicine physician for accuracy. Continuous variables with values outside biologically plausible ranges (e.g., interval from last neoadjuvant chemotherapy to surgery <14 days) were flagged and reviewed against the original source records before being included or excluded; the outlier rate and disposition are reported in Table S2. All comorbidity definitions were operationalized by combining International Classification of Diseases, 10th Revision (ICD-10) coded diagnoses with documented therapeutic or diagnostic confirmation (medication records, laboratory values, or imaging or procedural evidence); patient self-report without supporting EMR documentation was not accepted as evidence of comorbidity. Demographic and comorbidity variables. Age (years) and body mass index (BMI; kg/m2) were analyzed as continuous variables; for descriptive presentation only, age was additionally tabulated using the prespecified cutoff of 60 years and BMI using the Chinese adult cutoff of 24 kg/m2. Hypertension was defined as a documented physician diagnosis with current antihypertensive medication or two or more clinic blood pressure readings ≥140/90 mmHg before admission. Diabetes mellitus was defined as a documented physician diagnosis with current glucose-lowering therapy or fasting plasma glucose ≥7.0 mmol/L on at least two occasions. Coronary heart disease was defined as a documented physician diagnosis with imaging or angiographic confirmation (coronary angiography, coronary computed tomography angiography, or prior percutaneous coronary intervention or coronary artery bypass grafting); patient self-report alone was not accepted. Tumor and pathological characteristics—tumor size, pathological type, pathological stage, and axillary lymph node metastasis—were determined from the final postoperative pathology report. Pathological stage was assigned according to the American Joint Committee on Cancer (AJCC) 8th edition pTNM classification (13). For multifocal tumors, tumor size was defined as the maximum diameter of the largest invasive focus, and lesion location was recorded as the quadrant of the largest invasive focus. Treatment-related variables. Preoperative neoadjuvant chemotherapy was defined as administration of any cytotoxic chemotherapy regimen completed before the index surgery, with the interval from the last chemotherapy cycle to surgery recorded in days. Preoperative use of anticoagulants or antiplatelet agents was defined as documented use of any oral anticoagulant (warfarin or direct oral anticoagulants) or antiplatelet agent (aspirin, clopidogrel, ticagrelor, etc.) within 14 days before surgery, regardless of whether the medication was discontinued preoperatively. Operation time was recorded in minutes from skin incision to skin closure as documented in the anesthesia record. Primary endpoint and adjudication. The primary endpoint was clinically significant postoperative hemorrhage occurring within 30 days after surgery, defined as the occurrence of at least one of the following objective, action-driven criteria: (I) unplanned reoperation for bleeding control or evacuation of a surgical-site hematoma, including bedside or operating-room re-exploration; (II) red blood cell transfusion administered for postoperative bleeding; or (III) postoperative wound drainage exceeding 500 mL of grossly bloody fluid within any continuous 24-hour period during the first 7 postoperative days. Serous or serosanguineous drainage, isolated hemoglobin decline, dressing soiling not requiring intervention, and incidental imaging findings without clinical or therapeutic action were not classified as bleeding events. Bleeding outcome was independently adjudicated from the EMR by two attending-level adjudicators (one breast surgeon and one general surgeon), neither of whom was involved in model development, blinded to all candidate predictors. Each adjudicator independently classified bleeding (yes or no) and recorded the triggering criterion. Disagreements were resolved by a senior breast surgeon (chief physician). Inter-rater agreement before arbitration was assessed using Cohen’s kappa, and the full adjudication audit trail is provided in Table S3. Data preprocessing Duplicate features were eliminated, and features exhibiting complete collinearity were removed. Variables or patients with a missing rate exceeding 20% were excluded. For continuous variables with <20% missingness, multiple imputation by chained equations was performed using parameters fitted only on the development cohort; for categorical variables, missingness was treated as a separate category. Overall missingness was low (below the 20% exclusion threshold for all retained variables), and multiple imputation by chained equations was adopted under a missing-at-random assumption; this assumption is supported by the predominantly administrative nature of the missing values (non-systematic gaps in recorded fields such as operation time or BMI) rather than any relationship to the bleeding outcome. Imputation models were fitted within each cross-validation fold to prevent leakage, and categorical missingness was retained as a separate category rather than imputed, avoiding the same distributional assumption where it is less tenable. All preprocessing parameters (imputation values and the min–max scaler for normalization) were fitted exclusively on the development cohort and then applied to transform the temporal validation cohort, to prevent data leakage. Class imbalance in the development cohort was addressed by the synthetic minority over-sampling technique (SMOTE), applied solely to the development cohort after feature scaling and before model training (bleeding incidence 4.1%; 55 events among 1,340 patients); the temporal validation cohort retained its original class distribution (3.8% incidence; 19 events among 502 patients). Model development and validation All model development was conducted exclusively within the development cohort, with the temporal validation cohort strictly held out until final evaluation, following established prediction-model development principles (14). The pipeline comprised four sequential, leakage-protected steps. Candidate predictors were screened by least absolute shrinkage and selection operator (LASSO) regression with the penalty parameter λ selected by 10-fold cross-validation. Both λmin and λ1se solutions were examined, and the λ1se solution was used for final modeling to favor parsimony. Variance inflation factors were calculated for the LASSO-selected predictors, with VIF <5 considered acceptable for multicollinearity. Five algorithms, namely LR, DT, RF, SVM, and XGBoost (15), were then compared on this common feature set. For each algorithm, a hyperparameter grid was evaluated by 10-fold cross-validation on the development cohort. The final configuration for each model was fixed within the development-cohort workflow before the temporal validation cohort was scored; these configurations and the complete hyperparameter search space are reported below and in Table S4. The classification probability threshold for each model was determined by Youden’s index on cross-validated training predictions and is reported in Table S5. To prevent data leakage, missing-value imputation, feature scaling, and SMOTE were refit within each cross-validation fold using only the corresponding training fold. For XGBoost, the search space included number of trees [100, 200, 300, 500], maximum depth [3, 4, 5, 6], learning rate [0.01, 0.05, 0.10, 0.20], and subsample ratio [0.6, 0.8, 1.0]; the pre-specified hyperparameters used were n_estimators =300, max_depth =4, learning_rate =0.05, and subsample =0.8. Fourth, each final model was retrained on the entire development cohort using the fixed configuration and then evaluated by (I) 1,000-iteration optimism-corrected internal validation within the development cohort, reporting the bias-corrected C-statistic; and (II) temporal validation on the held-out cohort. Once the temporal validation cohort was scored, no further model adjustment was permitted. Discrimination was assessed by AUC with 95% confidence intervals (CIs) and pairwise AUC comparison by DeLong’s test. Calibration was assessed by calibration plots, the Hosmer-Lemeshow goodness-of-fit test, calibration intercept and slope, and the Brier score. Final predicted probabilities were additionally transformed by a fixed monotone probability-scale recalibration (parameters reported in Table S4); because the mapping is monotone it preserves case ranking and the AUC, but it changes the classification threshold, Brier score, calibration, and decision-curve analysis. Clinical utility was assessed by DCA across a clinically relevant range of threshold probabilities (16). Additional classification metrics (accuracy, precision, sensitivity, specificity, F1 score) and the confusion matrix were reported at the model-specific Youden threshold. A temporal-validation AUC ≥0.75 together with adequate calibration (calibration slope 0.8–1.2 and a non-significant Hosmer-Lemeshow test) was prespecified as the minimum benchmark for a potentially useful model, with independent external validation regarded as a prerequisite for any clinical application. Subgroup performance was evaluated by age (<60 vs. ≥60 years), neoadjuvant chemotherapy status, and number of comorbidities. Model interpretation was performed using Shapley additive explanations (SHAP) (17); the global mean absolute SHAP values (bar plot), SHAP summary plot (beeswarm), SHAP dependence plots for the top three predictors, and individual SHAP force plots for two representative cases were reported. Statistical analysis Continuous variables are expressed as mean ± standard deviation (SD) for normally distributed data and as median with interquartile range otherwise; categorical variables are expressed as n (%). Between-group comparisons used Student’s t-test or the Mann-Whitney U test for continuous variables and the Chi-squared test or Fisher’s exact test for categorical variables, as appropriate. Two-sided P<0.05 was considered statistically significant. All analyses were performed in Python 3.11 (with scikit-learn 1.4 and SciPy 1.13) and R 4.3 (with the rms and dca packages used for calibration and decision curve analysis). Results Cohort characteristics Of 2,346 breast cancer surgical cases performed at the study center between January 2022 and October 2024, 1,842 patients meeting the eligibility criteria for MRM were included in the final analysis (Figure 1). The development cohort comprised 1,340 patients operated between January 2022 and December 2023 (55 hemorrhage events; 4.1%), and the temporal validation cohort comprised 502 patients operated between January and October 2024 (19 hemorrhage events; 3.8%). Inter-rater agreement for outcome adjudication was excellent (Cohen’s κ=0.87, 95% CI: 0.81–0.93). The maximum SMD across all 14 candidate predictors between the two cohorts was 0.105 (Table S6), well below the prespecified threshold of 0.25. Comparison of baseline characteristics between bleeding and non-bleeding patients in the development cohort is shown in Table 1. Patients with bleeding were significantly older (57.2±9.1 vs. 54.3±10.8 years, P=0.02), had larger tumors (4.5±1.6 vs. 3.5±1.7 cm, P<0.001), more frequently received neoadjuvant chemotherapy (36.4% vs. 21.6%, P=0.01), and had higher rates of axillary lymph node metastasis (65.5% vs. 51.8%, P=0.046). Table 1Comparison of baseline characteristics between bleeding and non-bleeding groups in the development cohort (N=1,340)VariableBleeding (n=55)Non-bleeding (n=1,285)Test statisticP valueAge, years57.2±9.154.3±10.8t=2.3290.02BMI, kg/m225.3±3.624.8±3.4t=1.0320.31Tumor size, cm4.5±1.63.5±1.7t=4.652<0.001Operation time, min161.2±37.7153.5±34.9t=1.4900.14Hypertension17 (30.9)379 (29.5)χ2=0.0510.82Diabetes4 (7.3)149 (11.6)χ2=0.9740.32Coronary heart disease4 (7.3)70 (5.4)χ2=0.3370.56Preoperative anticoagulant use10 (18.2)129 (10.0)χ2=3.7620.052Neoadjuvant chemotherapy20 (36.4)278 (21.6)χ2=6.6170.01Operative side: left26 (47.3)634 (49.3)χ2=0.0900.76Lymph node metastasis36 (65.5)665 (51.8)χ2=3.9700.046Pathological stageχ2=2.4500.29 Stage I9 (16.4)324 (25.2) Stage II30 (54.5)658 (51.2) Stage III16 (29.1)303 (23.6)Data are presented as mean ± SD or n (%). BMI, body mass index; SD, standard deviation. Comparison of baseline characteristics between bleeding and non-bleeding groups in the development cohort (N=1,340) Data are presented as mean ± SD or n (%). BMI, body mass index; SD, standard deviation. Model performance LASSO regression with 10-fold cross-validation in the development cohort selected seven non-zero predictors at λ1se: tumor size, neoadjuvant chemotherapy, age, preoperative anticoagulant use, lymph node metastasis, operation time, and pathological stage (Table 2). Variance inflation factors for these predictors ranged from 1.08 to 2.14, indicating no significant multicollinearity. Table 2LASSO regression coefficients at λ1se in the development cohortFeatureλ1se coefficientλmin coefficientTumor size0.44530.5821Neoadjuvant chemotherapy0.24170.3145Age0.18870.2632Preoperative anticoagulant use0.15440.2106Lymph node metastasis0.14590.1882Operation time0.12010.1693Pathological stage0.06550.1115Body mass index0.02650.0617Lesion location−0.0618−0.0892Diabetes−0.0421−0.0683Hypertension0.00000.0395Coronary heart disease0.00000.0214Operative side0.00000.0156Pathological type0.0000−0.0089Coefficients shown on the standardized scale. λ1se was used for the final model (parsimony favored). LASSO, least absolute shrinkage and selection operator. LASSO regression coefficients at λ1se in the development cohort Coefficients shown on the standardized scale. λ1se was used for the final model (parsimony favored). LASSO, least absolute shrinkage and selection operator. Five prediction models were trained on the seven LASSO-selected predictors. On the development cohort (Table 3), all five models achieved AUCs ≥0.76, with XGBoost the highest (AUC 0.861, 95% CI: 0.823–0.905). On the temporal validation cohort (Table 4 and Figure 2), XGBoost again achieved the highest AUC (0.827, 95% CI: 0.734–0.920), followed by RF (0.819), SVM (0.794), LR (0.783), and DT (0.738). The development-to-validation AUC drop ranged from 0.005 to 0.034 across models, indicating limited overfitting. Pairwise DeLong comparisons on the validation cohort showed XGBoost numerically superior to all other models, with statistical significance reached only versus the DT (ΔAUC =+0.089, P=0.002); the XGBoost-versus-LR difference (ΔAUC =+0.044) did not reach significance (P=0.29). Full pairwise comparisons are reported in Table S7. Table 3Performance of the five prediction models on the development cohort (N=1,340; 55 events)ModelAUC (95% CI)AccuracyPrecisionSensitivitySpecificityF1 scoreBrier scoreXGBoost0.861 (0.823–0.905)0.8530.1870.7640.8570.3000.046RF0.845 (0.798–0.884)0.8380.1710.7450.8420.2780.052SVM0.799 (0.766–0.858)0.8020.1430.7270.8060.2390.058LR0.791 (0.751–0.845)0.7940.1390.7090.7980.2320.062DT0.762 (0.715–0.811)0.7690.1180.6730.7730.2010.071AUC, area under the curve; CI, confidence interval; DT, decision tree; LR, logistic regression; RF, random forest; SVM, support vector machine; XGBoost, extreme gradient boosting. Performance of the five prediction models on the development cohort (N=1,340; 55 events) AUC, area under the curve; CI, confidence interval; DT, decision tree; LR, logistic regression; RF, random forest; SVM, support vector machine; XGBoost, extreme gradient boosting. Table 4Performance of the five prediction models on the temporal validation cohort (N=502; 19 events)ModelAUC (95% CI)AccuracyPrecisionSensitivitySpecificityF1 scoreBrier scoreXGBoost0.827 (0.734–0.920)0.8110.1350.7370.8140.2280.036RF0.819 (0.731–0.906)0.7850.1130.6840.7890.1940.049SVM0.794 (0.705–0.882)0.7650.1040.6840.7680.1810.058LR0.783 (0.693–0.873)0.7550.1000.6840.7580.1740.061DT0.738 (0.625–0.851)0.7330.0860.6320.7370.1520.073Sensitivity and specificity reported at each model’s Youden-derived threshold (see Table S5). AUC, area under the curve; CI, confidence interval; DT, decision tree; LR, logistic regression; RF, random forest; SVM, support vector machine; XGBoost, extreme gradient boosting. Performance of the five prediction models on the temporal validation cohort (N=502; 19 events) Sensitivity and specificity reported at each model’s Youden-derived threshold (see Table S5). AUC, area under the curve; CI, confidence interval; DT, decision tree; LR, logistic regression; RF, random forest; SVM, support vector machine; XGBoost, extreme gradient boosting. ROC curves of the five prediction models. (A) Development cohort, mean of 10-fold cross-validation. (B) Temporal validation cohort, with the optimal Youden cutoff annotated for the XGBoost model (sensitivity =0.737, specificity =0.814). AUC, area under the curve; DT, decision tree; LR, logistic regression; RF, random forest; ROC, receiver operating characteristic; SVM, support vector machine; XGBoost, extreme gradient boosting. At the model-specific Youden-derived threshold (Table S5), the XGBoost model achieved sensitivity 0.737 and specificity 0.814 on the temporal validation cohort, with positive and negative predictive values of 13.5% and 98.7%, respectively (Table 5); the low positive predictive value reflects the 3.8% base rate. As shown in Figure 3, the XGBoost model demonstrated good calibration on the temporal validation cohort (Hosmer-Lemeshow P=0.32, Brier score 0.036, calibration intercept −0.15, calibration slope 1.02), with the four other models showing acceptable but inferior calibration. DCA illustrated in Figure 3 demonstrated that XGBoost provided greater net clinical benefit than treat-all, treat-none, and the four other models across the threshold-probability range of 5–30%; at a threshold of 10%, XGBoost yielded a net benefit of 0.018 (equivalent to correctly identifying 1.8 additional bleeding patients per 100 evaluated) versus 0.0167 for LR. Table 5Confusion matrix of the XGBoost model on the temporal validation cohort at the Youden-derived threshold (N=502)Actual outcomePredicted: non-bleedingPredicted: bleedingTotal (actual)Actual: non-bleeding393 (TN)90 (FP)483Actual: bleeding5 (FN)14 (TP)19Total (predicted)398104502Threshold =0.078. FN, false negative; FP, false positive; TN, true negative; TP, true positive; XGBoost, extreme gradient boosting. Confusion matrix of the XGBoost model on the temporal validation cohort at the Youden-derived threshold (N=502) Threshold =0.078. FN, false negative; FP, false positive; TN, true negative; TP, true positive; XGBoost, extreme gradient boosting. Decision curve analysis, calibration, and predicted-probability distribution on the temporal validation cohort. (A) Decision curve analysis showing the net benefit of the five prediction models, treat-all, and treat-none strategies across threshold probabilities of 0.02–0.50. (B) Calibration curves of the five prediction models with bin-wise observed proportions and 95% confidence intervals for XGBoost; calibration trends of the four other models are overlaid. (C) Distribution of XGBoost-predicted probabilities in the temporal validation cohort; the dashed red line marks the cohort event rate (3.8%). DT, decision tree; LR, logistic regression; RF, random forest; SVM, support vector machine; XGBoost, extreme gradient boosting. Model interpretation Shapley additive explanations were computed for the XGBoost model. As shown in Figure 4A, the seven predictors ranked by mean absolute SHAP value were tumor size (1.982), age (1.105), operation time (0.955), neoadjuvant chemotherapy (0.785), lymph node metastasis (0.622), pathological stage (0.483), and preoperative anticoagulant use (0.160). The SHAP summary plot depicted in Figure 4B revealed that higher tumor size, older age, and longer operation time separated toward positive SHAP values (higher predicted hemorrhage risk), with neoadjuvant chemotherapy-exposed patients also shifting toward positive SHAP values. Dependence plots for the top three continuous predictors (Figure 5A-5C) revealed approximately monotonic relationships, with steepening contributions for tumor size >4 cm and age >60 years. Force plots illustrating individualized risk attribution for one true-positive and one false-positive case are provided in Figure S1. SHAP-based global interpretation of the XGBoost model on the temporal validation cohort. (A) Global feature importance ranked by mean absolute SHAP value. (B) SHAP summary (beeswarm) plot, with feature value coded by color (blue, low; red, high). SHAP, Shapley additive explanations; XGBoost, extreme gradient boosting. SHAP dependence plots for the top three continuous predictors of postoperative hemorrhage from the XGBoost model on the temporal validation cohort. (A) SHAP dependence plot for tumor size; the dashed vertical line at 4 cm indicates a clinically meaningful inflection point. (B) SHAP dependence plot for age; the dashed vertical line at 60 years indicates a clinically meaningful inflection point. (C) SHAP dependence plot for operation time. Points in all panels are coloured by neoadjuvant chemotherapy status (red, NACT yes; blue, NACT no). NACT, neoadjuvant chemotherapy; SHAP, Shapley additive explanations; XGBoost, extreme gradient boosting. Bootstrap optimism correction (1,000 resamples) of the development AUCs yielded optimism-corrected C-statistics of 0.858 (XGBoost), 0.844 (RF), 0.773 (SVM), 0.783 (LR), and 0.732 (DT). The XGBoost optimism-corrected C of 0.858 closely matched its temporal validation AUC of 0.827 (difference 0.031), supporting good generalization. Learning curves further showed that 10-fold cross-validation AUC plateaued near N=1,000, indicating that the development cohort size of 1,340 was sufficient for stable model fitting (Figure S2). Subgroup analyses Subgroup analyses on the temporal validation cohort showed stable model performance across age strata (<60 vs. ≥60 years: AUC 0.819 vs. 0.843) and across comorbidity-burden strata (0, 1, or ≥2 comorbidities: AUC 0.806, 0.832, and 0.855, respectively). Discrimination in the neoadjuvant chemotherapy-exposed subgroup (AUC 0.840) was comparable to, and numerically slightly higher than, that in the non-exposed subgroup (AUC 0.813). Detailed subgroup results are reported in Table S8. Discussion Postoperative hemorrhage remains a clinically meaningful complication of MRM, capable of necessitating reoperation, delaying wound healing, and postponing the initiation of adjuvant systemic therapy; accurate preoperative risk stratification therefore has direct value for individualized surgical planning and perioperative management. In this study, we developed and internally validated five prediction models (LR as the reference standard plus four machine learning algorithms) for predicting clinically significant hemorrhage in 1,842 patients from a single tertiary center using a strictly held-out temporal validation cohort and optimism-corrected bootstrapping. The XGBoost model achieved the best performance (temporal validation AUC 0.827, Brier score 0.036, calibration slope 1.02), the highest decision-curve net benefit across the clinically relevant threshold range, and provided interpretable individualized risk attribution through SHAP analysis. On the temporal validation cohort, XGBoost outperformed LR numerically (ΔAUC =0.044, DeLong P=0.29), but the difference did not reach conventional statistical significance, consistent with prior systematic comparisons reporting that machine learning rarely outperforms well-specified LR on low-dimensional clinical data when methodological rigor is matched (6). The clinically meaningful advantages of XGBoost in our cohort instead lay in better calibration (Brier 0.036 vs. 0.061; calibration slope 1.02 vs. 0.54), greater net benefit on DCA across thresholds of 5–30%, and the ability to deliver individualized risk attribution through SHAP—features that are not natively available from linear models. Tumor size emerged as the strongest predictor on SHAP analysis, followed by age and operation time, with neoadjuvant chemotherapy ranking fourth. The prominence of tumor size, together with the contributions of lymph node metastasis and pathological stage, is biologically coherent in an MRM population enriched for cT2–4 and cN+ disease, in which greater tumor burden and more extensive resection increase the dissection surface and the number of vessels at risk. Older age and longer operation time plausibly index frailty and greater surgical complexity, respectively. Neoadjuvant chemotherapy nonetheless remained an important contributor: in the development cohort the unadjusted bleeding incidence was higher in chemotherapy-exposed than in non-exposed patients [6.7% (20/298) vs. 3.4% (35/1,042)], but because chemotherapy recipients also differed at baseline in tumor size, pathological stage, and operation time, this unadjusted contrast should be interpreted as part of the multivariable predictive signal rather than as an independent causal effect. Mechanistically, neoadjuvant chemotherapy may elevate bleeding risk through systemic effects (transient thrombocytopenia and impaired hemostasis) and local tissue alterations (stromal edema, fibrosis, increased microvascular fragility, and obscured anatomical planes) that become clinically relevant during extensive flap dissection (18). XGBoost captured these features together with the extent of axillary dissection, modeling interactions that linear models resolve less completely, which may partly explain the more prominent multivariable signal observed here than in pooled analyses dominated by breast-conserving surgery cohorts (2) that also used narrower reoperation-only endpoints (19). Lymph node metastasis and pathological stage completed the seven retained predictors, reinforcing tumor burden and surgical extent as the dominant drivers of hemorrhage risk in this population; their positive contributions on the SHAP dependence plots were approximately monotonic. Preoperative anticoagulant use, although retained by LASSO and clinically important, ranked lowest of the seven on SHAP analysis (mean |SHAP| 0.160). In our cohort, the prevalence of any oral anticoagulant or antiplatelet use within 14 days of surgery was 10.4%, and the unadjusted bleeding incidence was 7.2% (10/139) among users versus 3.7% (45/1,201) among non-users. Its comparatively modest global SHAP contribution likely reflects both its low prevalence and the fact that, once tumor- and procedure-related factors are accounted for, its marginal contribution to the model is small; this contrasts with nationwide administrative studies in which antithrombotic use ranked among the strongest independent predictors of post-mastectomy bleeding (4), a discrepancy that may reflect differences in case mix, endpoint definition, and the multivariable, non-linear structure captured by XGBoost. Notably, BMI and hypertension—retained in some prior association analyses—were not among the seven LASSO-selected predictors and therefore did not enter the SHAP analysis, underscoring that the present interpretation is conditional on the LASSO-selected feature set. Our study extends the recent single-center machine learning literature on breast surgical complications and prognosis (7,9,10,11,20) to post-mastectomy hemorrhage and adds two methodological enhancements: temporal rather than random validation, and SHAP-based individualized interpretation of the dominant predictors. The temporal validation AUC of 0.827 we obtained is comparable to single-center random-split studies on related outcomes published in the same journal (10,11), with the modest difference explained by the lower base rate of our outcome and the greater rigor of temporal versus random splitting. To our knowledge, this is the first machine learning-derived, individually interpretable risk model developed and internally validated specifically for hemorrhage after MRM. Methodological strengths include three layers of internal validation (10-fold cross-validation, optimism-corrected bootstrap with 1,000 resamples, and temporal validation on a strictly held-out cohort) and blinded double adjudication of bleeding events with senior arbitration (Cohen’s κ=0.87). Clinically, the model can be deployed at preoperative consultation to stratify candidates for MRM: patients with predicted probabilities ≥10% (corresponding to a treat-to-no-treat ratio of 9:1 on the decision curve) may benefit from prolonged perioperative monitoring, more aggressive intraoperative hemostasis, lower thresholds for type-and-screen blood preparation, intraoperative hemostatic adjuncts (21), and a multidisciplinary review of perioperative anticoagulation, with SHAP force plots supporting transparent shared decision-making. Several limitations should be acknowledged. As a single-center retrospective analysis, the model requires geographic external validation in multicenter prospective cohorts, which remains the planned next step before clinical deployment. The outcome was rare, and the events-per-predictor ratio in the development cohort was approximately 7.9 for the seven predictors retained in the final models—modestly below the conventional threshold of 10—reflecting the low incidence of clinically significant hemorrhage. Although the learning curves indicated stable fitting and discrimination at this sample size, the limited absolute number of events reduces the precision of effect estimates and the statistical power of pairwise AUC comparisons and constrains sensitivity at the population base rate; the model should therefore be regarded as exploratory pending external validation. The rarity of the outcome also raises the question of whether it is a suitable target for machine learning at all. Two observations suggest that it is, with qualifications. First, simulation work has shown that correcting class imbalance can degrade calibration and produce systematic over-estimation of risk without improving discrimination (22); we therefore applied SMOTE only within the training folds of the development cohort and left the temporal validation cohort at its native prevalence, and the model remained well calibrated on that untouched cohort (calibration slope 1.02, intercept −0.15, Hosmer-Lemeshow P=0.32). Second, the model is useful for stratification rather than diagnosis: at the Youden-derived threshold the positive predictive value was 13.5%, so approximately seven patients are flagged for every hemorrhage that occurs, whereas the negative predictive value of 98.7% supports its use in identifying patients who can safely follow routine postoperative monitoring. Machine learning is thus a reasonable approach for this task as a low-cost triage aid, but the gain over LR was modest and the model should not be used as a diagnostic test. Our composite hemorrhage endpoint, while clinically meaningful, is more sensitive than reoperation-only definitions and may not be directly comparable across studies. A pre-specified sensitivity analysis re-fit using a stricter reoperation-only endpoint yielded directionally consistent discrimination and calibration (Table S9), supporting the robustness of the primary findings. In addition, all patients underwent axillary lymph node dissection as part of the MRM per institutional practice during the study period, which may limit generalizability to centers that use sentinel-lymph-node-guided axillary staging. The deliberately narrow study window of 2 years and 10 months was chosen to minimize temporal drift in surgical technique and perioperative anticoagulation guidelines, but reduces generalizability over longer time horizons. Future research priorities include external multicenter validation, integration with other postoperative-complication prediction tools to provide a comprehensive perioperative risk dashboard (23), and development of a web-based clinical decision support tool incorporating the SHAP individualized-risk interface. Conclusions Among five prediction models developed and internally validated for predicting clinically significant postoperative hemorrhage after MRM in 1,842 patients, the XGBoost model achieved the best discrimination, good calibration, and the highest decision-curve net benefit across the clinically relevant threshold range. Tumor size, age, and operation time were the leading predictors on SHAP analysis, providing interpretable individualized risk attribution. The model can support individualized perioperative risk stratification and hemostatic management, and provides the first machine learning-derived, interpretable risk tool developed specifically for hemorrhage after MRM. Because validation to date is internal (temporal) only, geographic external validation in multicenter prospective cohorts is required before clinical deployment. Supplementary The article’s supplementary files as Acknowledgments None. Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the Ethics Committee of Affiliated Hospital of Xuzhou Medical University (No. XYFY2024-KL450-01), and individual consent for this retrospective analysis was waived. Footnotes Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://gs.amegroups.com/article/view/10.21037/gs-2026-0268/rc Funding: This work was supported by the Affiliated Hospital of Xuzhou Medical University (Nos. PYJH2024308 and 2023ZL15). Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://gs.amegroups.com/article/view/10.21037/gs-2026-0268/coif). All authors report that this work was supported by the Affiliated Hospital of Xuzhou Medical University (Nos. PYJH2024308 and 2023ZL15). The authors have no other conflicts of interest to declare. Data Sharing Statement Available at https://gs.amegroups.com/article/view/10.21037/gs-2026-0268/dss References Available at https://gs.amegroups.com/article/view/10.21037/gs-2026-0268/prf Supplementary Materials The article’s supplementary files as Data Availability Statement Available at https://gs.amegroups.com/article/view/10.21037/gs-2026-0268/dss