Development and evaluation of an explainable machine learning selection pipeline for predicting neoadjuvant chemoradiotherapy response in locally advanced rectal cancer Authors: Yijun Xu, Wei Gong, Lei Ji, Zhongxu Xing, Zhiyu Zhu, Zhibo Jin, Jiayu Zhang, Xinyi Wang, Songbing Qin, Yang Jiao, Lili Wang Published: 2026 ABSTRACT AbstractBackgroundThe selection of appropriate machine learning (ML) methods for clinical research remains challenging, particularly when both predictive performance and model explainability are required in small-sample datasets. Conventional approaches often rely on limited variables and expert-driven choices, whereas many ML models remain difficult to justify clinically. This study aimed to develop an explainable ML model selection pipeline and demonstrate its application in predicting neoadjuvant chemoradiotherapy (nCRT) response in locally advanced rectal cancer (LARC).MethodsWe proposed a six-stage explainable ML model selection pipeline comprising data selection and preprocessing, algorithm pool construction, model training, model evaluation, model explanation, and an internal clinical logic consistency check (non-deployable sanity check). The workflow was applied to a retrospective cohort of 128 patients with LARC treated with nCRT. Four ML algorithms, including support vector machine (SVM), decision tree (DT), random forest (RF), and logistic regression (LR), were evaluated. Model performance was assessed using accuracy, F1-score, AUROC, AUPRC, bootstrap confidence intervals, and five-fold cross-validation, followed by explainability screening using SHAP.ResultsUnder predefined screening criteria, DT models achieved the most favorable balance between predictive performance and explainability. Using pretreatment tumor markers alone, the selected DT models achieved an accuracy of 0.82 and F1-score of 0.71 for pathological complete response (pCR) prediction, and an accuracy of 0.76 and F1-score of 0.72 for tumor regression grade (TRG) prediction. SHAP analysis consistently identified carcinoembryonic antigen (CEA) and carbohydrate antigen 19–9 (CA19-9) as the most influential predictors, and lower baseline levels of these markers were associated with better pathological response.ConclusionsThis study provides a practical and reproducible framework for selecting interpretable ML models in small clinical datasets. In the present LARC case study, DT-based models showed acceptable discrimination and transparent decision logic, while tumor markers emerged as clinically plausible predictors of nCRT response. The proposed workflow may support future multi-center validation and broader application in clinically interpretable predictive modeling.Supplementary informationThe online version contains supplementary material available at https://doi.org/10.1186/s12967-026-08307-7.Keywords: Machine learning, Predictive modeling, Explainable artificial intelligence, Locally advanced rectal cancer, Carcinoembryonic antigen, Explainability Introduction Currently, Machine Learning (ML) has been a powerful tool for data analysis, prediction, and decision-making in various fields. Its advantages, including high efficiency and automated data processing, have contributed to its growing prominence in clinical practice [1–3]. Compared to Deep Learning (DL), ML methods require less data and computing resources, making them more suitable for certain medical contexts [4, 5]. However, medical data are typically highly diverse and heterogeneous in quality. Employing ML algorithms that are not well-suited to the available sample size can lead to misleading results, which may potentially compromise the transparency and credibility of the conclusions drawn. Existing evaluation metrics alone are often insufficient to address these challenges. Although some studies have explored model-sample matching [6, 7], comprehensive analyses within specific clinical contexts remain scarce. Therefore, selecting the most appropriate ML algorithms for a given clinical research project is of paramount importance. Clinical datasets are often limited in size, particularly for cancers and rare diseases, which can result in suboptimal model performance, and increased risk of unreliable predictions. Furthermore, ML models are also susceptible to opacity in their decision-making process [8] and lack robust explainability [9]. Our objective is to focus on this type of clinical data and emphasize the explainability of the selected ML methods. Accordingly, this paper proposes a pipeline that incorporates these considerations into its design. Colorectal cancer is the third most commonly diagnosed cancer and the second most common cause of cancer-related mortality worldwide, with rectal cancer accounting for approximately one-third of cases [10]. Due to the frequent absence of representative symptoms in early stages, most patients are diagnosed with locally advanced rectal cancer (LARC), typically defined as clinically advanced primary tumors (cT3–4) and/or regional lymph node-positive disease (N+), without distant metastasis (M0). The recommended treatment pathway involves neoadjuvant chemoradiotherapy (nCRT), followed by surgery and adjuvant chemotherapy [11]. Pathological assessment remains the gold standard for evaluating pathological response to nCRT. Patients achieving a pathological complete response (pCR) or a sustained clinical complete response (cCR) may be candidates for organ-preserving strategies and, in selected settings, may avoid radical surgery. However, precise restaging and identification of cCR can be constrained by the limitations of imaging acquisition technology [12]. This underscores the need to identify suitable indicators for constructing nCRT efficacy prediction models. In this context, “efficacy prediction” specifically refers to forecasting the pathological response to nCRT. Pretreatment clinical features are objective, readily available and inexpensive, making them reliable candidates for the construction of LARC nCRT efficacy prediction models and potentially offsetting challenges associated with obtaining pathological and imaging information. Based on the overarching challenge of selecting appropriate machine learning models in clinical research, we hypothesized that a structured, six-stage pipeline—incorporating pre-specified benchmarks for both predictive performance and explainability—would effectively identify models that optimally balance these criteria for a given task. Furthermore, we postulated that, when applied to the specific case of predicting neoadjuvant chemoradiotherapy efficacy in locally advanced rectal cancer, this pipeline would not only identify highly interpretable models, but also highlight clinically plausible pretreatment predictors, notably tumor markers, thereby yielding insights that are consistent with established biological and clinical evidence. Related works and our contributions to this field Related works This section examines application trends of ML in clinical research. Much existing medical research focuses on model construction using genomics, radiomics, and pathomics [13, 14]. However, the extraction and quantification of histological features from imaging and pathological images requires Computer-Aided Diagnosis (CAD) techniques. This process demands specialized expertise and substantial human resources, rendering it costly and time-consuming. Crucially, features learned by ML models from image data are often abstract representations, involving complex relationships (e.g., spatial relationships between pixels, interactions between features) that are not directly comprehensible to humans, leading to their characterization as “black-box” models. Given the significant impact of medical decisions on patient health, the requirement for the explainability of ML models in clinical applications is considerably higher than in other domains. Our contributions Given the broad application of ML in oncology research [15, 16], we used rectal cancer as a clinically relevant case study to demonstrate the proposed pipeline. We curated a single-center retrospective cohort of 128 consecutive LARC patients with complete pretreatment clinical variables available in the electronic medical record system and long-term follow-up. This cohort was assembled specifically to support a methodological demonstration of model selection under real-world clinical constraints in small-sample settings, rather than to serve as a large-scale reference dataset. We selected four ML algorithms commonly employed in tumor research for analysis, including support vector machine (SVM) [17–19], decision tree (DT) [20, 21], random forest (RF) [22–24] and logistic regression (LR) [22, 25]. The dataset was divided into training and testing sets. The training set was used for model development, and the testing set for evaluation. High-performed models underwent further analysis. Nevertheless, a clinically viable model must demonstrate not only promising performance but also satisfactory explainability of its predictions [26]. Thus, in addition to accuracy screening, models must undergo explainability screening. To enhance model explainability, we utilized SHAP (SHapley Additive exPlanations) [27], which is based on the Shapley value concept from game theory. SHAP provides a unified approach to explaining the output of any ML model, offering both global and local interpretations. Additionally, consensus on explainability standards among experts, regulatory agencies, and clinicians is lacking [28]. To address this gap, this study defines minimum explainability standard, thereby supporting doctors in explaining predictions and patients in trusting the models. Furthermore, we include an internal clinical logic consistency check (non-deployable sanity check) to examine whether model behavior is directionally consistent with known relationships between correlated pathological endpoints (pCR and TRG). This check is reported explicitly as non-deployable and is not used to claim predictive generalizability. Overall, this work demonstrates the feasibility of a structured selection workflow that jointly considers predictive performance and interpretability in a small, single-center clinical dataset. The selected models achieved modest yet clinically interpretable performance, and the explainability analysis highlighted tumor markers that are consistent with prior evidence. We emphasize that the primary contribution is methodological: providing a reproducible pipeline that can be extended and externally validated in future multi-center cohorts. Importantly, the contribution of this study is primarily methodological rather than algorithmic. Unlike AutoML frameworks, which are typically designed to maximize predictive performance through automated search and tuning, our pipeline explicitly embeds clinically oriented screening for both performance and explainability under small-sample constraints. Unlike post hoc SHAP-based interpretation alone, the proposed workflow treats explainability as a formal selection criterion rather than a descriptive add-on after model choice. Therefore, the value of this study lies in a transparent, clinician-oriented model selection framework that is conceptually simple, methodologically reproducible, and practically applicable to small routine clinical datasets. The overall study workflow is summarized in Fig. 1. Workflow diagram SVM support vector machine; DT decision tree; RF random forest; LR logistic regression; SHAP SHapley Additive exPlanations; ML machine learning; the final step represents a non-deployable internal consistency check rather than predictive validation Materials This study utilized a retrospectively collected clinical dataset of patients with LARC from the First Affiliated Hospital of Soochow University. The data were extracted from the hospital’s electronic medical record system and were not prospectively managed for the purpose of this analysis. This was a single-institution study and it was approved by the Ethics Committee of the First Affiliated Hospital of Soochow University (Approval number: NO. 2023–559). All procedures involving human participants were conducted in accordance with the ethical standards of the institutional research committee and with the Declaration of Helsinki (as revised in 2013). Informed consent was waived due to the retrospective design of the study. Population studied Inclusion criteria: Rectal cancer diagnosed by pathological examination; Rectal cancer diagnosed by pathological examination; Clinical stage T3/4 or N+, M0; Clinical stage T3/4 or N+, M0; Chemotherapy plus long-term neoadjuvant radiotherapy and total mesorectal excision (TME); Chemotherapy plus long-term neoadjuvant radiotherapy and total mesorectal excision (TME); Eastern Cooperative Oncology Group (ECOG) performance status of 0-2. Eastern Cooperative Oncology Group (ECOG) performance status of 0-2. Exclusion criteria: Short-course neoadjuvant radiotherapy; Short-course neoadjuvant radiotherapy; Missing preoperative clinical indicators or missing pathological results after nCRT treatment; Missing preoperative clinical indicators or missing pathological results after nCRT treatment; No surgical resection or receipt of palliative treatment only; No surgical resection or receipt of palliative treatment only; History of other primary tumors within 5 years. History of other primary tumors within 5 years. The patient screening process, including exclusion reasons and final cohort allocation into training and testing sets. Feature selection clinical indicators were collected from the hospital medical record system and categorized into 4 categories: General indicators: sex, age, body mass index (BMI), maximum tumor diameter, distance from the tumor to the anal verge, tumor circumference proportion, clinical T stage, clinical N stage, clinical TNM stage. (Endoscopic findings were interpreted by two experienced gastroenterologists. Clinical staging was determined by radiologists). General indicators: sex, age, body mass index (BMI), maximum tumor diameter, distance from the tumor to the anal verge, tumor circumference proportion, clinical T stage, clinical N stage, clinical TNM stage. (Endoscopic findings were interpreted by two experienced gastroenterologists. Clinical staging was determined by radiologists). Blood routine indicators: counts of white blood cells (WBC), neutrophils (NEUT), platelets (PLT), lymphocytes (LYM), and monocytes (MONO). Derived ratios: neutrophil-to-lymphocyte ratio (NLR = NEUT/LYM), platelet-to-lymphocyte ratio (PLR = PLT/LYM), lymphocyte-to-monocyte ratio (LMR = LYM/MONO), systemic immune-inflammation index (SII = PLT × NEUT/LYM), pan-immune-inflammation value (PIV = NEUT × PLT × MONO/LYM). Blood routine indicators: counts of white blood cells (WBC), neutrophils (NEUT), platelets (PLT), lymphocytes (LYM), and monocytes (MONO). Derived ratios: neutrophil-to-lymphocyte ratio (NLR = NEUT/LYM), platelet-to-lymphocyte ratio (PLR = PLT/LYM), lymphocyte-to-monocyte ratio (LMR = LYM/MONO), systemic immune-inflammation index (SII = PLT × NEUT/LYM), pan-immune-inflammation value (PIV = NEUT × PLT × MONO/LYM). Tumor markers: Carcinoembryonic antigen (CEA), carbohydrate antigen 19–9 (CA19-9), CA125, CA153, CA724. Tumor markers: Carcinoembryonic antigen (CEA), carbohydrate antigen 19–9 (CA19-9), CA125, CA153, CA724. Treatment-related indicators: chemotherapy regimen, number of chemotherapy cycles, time interval from nCRT completion to surgery (operation interval), and surgical procedure. Treatment-related indicators: chemotherapy regimen, number of chemotherapy cycles, time interval from nCRT completion to surgery (operation interval), and surgical procedure. Treatment In this paper, patients with LARC received long-course radiotherapy and chemotherapy. According to the radiation therapy oncology group, the neoadjuvant long-course radiotherapy was 45.0–50.4 Gy in 25–28 fractions (1.8–2.0 Gy per fraction), delivered 5 times per week for 5–5.6 weeks. Neoadjuvant chemotherapy regimens included: (1) XELOX: Oxaliplatin 130 mg/m2 on day 1, plus capecitabine 1000 mg/m2 orally twice daily on days 1–14, repeated every 3 weeks; (2) Capecitabine monotherapy (825 mg/m2 orally twice daily on days of radiotherapy). After neoadjuvant therapy, the surgical procedures included low anterior resection (LAR) and abdominoperineal resection (APR). A small number of patients underwent intersphincteric resection (ISR), perineal coloanal pull-through procedures, and proctosigmoidectomy with end colostomy (Hartmann’s procedure). Efficacy evaluation Tumor regression after treatment was evaluated using pathological complete response (pCR) and tumor regression grade (TRG) respectively. pCR was defined as the absence of viable tumor cells in the surgical specimen upon pathological examination after nCRT. TRG was graded according to the American Joint Committee on Cancer (AJCC) 9th edition criteria: TRG 0: Complete regression, with no visible tumor cells under the microscope. TRG 1: Near complete regression, with only single or small clusters of tumor cells visible under the microscope. TRG 2: Partial regression, with significant regression but more residual tumor than single or small clusters of tumor cells. TRG 3: Poor or no regression, with a large amount of residual tumor cells and no significant regression. TRG and pCR were independently assessed by two experienced pathologists (each with >10 years of experience). Discrepancies were resolved by consensus. For analysis, TRG 0–1 were categorized as good tumor regression, and TRG 2–3 as poor tumor regression. Statistical analysis Statistical analysis was performed using SPSS 26.0 and Python 3.10. Continuous variables in this paper did not conform to normal distributions and were expressed as median (quartile) [M (Q1, Q3)], using the Mann-Whitney U test. Categorical variables were expressed as frequency (percentage) and analyzed using the χ2 test and Fisher’s exact test, as appropriate. p < 0.05 was considered statistically significant. Methods Data selection and pre-processing This step includes initial data inspection, definition of prediction targets, and preprocessing to ensure data integrity and usability. We screened candidate variables for completeness and plausibility. Patients with missing key pretreatment indicators or missing pathological outcomes after nCRT were excluded according to prespecified criteria, resulting in a final modeling dataset with no missing values; therefore, no imputation was performed. Outliers were screened using clinical range checks and distribution-based inspection. Values considered likely to represent data-entry errors were verified against source records when possible, while biologically plausible extreme values were retained. Continuous variables were used in their original scale unless discretization was clinically justified for interpretability, and categorical variables were encoded numerically for modeling. The final cohort comprised 128 consecutive eligible patients treated between September 2013 and November 2022. This sample size reflects all available patients meeting the inclusion criteria during the study period at our institution, which is common in retrospective biomarker and clinical prediction model studies. We used a stratified hold-out split to preserve class proportions while retaining sufficient data for model development: 70% of the data (n = 89) were assigned to the training set and 30% (n = 39) to an independent testing set. Given the limited sample size, we further complemented the hold-out evaluation with five-fold cross-validation and bootstrap-based confidence intervals to assess robustness and uncertainty of model performance. [29]. Algorithm pool construction An algorithm pool for selecting algorithms was constructed based on the following criteria: Compatibility with the dataset characteristics. Compatibility with the dataset characteristics. Ability to complete the process from data input to prediction output. Ability to complete the process from data input to prediction output. Trainability based on existing algorithms. Trainability based on existing algorithms. We selected SVM, DT, RF, and LR to construct LARC efficacy prediction models. These four algorithms are widely used and validated in clinical research for tasks like tumor diagnosis and efficacy prediction [30–33]. However, they have potential drawbacks with small samples, such as limited explainability or assumptions about linearity, necessitating further screening. Model training and model evaluation Models were trained using the selected algorithms on the training set to learn the relationship between features and the target variable, then tested on the testing set. The four ML algorithms were implemented using the Scikit-learn library in Python. In alignment with the primary objective of this study—to validate a model selection pipeline rather than to exhaustively optimize any single model—all models were constructed using their default hyperparameters. This provides a consistent baseline that enables a transparent evaluation of the fundamental trade-offs between performance and explainability across different algorithm families, free from the bias introduced by algorithm-specific tuning. The pipeline aims to filter out models with a high probability of misclassification, as both false positive and false negative predictions carry significant clinical risks. Using Scikit-learn’s confusion matrix, we derived True Positives (TP), False Positives (FP), True Negatives (TN), and False Negatives (FN). Based on these, we calculated: Accuracy = (TP + TN)/ (TP + FP + TN + FN) Accuracy = (TP + TN)/ (TP + FP + TN + FN) Precision = TP / (TP + FP) Precision = TP / (TP + FP) Recall (Sensitivity) = TP / (TP + FN) Recall (Sensitivity) = TP / (TP + FN) F1-score = 2 × (Precision × Recall) / (Precision + Recall) F1-score = 2 × (Precision × Recall) / (Precision + Recall) In addition to accuracy and F1-score, we reported AUROC and AUPRC as complementary discrimination metrics, particularly given class imbalance in pCR prediction. AUROC and AUPRC were reviewed alongside the primary screening metrics when summarizing model performance and stability. Increased Recall reduces false negatives, and increased Precision reduces false positives; both are crucial for LARC efficacy prediction. The F1-score, the harmonic mean of Precision and Recall, provides a balanced assessment, especially for imbalanced data. A high F1-score indicates robust performance across both positive and negative classes with fewer misclassifications. To ensure robust evaluation given the dataset size, we implemented rigorous stability assessments. For all performance metrics, we performed bootstrap resampling with 1000 iterations to calculate 95% confidence intervals (95%CI) providing uncertainty estimates for model performance. The lower threshold of 0.60 for both accuracy and F1-score was used as a pragmatic internal screening threshold within this workflow, rather than as a universal benchmark for clinical deployment. These performance levels were intended to facilitate transparent model filtering in the current methodological study and should not be interpreted as externally validated clinical standards. In the context of LARC efficacy prediction, models with F1-scores below 0.5 exhibit near-random behavior, while values between 0.6 and 0.7 generally indicate acceptable discrimination performance in clinical prediction studies [34]. Therefore, 0.6 was used as a practical screening cutoff to exclude underperforming models while preserving sufficient model diversity for subsequent stability and explainability analyses. Models were evaluated based on both Accuracy and F1-score. Model performance was stratified into three levels based on metric values, with higher levels indicating greater acceptability: Performance Level 1 (0.60 ≤ value < 0.70) Performance Level 1 (0.60 ≤ value < 0.70) Performance Level 2 (0.70 ≤ value < 0.80) Performance Level 2 (0.70 ≤ value < 0.80) Performance Level 3 (0.80 ≤ value ≤ 1.00) Performance Level 3 (0.80 ≤ value ≤ 1.00) Models achieving higher scores and at least Performance Level 1 for both metrics proceeded to explainability analysis. Additionally, we conducted five-fold cross-validation analysis on qualified models to evaluate stability across different data partitions, reporting mean accuracy with standard deviation (SD) and coefficient of variation (CV = SD/mean × 100%). According to previous studies [35, 36], models with CV < 10% were considered to have acceptable stability, while CV < 5% indicated high stability. Model selection within the proposed pipeline was therefore based on a combination of predefined performance screening (accuracy and F1-score), complementary discrimination assessment (AUROC and AUPRC), stability analysis, and explainability screening, rather than on hyperparameter-optimized peak performance. Explainability Explainability refers to the extent to which humans can understand and interpret an ML model’s output. We defined five levels of explainability, where a model must reach at least Level 3 to be considered understandable: Explainability Level 1: Results cannot be understood. Explainability Level 1: Results cannot be understood. Explainability Level 2: Results can be understood by machines. Explainability Level 2: Results can be understood by machines. Explainability Level 3: Results can be understood by data scientists. Explainability Level 3: Results can be understood by data scientists. Explainability Level 4: Results can be understood by doctors. Explainability Level 4: Results can be understood by doctors. Explainability Level 5: Results can be understood by patients. Explainability Level 5: Results can be understood by patients. SHAP, based on Shapley values from game theory, provides a model-agnostic explanation method understandable by data scientists (Level 3). We used SHAP to visualize feature importance, making it easier for doctors to understand the model (Level 4) and explain it to patients (Level 5), thereby building patient trust in the model and avoiding potential misunderstandings. The empirical application and validation of this framework, demonstrating how explanations for our selected model traverse these levels, are presented in the Results section. Furthermore, SHAP allows clinicians to compare model-identified important features with clinical experience, assessing alignment with clinical logic and judging experimental design and feature selection reasonableness. For instance, if an expensive, rarely available test is identified as most important, its inclusion might limit model usability, suggesting the need for alternative variables. In practical terms, Level 3 explainability in this study corresponds to the ability of technically trained readers to understand SHAP-based feature rankings and directional contributions. Level 4 is represented by the ability of clinicians to interpret whether the model relies on medically plausible predictors, such as lower CEA and CA19-9 levels being associated with favorable response. Level 5 does not imply that patients directly interpret SHAP plots themselves; rather, it refers to the clinician’s ability to translate the model rationale into simplified patient-facing explanations, for example by stating that the prediction was influenced mainly by routinely measured tumor markers and other standard pretreatment features, rather than by obscure or inaccessible variables. Internal clinical logic consistency check (non-deployable sanity check) Models that passed the performance screening (Accuracy and F1-score ≥ 0.60, Performance Level 1) and explainability screening (Explainability ≥ Level 3) underwent an additional internal consistency analysis. This analysis was designed only as a non-deployable sanity check, rather than a validation of predictive generalizability. Because pCR and TRG are correlated pathological endpoints, we performed an exploratory association check by introducing one endpoint as an additional feature when predicting the other. As expected, performance metrics increased and the added endpoint became highly important in SHAP analysis. These findings support endpoint concordance and internal logic consistency, but must not be interpreted as real-world predictive performance, since the added endpoint is outcome-related information and unavailable at pretreatment prediction time. Therefore, results of this analysis are reported explicitly as a sanity check and are not used to recommend a deployable clinical workflow. Data availability The dataset analyzed in this study was retrospectively collected from the electronic medical record system of the First Affiliated Hospital of Soochow University. Owing to institutional and ethical restrictions related to patient privacy, the raw data are not publicly deposited in an open repository. De-identified data may be made available from the corresponding author upon reasonable request and subject to approval by the institutional ethics committee. Results Baseline data of patients The study included a total of 128 patients with LARC who received nCRT and surgery. Patients were stratified by pCR status and TRG group (0–1 vs. 2–3). The training set (n = 89) included 31 patients with pCR and 41 with TRG 0–1, while the testing set (n = 39) included 11 patients with pCR and 13 with TRG 0–1. Clinical indicators of patients in different categories are shown in Table 1. Table 1Baseline characteristics of patients with locally advanced rectal cancerFactorAllpCRNon-pCRX2/Z valuePvalueTRG 0–1TRG 2–3X2/Z valuePvalueGender0.6450.4220.2030.652Male97 (75.8)30674255Female31 (24.2)12191219Age (year)59.00(48.25, 67.00)59.50(51.00, 69.00)59.00(48.00, 66.00)−0.8860.37657.50(48.75, 67.00)60.00(48.00, 67.00)−0.0360.971BMI (kg/m2)23.78(21.30, 25.78)23.31(21.48, 24.80)23.28(20.48, 26.20)−0.3040.76123.59(21.63, 25.29)23.17(20.45, 26.26)−0.1910.849cT stage0.5750.4481.5880.208cT3105 (82.0)36694758cT423 (18.0)617716cN stage0.0001.0002.2700.132cN011 (8.6)4774cN+117 (91.4)38794770TNM0.0001.0001.4100.235II11 (8.6)4774III117 (91.4)38794770Distance from the anal verge to rectal cancer (cm)5.00(3.80, 6.60)5.05(3.60, 6.50)5.00(4.00, 6.60)−0.0330.9745.00(3.58, 6.50)5.00(3.95, 6.78)−0.3930.695Tumor size (cm)5.00(3.50, 5.86)4.95(3.00, 5.50)5.00(3.70, 6.00)−1.1200.2634.95(3.00, 5.50)5.00(3.65, 6.00)−1.1740.240Tumor circumference proportion0.3510.5541.5350.215≤1/218 (14.1)711108>1/2110 (85.9)35754466WBC (109/L)6.35(5.14, 7.80)6.41(5.05, 7.83)6.33(5.16, 7.77)−0.2970.7676.41(5.03, 7.84)6.33(5.17, 7.78)−0.2770.781NEUT (109/L)3.92(3.04, 5.07)3.94(2.96, 5.24)3.92(3.08, 4.93)−0.0280.9783.82(2.95, 5.22)3.95(3.07, 4.95)−0.0630.95PLT (109/L)230.50(193.00, 283.00)222.00(191.75, 269.50)237.50(194.50, 292.50)−1.0960.273220.50(193.00, 269.50)241.00(191.50, 292.50)−1.0540.292LYM (109/L)1.66(1.38, 2.14)1.65(1.21, 2.04)1.67(1.45, 2.17)−0.8300.4071.65(1.28, 2.04)1.68(1.45, 2.18)−0.6970.486MONO (109/L)0.42(0.36, 0.56)0.40(0.36, 0.57)0.45(0.36, 0.57)−0.5920.1110.40(0.36, 0.54)0.45(0.36, 0.58)−1.0230.776NLR2.31(1.71, 3.18)2.25(1.67, 3.32)2.34(1.72, 3.06)−0.4620.6442.23(1.72, 3.34)2.34(1.68, 3.02)−0.2850.860PLR136.00(108.79, 178.89)142.79(108.49, 182.56)133.95(108.54, 175.55)−0.0330.974136.06(107.42, 180.07)135.35(109.34, 178.33)−0.1760.860LMR3.82(2.85, 4.74)3.81(2.90, 4.68)3.83(2.82, 4.78)−0.4470.6553.79(2.93, 4.54)3.88(2.76, 4.86)−0.1040.917PIV243.11(142.77, 405.12)200.66(140.77, 329.13)269.07(142.70, 446.49)−1.1420.254202.05(140.77, 343.54)275.62(142.70, 446.49)−0.9360.349SII528.62(381.12, 866.75)528.19(375.66, 870.43)530.97(382.97, 872.11)−0.0610.951515.22(368.16, 870.43)536.82(389.02, 872.11)−0.2070.836CEA (ng/mL)3.81(2.11,10.99)2.69(1.88, 5.86)5.01(2.28, 13.38)−3.0170.0032.69(1.91, 6.32)5.10(2.97, 14.11)−3.3170.001CA125 (U/mL)9.55(7.15, 13.70)9.55(7.45, 13.75)9.65(7.00, 13.73)−0.0690.9459.55(7.38, 13.25)9.60(7.00, 13.85)−0.1380.891CA153 (U/mL)7.55(5.50, 11.78)7.35(5.35, 11.75)7.70(5.50, 11.80)−0.4310.6667.00(5.28, 11.75)7.75(5.65,11.80)−0.7460.456CA19-9 (U/mL)8.51(3.65, 23.30)7.54(5.48, 20.02)9.25(2.57, 24.72)−0.1400.8898.57(5.48, 23.68)8.11(2.49, 21.21)−1.0540.292CA724 (U/mL)1.75(0.91, 2.81)1.84(0.89, 2.91)1.72(0.91, 2.70)−0.1830.8551.84(0.85, 2.94)1.72(0.94, 2.64)−0.1690.866Chemotherapy2.3130.3230.8110.675XELOX88 (68.8)29593751Capecitabine monotherapy7 (5.5)4343Others33 (25.8)9241320Number of cycles of Chemotherapy1.4470.2291.1460.284≤3105 (82.0)327342634 ~ 623 (18.0)10131211Operation interval (d)56.00(48.00, 68.00)58.00(52.75, 87.50)55.00(46.75, 66.00)−2.0490.04056.50(52.00, 74.50)55.50(47.00, 67.00)−1.2940.196Surgery1.1820.5543.6040.162LAR87 (67.9)28593552APR33 (25.7)10231320others8 (6.2)4462pCR pathological complete response; TRG tumor regression grade; BMI body mass index; WBC white blood cells; NEUT neutrophils; PLT platelets; LYM lymphocytes; MONO monocytes; NLR neutrophil-to-lymphocyte ratio; PLR platelet-to-lymphocyte ratio; LMR lymphocyte-to-monocyte ratio; SII systemic immune-inflammation index; PIV pan-immune-inflammation value; CEA Carcinoembryonic antigen; CA carbohydrate antigen; LAR low anterior resection; APR abdominoperineal resection Baseline characteristics of patients with locally advanced rectal cancer 59.00 (48.25, 67.00) 59.50 (51.00, 69.00) 59.00 (48.00, 66.00) 57.50 (48.75, 67.00) 60.00 (48.00, 67.00) 23.78 (21.30, 25.78) 23.31 (21.48, 24.80) 23.28 (20.48, 26.20) 23.59 (21.63, 25.29) 23.17 (20.45, 26.26) 5.00 (3.80, 6.60) 5.05 (3.60, 6.50) 5.00 (4.00, 6.60) 5.00 (3.58, 6.50) 5.00 (3.95, 6.78) 5.00 (3.50, 5.86) 4.95 (3.00, 5.50) 5.00 (3.70, 6.00) 4.95 (3.00, 5.50) 5.00 (3.65, 6.00) 6.35 (5.14, 7.80) 6.41 (5.05, 7.83) 6.33 (5.16, 7.77) 6.41 (5.03, 7.84) 6.33 (5.17, 7.78) 3.92 (3.04, 5.07) 3.94 (2.96, 5.24) 3.92 (3.08, 4.93) 3.82 (2.95, 5.22) 3.95 (3.07, 4.95) 230.50 (193.00, 283.00) 222.00 (191.75, 269.50) 237.50 (194.50, 292.50) 220.50 (193.00, 269.50) 241.00 (191.50, 292.50) 1.66 (1.38, 2.14) 1.65 (1.21, 2.04) 1.67 (1.45, 2.17) 1.65 (1.28, 2.04) 1.68 (1.45, 2.18) 0.42 (0.36, 0.56) 0.40 (0.36, 0.57) 0.45 (0.36, 0.57) 0.40 (0.36, 0.54) 0.45 (0.36, 0.58) 2.31 (1.71, 3.18) 2.25 (1.67, 3.32) 2.34 (1.72, 3.06) 2.23 (1.72, 3.34) 2.34 (1.68, 3.02) 136.00 (108.79, 178.89) 142.79 (108.49, 182.56) 133.95 (108.54, 175.55) 136.06 (107.42, 180.07) 135.35 (109.34, 178.33) 3.82 (2.85, 4.74) 3.81 (2.90, 4.68) 3.83 (2.82, 4.78) 3.79 (2.93, 4.54) 3.88 (2.76, 4.86) 243.11 (142.77, 405.12) 200.66 (140.77, 329.13) 269.07 (142.70, 446.49) 202.05 (140.77, 343.54) 275.62 (142.70, 446.49) 528.62 (381.12, 866.75) 528.19 (375.66, 870.43) 530.97 (382.97, 872.11) 515.22 (368.16, 870.43) 536.82 (389.02, 872.11) 3.81 (2.11,10.99) 2.69 (1.88, 5.86) 5.01 (2.28, 13.38) 2.69 (1.91, 6.32) 5.10 (2.97, 14.11) 9.55 (7.15, 13.70) 9.55 (7.45, 13.75) 9.65 (7.00, 13.73) 9.55 (7.38, 13.25) 9.60 (7.00, 13.85) 7.55 (5.50, 11.78) 7.35 (5.35, 11.75) 7.70 (5.50, 11.80) 7.00 (5.28, 11.75) 7.75 (5.65,11.80) 8.51 (3.65, 23.30) 7.54 (5.48, 20.02) 9.25 (2.57, 24.72) 8.57 (5.48, 23.68) 8.11 (2.49, 21.21) 1.75 (0.91, 2.81) 1.84 (0.89, 2.91) 1.72 (0.91, 2.70) 1.84 (0.85, 2.94) 1.72 (0.94, 2.64) 56.00 (48.00, 68.00) 58.00 (52.75, 87.50) 55.00 (46.75, 66.00) 56.50 (52.00, 74.50) 55.50 (47.00, 67.00) pCR pathological complete response; TRG tumor regression grade; BMI body mass index; WBC white blood cells; NEUT neutrophils; PLT platelets; LYM lymphocytes; MONO monocytes; NLR neutrophil-to-lymphocyte ratio; PLR platelet-to-lymphocyte ratio; LMR lymphocyte-to-monocyte ratio; SII systemic immune-inflammation index; PIV pan-immune-inflammation value; CEA Carcinoembryonic antigen; CA carbohydrate antigen; LAR low anterior resection; APR abdominoperineal resection Comparison between the non-pCR group and pCR groups revealed significantly lower pre-treatment CEA levels and a longer operation interval in the pCR group. Patients with TRG 0–1 exhibited lower pre-treatment CEA levels than those with TRG 2–3 (p < 0.05). Experimental results based on all or part of the clinical features All clinical indicators were initially included. The confusion matrices are shown in Fig. 2, and model performance metrics are detailed in Table 2. For pCR prediction, the SVM and RF models achieved accuracy > 0.70. As illustrated in Fig. 2A and B, these models predominantly predicted negative outcomes (38/38 and 37/38, respectively), resulting in F1-scores below 0.60 (failing Performance Level 1). Consequently, these pCR prediction models using all features were excluded from further explainability screening. For TRG prediction, the DT and RF models showed comparable accuracy, but only the DT model’s F1-score met Performance Level 1 (0.63), establishing it as the most suitable TRG prediction model among those using all features. Confusion matrix. (A) The SVM model using all clinical indicators predicting pCR; (B) The RF model using all clinical indicators predicting pCR; (C) The DT model using all clinical indicators predicting TRG; (D) The RF model using all clinical indicators predicting TRG; (E) The DT model using only tumor markers predicting pCR; (F) The DT model using only tumor markers predicting TRG. pCR, pathological complete response; TRG, tumor regression grade; SVM, support vector machine; DT, decision tree; RF, random forest Table 2Predictive efficacy of four machine learning classifiersPrecisionRecallF1-scoreAccuracy [95% CI]pCRSVM0.360.500.420.71 [0.55,0.84]DT0.590.600.590.66 [0.50,0.79]RF0.880.590.580.76 [0.63,0.89]LR0.330.370.370.58 [0.42,0.74]TRGSVM0.360.500.420.71 [0.55,0.84]DT0.640.630.630.68 [0.52,0.79]RF0.650.580.560.69 [0.53,0.82]LR0.330.410.370.58 [0.42,0.74]pCR pathological complete response; TRG tumor regression grade; SVM support vector machine; DT decision tree; RF random forest; LR logistic regression Predictive efficacy of four machine learning classifiers pCR pathological complete response; TRG tumor regression grade; SVM support vector machine; DT decision tree; RF random forest; LR logistic regression In clinical practice, patients often transfer care between hospitals, potentially leading to incomplete records. To address this, we grouped indicators that are either readily synchronisable or susceptible to loss into four categories (General indicators, Blood routine indicators, Tumor markers, and evaluated models using single indicator groups or combinations thereof (15 scenarios in total). Performance across these scenarios is shown in Table 3. Table 3Predictive efficacy of four ML classifiers including different clinical featuresIndicatorMLModelGeneral data+++++++Blood routine+++++++Tumor markers+++++++Treatment information+++++++pCRSVMPrecision0.360.360.360.360.360.350.360.360.360.360.360.360.360.36Recall0.500.500.500.500.500.500.500.500.500.480.500.500.500.50F1-score0.420.420.420.420.420.420.420.420.420.420.420.420.420.42Accuracy [95% CI]0.71 [0.58,0.84]0.71 [0.58,0.87]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.58,0.84]0.71 [0.58,0.84]0.71 [0.55,0.84]0.71 [0.58,0.87]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.55,0.84]DTPrecision0.640.560.900.680.560.790.590.710.630.910.610.560.900.71Recall0.630.540.680.600.540.660.560.670.610.730.600.540.680.67F1-score0.630.530.710.600.530.680.560.680.610.760.600.530.710.68Accuracy [95% CI]0.68 [0.53,0.84]0.68 [0.53,0.84]0.82 [0.68,0.92]0.74 [0.60,0.87]0.68 [0.53,0.82]0.79 [0.66,0.92]0.68 [0.52,0.82]0.76 [0.63,0.89]0.71 [0.55,0.84]0.84 [0.71,0.95]0.68 [0.55,0.82]0.68 [0.53,0.82]0.82 [0.68,0.92]0.76 [0.63,0.92]RFPrecision0.520.700.580.760.610.700.610.360.860.860.610.610.860.86Recall0.510.570.590.620.530.570.530.500.550.550.530.530.550.55F1-score0.470.560.580.630.490.560.490.420.510.510.490.490.510.51Accuracy [95% CI]0.68 [0.56,0.84]0.74 [0.61,0.87]0.76 [0.63,0.89]0.76 [0.63,0.89]0.71 [0.55,0.87]0.74 [0.61,0.87]0.71 [0.58,0.84]0.71 [0.58,0.84]0.74 [0.60,0.87]0.74 [0.58,0.87]0.71 [0.55,0.84]0.71 [0.55,0.84]0.74 [0.61,0.87]0.74 [0.58,0.87]LRPrecision0.330.350.620.360.350.530.410.680.700.760.760.480.480.63Recall0.410.480.550.500.470.530.430.600.570.630.620.490.480.58F1-score0.370.410.540.420.450.530.420.600.560.760.630.460.470.58Accuracy [95% CI]0.58 [0.42,0.74]0.68 [0.53,0.82]0.71 [0.58,0.84]0.58 [0.55,0.87]0.63 [0.47,0.80]0.63 [0.47,0.79]0.58 [0.42,0.74]0.74 [0.58,0.87]0.74 [0.65,0.89]0.76 [0.63,0.89]0.76 [0.63,0.89]0.66 [0.50,0.82]0.61 [0.46,0.79]0.71 [0.55,0.84]TRGSVMPrecision0.360.360.360.360.360.530.360.360.360.350.360.360.360.36Recall0.500.500.500.500.500.520.500.500.500.480.500.500.500.50F1-score0.420.420.420.420.420.490.420.420.420.420.420.420.420.42Accuracy [95% CI]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.55,0.84]0.63 [0.47,0.79]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.58,0.87]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.55,0.84]0.71 [0.55,0.84]DTPrecision0.590.590.740.590.600.740.590.680.590.740.640.590.680.62Recall0.600.600.710.600.610.730.600.650.600.710.630.600.690.61F1-score0.590.590.720.590.570.730.590.660.590.720.630.590.680.61Accuracy [95% CI]0.66 [0.53,0.79]0.66 [0.50,0.82]0.76 [0.63,0.89]0.66 [0.50,0.79]0.58 [0.42,0.74]0.76 [0.63,0.87]0.66 [0.50,0.82]0.71 [0.60,0.89]0.66 [0.50,0.82]0.76 [0.61,0.89]0.68 [0.53,0.82]0.66 [0.50,0.82]0.71 [0.55,0.84]0.76 [0.53,0.79]RFPrecision0.610.520.520.360.650.860.610.740.650.620.350.360.540.61Recall0.530.510.510.500.580.550.530.710.580.610.480.500.530.59F1-score0.490.470.470.420.560.510.490.760.560.610.410.420.530.59Accuracy [95% CI]0.71 [0.55,0.87]0.68 [0.53,0.84]0.68 [0.55,0.84]0.71 [0.55,0.84]0.68 [0.52,0.84]0.74 [0.61,0.87]0.71 [0.55,0.84]0.72 [0.63,0.89]0.68 [0.53,0.82]0.66 [0.50,0.79]0.68 [0.53,0.84]0.71 [0.58,0.84]0.61 [0.45,0.76]0.66 [0.50,0.82]LRPrecision0.330.520.620.370.400.510.470.590.430.330.510.470.400.60Recall0.410.520.630.480.420.510.480.590.440.410.510.470.390.61F1-score0.370.510.610.370.410.510.460.590.430.370.510.470.390.60Accuracy [95% CI]0.58 [0.42,0.71]0.61 [0.45,0.78]0.63 [0.47,0.79]0.45 [0.29,0.61]0.50 [0.34,0.66]0.55 [0.39,0.71]0.58 [0.45,0.78]0.63 [0.50,0.79]0.50 [0.39,0.63]0.58 [0.42,0.74]0.55 [0.44,0.70]0.50 [0.34,0.66]0.42 [0.26,0.58]0.63 [0.49,0.80]pCR pathological complete response; TRG tumor regression grade; SVM support vector machine; DT decision tree; RF random forest; LR logistic regression; Boldface indicates models that satisfied the predefined screening criteria and were retained for subsequent explainability and stability analyses Predictive efficacy of four ML classifiers including different clinical features pCR pathological complete response; TRG tumor regression grade; SVM support vector machine; DT decision tree; RF random forest; LR logistic regression; Boldface indicates models that satisfied the predefined screening criteria and were retained for subsequent explainability and stability analyses The DT model exhibited superior capability in predicting both pCR and TRG across various feature subsets. Not only did its precision sometimes reach Performance Level 3, but its F1-score also consistently met at least Level 2 in the best scenarios. In contrast, despite the high accuracy of SVM, RF, and LR models in some settings, their F1-scores often fell below Performance Level 1, corroborating the viewpoint put forth that a narrow focus on a single indicator, such as accuracy, can be misleading. Notably, models using only tumor markers often performed well, sometimes outperforming models with additional variables. The confusion matrices for DT models using only tumor markers are shown in Fig. 2E and F. Using tumor markers alone, the DT model achieved an accuracy of 0.82 and F1-score of 0.71 for pCR prediction, and an accuracy of 0.76 and F1-score of 0.72 for TRG prediction. To further assess the robustness of the selected models, five-fold cross-validation was performed. The TRG-DT model achieved a mean accuracy of 0.64 ± 0.06 (CV = 9.3%), with AUROC = 0.64 and AUPRC = 0.68; the pCR-DT’ model achieved 0.65 ± 0.03 (CV = 4.6%), AUROC = 0.71, and AUPRC = 0.61; and the TRG-DT’ model achieved 0.68 ± 0.03 (CV = 4.4%), AUROC = 0.64, and AUPRC = 0.59 (Table 4). All remaining models’ performance metrics are provided in the Supplementary Tables. Specifically, five-fold cross-validation stability results for pCR and TRG prediction models are shown in Supplementary Tables S1 and S2, respectively, whereas AUROC and AUPRC results for pCR and TRG prediction models are shown in Supplementary Tables S3 and S4, respectively. These results indicate that the selected models not only achieved satisfactory predictive performance but also demonstrated acceptable to high stability across different data partitions. Table 4Predictive efficacy of ML classifiersPrecisionRecallF1-scoreAccuracy [95% CI]Mean accuracy ± SDCV (%)AUROCAUPRCTRG-DT Model0.640.630.630.68 [0.52,0.79]0.64 ± 0.069.30.640.68TRG-DT Model(+pCR)0.910.920.910.92 [0.84,1.00]0.88 ± 0.044.50.960.94pCR-DT’ Model0.900.680.710.82 [0.68,0.92]0.65 ± 0.034.60.710.61pCR-DT’ Model (+TRG)0.890.850.860.89 [0.79,0.97]0.83 ± 0.044.80.990.96TRG-DT’ Model0.740.710.720.76 [0.63,0.89]0.68 ± 0.034.40.640.59TRG-DT’ Model (+pCR)0.880.880.880.89 [0.79,0.97]0.87 ± 0.033.40.850.77pCR pathological complete response; TRG tumor regression grade; DT decision tree. Note: TRG-DT Model (using all clinical indicators), pCR-DT’ model (using only tumor markers), and TRG-DT’ model (using only tumor markers) Predictive efficacy of ML classifiers TRG-DT Model (+pCR) pCR pathological complete response; TRG tumor regression grade; DT decision tree. Note: TRG-DT Model (using all clinical indicators), pCR-DT’ model (using only tumor markers), and TRG-DT’ model (using only tumor markers) Explainability analysis Following the pipeline, models achieving at least Performance Level 1 (accuracy & F1-score ≥ 0.60) underwent explainability analysis. Three models met this specified criterion: the TRG-DT model (using all clinical indicators), the pCR-DT’ model (using only tumor markers), and the TRG-DT’ model (using only tumor markers). DT models inherently offer better explainability than others due to their tree structure, which can be visualized and intuitively understood. We used SHAP to analyze these models. TRG-DT: SHAP analysis identified CEA, SII, MONO, and CA19-9 as the most influential features (Fig. 3A). Low CEA, low CA19-9, and high SII values were associated with favorable TRG 0–1 (indicating good tumor regression), whereas high MONO was associated with TRG 2–3 (indicating poor tumor regression). SHAP value (A) TRG-DT Model; (B) pCR-DT’ Model; (C) TRG-DT’ model. pCR pathological complete response; TRG tumor regression grade; SII systemic immune-inflammation index; CEA carcinoembryonic antigen; MONO monocytes; CA carbohydrate antigen; PLT platelets; NEUT neutrophils; PIV pan-immune-inflammation value; WBC white blood cells; BMI body mass index; PLR platelet-to-lymphocyte ratio; NLR neutrophil-to-lymphocyte ratio; LYM lymphocytes pCR-DT’ and TRG-DT’ Models: For both models using only tumor markers, CEA was the most important variable, followed by CA19-9 (Fig. 3B and C). All three DT models used CEA ≤ 3.48 ng/ml as the root node, highlighting its paramount importance for efficacy prediction (Fig. 4). These findings align with clinical knowledge, and the explainability of these models successfully traversed multiple levels of our framework, reaching Level 4 (Clinician-Understandable). Explainability analysis of the selected DT models. (A) Global SHAP summary plot showing the overall ranking and directional contribution of key predictors in the selected model. (B) SHAP-based interpretation of tumor-marker effects, highlighting the dominant roles of CEA and CA19-9 in model decisions. (C) Example of a simplified clinically interpretable decision pathway derived from the DT model. Together, these panels illustrate how the selected model can be interpreted at both the global and case-level scales, thereby supporting clinician understanding of why a favorable or unfavorable response prediction is generated Internal clinical logic consistency check (non-deployable sanity check) We performed a post hoc internal consistency check to examine whether model behavior was aligned with known clinical relationships between correlated pathological endpoints. Specifically, we introduced pCR as an additional feature for TRG prediction and TRG as an additional feature for pCR prediction. Performance metrics increased substantially and SHAP ranked the added endpoint as the most influential feature, which is expected given their close relationship. Importantly, this analysis is non-deployable by design and represents intentional outcome information leakage. Therefore, it is presented only as a sanity check for internal consistency and does not constitute validation of predictive generalizability (see Fig. 5). SHAP analysis of all variables in the prediction model that included the efficacy feature (A) TRG-DT model (+pCR); (B) pCR-DT’ model (+TRG); (C) TRG-DT’ model (+pcr). pCR pathological complete response; TRG tumor regression grade; BMI body mass index; PIV pan-immune-inflammation value; CA carbohydrate antigen; LYM lymphocytes; PLT platelets; PLR platelet-to-lymphocyte ratio; CEA carcinoembryonic antigen; LMR lymphocyte-to-monocyte ratio; NLR neutrophil-to-lymphocyte ratio; SII systemic immune-inflammation index Discussion This study presents a structured ML model-selection pipeline for clinical research and demonstrates its application in a small single-center LARC cohort. The primary contribution of this work is methodological rather than algorithmic. Unlike AutoML-style frameworks, which are generally designed to maximize predictive performance through automated search and tuning, our workflow explicitly embeds pre-specified screening for both performance and explainability under small-sample clinical constraints. In addition, explainability is treated as a formal model-selection criterion rather than as a purely post hoc descriptive tool after model choice. Beyond conventional evaluation based only on statistical metrics, we emphasize that clinically useful models should also be interpretable and medically plausible. In this study, SHAP-based explanations were used to support multi-level interpretation of the selected models. We also included an internal clinical logic consistency check (non-deployable sanity check) to examine whether model behavior was directionally consistent with known relationships between correlated pathological endpoints. Importantly, this analysis does not represent predictive validation and should not be interpreted as evidence of generalizability. The performance of our selected model is of direct clinical importance. In the context of LARC treatment decision-making, misclassification carries significant risks. Small sample sizes in clinical studies can lead to class imbalance. Models may achieve high accuracy by simply predicting the majority class, masking poor performance on the minority class. Relying solely on accuracy can be misleading. Misclassification in healthcare can have serious consequences: False negatives (pCR predicted as non-pCR) might lead to unnecessary surgery [37], while false positives (non-pCR predicted as pCR) might result in inadequate treatment and disease progression. Therefore, our pipeline’s emphasis on minimizing such errors through the F1-score is critical. Based on the experimental results, models predicting pCR using all clinical indicators failed performance screening. The best model for predicting TRG using all features was the DT model (TRG-DT model). When using only tumor markers, the most effective models for predicting both pCR and TRG were also DT models (pCR-DT’ Model and TRG-DT’ Model), underscoring the strength of tree structures in clinical settings. This may be due to DT’s suitability for smaller datasets. Their structure mirrors human decision-making processes, with clear node conditions and branches that readily explain prediction rationale, making them easier for doctors to interpret and explain to patients, in contrast to the complex decision boundaries of models like SVM. Many studies do not perform or only perform explainability analysis on the best-performing model [38, 39]. However, the highest-performing model may not be the most explainable. In medicine, explainability plays a very important role [40]. Understanding and trusting a model is as important as its predictive power. To balance performance and explainability, our pipeline does not simply select the top-performing model but sets performance thresholds (Performance Level 1–3). Models exceeding the lowest threshold proceed to explainability screening. The goal is not exhaustive explanation but to enable clinicians and patients to understand the clinical rationale underlying model predictions, thereby facilitating the safe integration of ML into practice [28]. To empirically illustrate this explainability, we trace the interpretation of a representative prediction from our pCR-DT’ model. For a case predicted as ‘Good Response’, the SHAP force plot identified low pre-treatment CEA (2.1 ng/ml) and CA19-9 (15 U/ml) as the primary drivers. This explanation successfully traversed our defined levels: a data scientist can confirm these as key features from the summary plot (Level 3); a clinician immediately validates this logic against clinical knowledge (Level 4); and this rationale can be translated to the patient using an analogy such as ‘favorable cancer indicator levels’ (Level 5). This pathway demonstrates the model’s capacity for genuine clinical interpretability. The critical role of clinically meaningful features is further highlighted by a counterexample: in a radiomics model, even SHAP cannot bridge the “semantic gap” for features like “Wavelet-LHL_GLCM_Correlation,” stalling at Level 3 and preventing clinical trust. Beyond conventional evaluation that focuses on statistical metrics alone, we emphasize that clinical deployment requires models to be interpretable and clinically plausible. In this work, SHAP-based explanations were used to support multi-level interpretability and to enable clinicians to assess whether model-identified features and decision patterns align with clinical reasoning. [41, 42] We acknowledge that introducing outcome-related endpoints (e.g., adding pCR to predict TRG and vice versa) does not represent a valid prospective prediction setting and would constitute information leakage. We have therefore renamed this step as an internal clinical logic consistency check (non-deployable sanity check). The purpose of this check is limited to verifying whether model behavior and feature attribution are directionally consistent with the known concordance between pathological endpoints. This analysis is not used to claim predictive generalizability and should not be considered a recommended clinical prediction workflow. According to SHAP, CEA and CA19-9, which are both tumor markers, were strongly associated with outcomes. After first being found in colon cancer tissue extract in 1965, CEA began to become a test indicator for cancer detection, which is typically low in healthy adults (about 2–4 ng/ml) [43]. CA19-9 is an oligosaccharide tumor-associated antigen that exists in serum in the form of salivary mucin. It is the most sensitive marker for pancreatic cancer [44] and is also related to gastric cancer [45] and colorectal cancer [46]. Beyond diagnosis, both are implicated in efficacy prediction. Chen compared nomogram and ML (XGBoost) models for LARC, finding CEA and CA19-9 significant in both models [47, 48]. Clinically, pre-treatment CEA-negative rectal cancer accounts for 44%- 65% of cases [49, 50], yet monitoring strategies for these patients are insufficient [51]. Prognosis prediction using optimal CEA cut-offs appears superior to using standard reference values [52]. In terms of efficacy prediction, a meta-analysis indicated a significant negative correlation between pre-treatment CEA levels and pCR rate, suggesting a cut-off between 3 and 5 ng/ml may be suitable for LARC [53]. Moreno García [54] considered CEA ≤ 2.5 ng/ml an independent predictor of pCR, while Wallin [55] believed a cut-off of 3.4 ng/ml was more appropriate. In our DT models, CEA ≤ 3.48 ng/ml consistently appeared as the root-node split, highlighting its dominant contribution to model decisions. The direction of model interpretation was consistent across the Results and Discussion. Specifically, lower pretreatment CEA, together with lower CA19-9, tended to be associated with favorable pathological response (pCR and/or TRG 0–1), which is in line with prior evidence suggesting that lower baseline CEA is linked to a higher probability of pCR. The joint pattern of CEA and CA19-9 further indicates that combined tumor-marker profiles may provide more informative stratification than either marker alone. Clinicians should remain attentive even to moderately elevated pretreatment CEA values below the conventional positivity threshold, as such patterns may still carry prognostic and response-related information in selected patients. Both CEA and CA19-9 have been confirmed as independent predictors of nCRT efficacy in rectal cancer [56, 57]. In addition, Zhao proposed a serum tumor biomarker change (STBC) score based on CEA and CA19-9 dynamics, finding it predictive of efficacy [58]. Focusing on CEA alone, our findings do not support an adverse interpretation of lower pretreatment CEA. Instead, in line with the SHAP-based model interpretation and prior evidence, lower pretreatment CEA (≤3.48 ng/ml) was generally associated with a more favorable pathological response. [53]. Moreover, the interaction with CA19-9 in our DT models suggests that combined tumor-marker profiles may provide more informative stratification than either marker alone. Therefore, clinicians should pay greater attention to integrated tumor-marker patterns rather than interpreting CEA in isolation.CA19-9CA19-9 Different predictions based on CEA and the combination of CEA and CA19-9 may suggest that clinicians should pay more attention to comprehensive indicators of tumor markers. Existing studies on CA19-9 and rectal cancer often focus on long-term efficacy like overall survival (OS) and progression free survival (RFS) [46, 59]. Our study contributes by demonstrating its association with short-term pathological response (pCR, TRG). It is important to acknowledge the limitations of our research. First, our models were built exclusively on routine clinical and blood-based variables and did not incorporate imaging biomarkers (e.g., extramural vascular invasion), which may have limited the absolute predictive performance. Second, the overall performance of most candidate models was modest, and several test-set estimates exhibited relatively wide 95% confidence intervals, indicating uncertainty and potential performance fluctuation across different data distributions in this small single-center cohort. We acknowledge that, in a dataset of this size, model selection itself may be unstable, particularly when multiple feature groups and several algorithm families are compared simultaneously. Although we used five-fold cross-validation and bootstrap-based confidence intervals to partially address this issue, these procedures reduce but do not eliminate the risk of overfitting or optimistic selection in a single-center retrospective cohort. Third, all models were constructed using default hyperparameters to provide a transparent baseline comparison; however, this may underestimate the potential of some algorithms, particularly SVM and RF, under systematic tuning. Fourth, we did not perform formal probability calibration analyses (e.g., calibration curves or Brier scores), and therefore the reliability of predicted probabilities for clinical decision support remains to be established. Finally, validation was limited to a single hold-out test set from a relatively small, single-center cohort composed exclusively of patients of Asian descent, which limits external generalizability. Future multi-center and multi-ethnic studies with external validation are required to improve robustness, narrow uncertainty intervals, and support clinical translation. Future work will focus on validating and refining the proposed pipeline in larger multi-center cohorts with more heterogeneous patient populations and treatment settings. Conclusion In conclusion, this study presents a novel and systematic framework for machine learning model selection in clinical research, which moves beyond the conventional paradigm of solely pursuing predictive performance. Our primary contribution lies in the introduction of a structured, six-stage pipeline that explicitly and simultaneously evaluates models based on predefined, dual criteria of performance and explainability. This methodology addresses a critical gap in the translation of ML to clinical practice, where trust and interpretability are as crucial as accuracy. The case study in locally advanced rectal cancer effectively demonstrates the utility of this pipeline. It successfully identified a decision tree model that offered an optimal balance for clinical application and generated clinically plausible insights into the role of tumor markers. More importantly, the core innovation is the pipeline itself—a generalizable, transparent, and principled approach that can guide clinicians and researchers in selecting appropriate, trustworthy ML models tailored to specific clinical needs. By providing a clear pathway to navigate the trade-offs between model complexity and interpretability, this work offers a meaningful contribution to the crowded field of medical ML, facilitating more reliable and adoptable tools for advancing personalized oncology. Electronic supplementary material Below is the link to the electronic supplementary material. Supplementary Material 1 Acknowledgements Not applicable. Abbreviations Machine Learning Deep Learning Locally Advanced Rectal Cancer Neoadjuvant ChemoRadiotherapy Pathological Complete Response Clinical Complete Response Computer-Aided Diagnosis Support Vector Machine Decision Tree Random Forest Logistic Regression Local Interpretable Model-agnostic Explanations SHapley Additive exPlanations Tumor Regression Grade Body Mass Index White Blood Cells Neutrophils Platelets Lymphocytes Monocytes Neutrophil-to-Lymphocyte Ratio Platelet-to-Lymphocyte Ratio Lymphocyte-to-Monocyte Ratio Systemic Immune-inflammation Index Pan-Immune-inflammation Value Carcinoembryonic Antigen Carbohydrate Antigen Low Anterior Resection AbdominoPerineal Resection Intersphincteric Resection American Joint Committee on Cancer Confidence Interval True Positives False Positives True Negatives False Negatives Standard Deviation Coefficient of Variation Area Under the Receiver Operating Characteristic curve Area Under the Precision-Recall Curve Extramural Vascular Invasion Overall Survival Recurrence-Free Survival Serum Tumor Biomarker Change Author contributions YJ Xu: Data curation, formal analysis, investigation, writing original draft, writing–review and editing. W Gong: Data curation, formal analysis, writing–review and editing. L Ji: Methodology. ZX Xing: Formal analysis, investigation, writing original draft. ZY Zhu: Project administration, methodology, writing–review and editing. ZB Jin: Data curation, visualization, writing–review and editing. JY Zhang: Data curation, visualization, writing–review and editing. XY Wang: Language polishing. SB Qin: Resources, supervision, funding acquisition, investigation. Y Jiao: Resources, supervision, funding acquisition, investigation. LL Wang: Resources, supervision, funding acquisition, investigation, writing original draft, writing–review and editing. Funding This work is supported by the Project of Natural Science Foundations of China (82073476); the National Key R&D Program of China (2022YFC2503700, 2022YFC2503703); Special fund of Science and Technology Program of Jiangsu Province (BE2022727); Jiangsu Provincial Medical Key Discipline (ZDXK202235); Innovation Research Project of Medical and Industrial Cooperation in Suzhou (SLJ2021005). The sponsor or funding organization had no role in the design or conduct of this research. Data availability The datasets generated and analyzed during the current study are not publicly available due to patient confidentiality and privacy restrictions imposed by the ethics committee of the First Affiliated Hospital of Soochow University. However, de-identified data are available from the corresponding author on reasonable request. Data requestors may need to sign a data access agreement and obtain approval from the aforementioned ethics committee. Declarations Ethics approval and consent to participate This study was approved by the Ethics Committee of the First Affiliated Hospital of Soochow University (Approval number: NO. 2023–559). Consent for publication Not applicable. Registration Not applicable. This is a retrospective study for which formal prospective registration was not required. Patient and public involvement Patients or the public were not involved in the design, conduct, reporting, or dissemination of our research. Competing interests No conflicting relationship exists for any author. Footnotes Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Yijun Xu and Wei Gong contributed equally to this work. Contributor Information Songbing Qin, Email: qin92244@163.com. Yang Jiao, Email: jiaoyang@suda.edu.cn. Lili Wang, Email: wanglili@suda.edu.cn. References Associated Data Supplementary Materials Supplementary Material 1 Data Availability Statement The dataset analyzed in this study was retrospectively collected from the electronic medical record system of the First Affiliated Hospital of Soochow University. Owing to institutional and ethical restrictions related to patient privacy, the raw data are not publicly deposited in an open repository. De-identified data may be made available from the corresponding author upon reasonable request and subject to approval by the institutional ethics committee. The datasets generated and analyzed during the current study are not publicly available due to patient confidentiality and privacy restrictions imposed by the ethics committee of the First Affiliated Hospital of Soochow University. However, de-identified data are available from the corresponding author on reasonable request. Data requestors may need to sign a data access agreement and obtain approval from the aforementioned ethics committee.