Multi-task deep learning assists detection and diagnosis of gliomas and brain metastases Authors: Xiao Liu, Kun Lv, Peng Du, Kun Zhou, Qianru Zhao, Sen Jiao, Hongyi Chen, Danni Zhang, Hui Fang, Qiuyue Han, Yanwei Zeng, Xin Cao, Haiqing Li, Jian Dai, Zhiji Zheng, Hao Wu, Xin Wang, Yuxin Li, Daoying Geng Published: 2026 ABSTRACT Gliomas and brain metastases (BMs) on MRI pose significant diagnostic challenges for radiologists. This study aims to develop a multi-task model and a computer-aided diagnosis (CAD) system for the detection and diagnosis of gliomas and BMs. This study enrolled 3909 participants from seven centers, and developed a brain tumor segmentation and classification network (BTSC-Net) and BTSC-CAD with visualization of tumor masks. For detection, BTSC-Net achieved a Dice coefficient of 0.888 and 0.872 on the internal and external test sets, respectively. For diagnosis, BTSC-Net achieved AUCs of 0.941 and 0.933 on the internal and external test sets, respectively. With BTSC-CAD assistance, junior radiologists achieved mean AUC improvements of 4.8% (P < 0.05) for detection and 17.3% (P < 0.001) for diagnosis, along with an average reduction of 64.75 s in reading time. BTSC-CAD significantly improved radiologists’ diagnostic accuracy and efficiency. Introduction Gliomas and brain metastases (BMs) represent the two most prevalent malignant intracranial tumors in adults1,2. As the most common primary central nervous system tumor, glioma (specifically glioblastoma) exhibits rapid growth and invasive behavior, severely compromising patient survival and quality of life3. Distinguishing between glioma and BMs poses a challenge due to their shared characteristics on conventional magnetic resonance imaging (MRI)4,5, including irregular, ring-like enhancement, central cystic necrosis, and significant peritumoral edema6,7. Notably, around 40%–50% of BMs manifest as solitary lesions, sometimes lacking a clear primary tumor history, serving as an initial sign of systemic malignancy8. Conversely, GBM can arise in individuals with prior extracranial malignancies, further complicating diagnostic differentiation. Inexperienced radiologists have a higher tendency toward misdiagnosis, and diagnostic accuracy may additionally be compromised by visual fatigue, leading to a misdiagnosis rate of up to 10%–30%9,10. Despite imaging similarities, notable differences exist in histopathology and treatment strategies. BMs exhibit microvessels with compromised blood-brain barrier integrity and heightened capillary permeability, leading to vasogenic peritumoral regions with substantial growth. In contrast, glioma displays abnormal neovascularization with aggressive tumor cell invasion into the peritumoral area, resulting in pronounced peritumoral edema. Treatment approaches vary, with GBM typically managed through maximal surgical resection and concurrent chemotherapy11, while BMs often favor localized interventions like stereotactic radiosurgery12. Currently, histopathological examination remains the standard for the diagnosis of brain tumors, yet this invasive intervention entails considerable risks13. Therefore, achieving a noninvasive and precise preoperative differential diagnosis holds paramount importance for tailoring individualized treatment strategies and predicting outcomes. Deep learning enables the extraction of MRI features that extend beyond human visual perception, providing a foundation for predictive modeling in brain tumor analysis14. These models support accurate segmentation and automated diagnostic assistance, advancing computer-aided diagnosis (CAD) for brain tumors15. While segmentation models have demonstrated strong performance in prior studies16–18, and diagnostic models based on radiomics or deep learning features show promising classification ability19,20, many existing approaches adopt a cascaded segmentation-then-classification framework21,22. This two-stage design, however, often overlooks inter-task dependencies, making classification susceptible to segmentation errors and prone to error propagation across stages. In response, multi-task learning has emerged as an integrative strategy, allowing joint training of related tasks within a unified network23,24. By enabling feature sharing between detection and diagnosis, multi-task learning offers a promising pathway toward clinically viable CAD systems for gliomas and BMs. However, the similar appearance characteristics of gliomas and BMs, poor generalizability across datasets25, and lack of clinical validation pose challenges to the clinical implementation of CAD systems for brain tumors. This study aims to develop a multi-task deep learning model, the brain tumor segmentation and classification network (BTSC-Net), for the detection and diagnosis of gliomas and BMs using contrast-enhanced T1-weighted imaging (CE-T1WI) and T2-weighted fluid-attenuated inversion recovery (T2-FLAIR) sequences. Additionally, the model will be integrated into a computer-aided diagnosis system (BTSC-CAD). As a secondary aim, a clinical evaluation of BTSC-CAD was conducted. This system combines automatic segmentation and classification to assist radiologists in detecting and diagnosing gliomas and BMs, reducing manual workload and improving overall diagnostic accuracy. Results Patient characteristics The clinical characteristics of included participants are summarized in Table 1. In the training set (n = 3133), the mean age (±SD) was 49.15 ± 14.81 years, and 54.04% (1693 of 3133) were male. There were no significant differences in age (P = 0.386) or sex (P = 0.925) between datasets. The prevalence of gliomas was 29.91% in the internal test set and 33.87% in the external test set. Across all datasets, Across all datasets, the 2D maximum diameter of maximum abnormal regions (MAR) was approximately 70 mm (P = 0.483). The mean 2D maximum diameter of MAR in the external test set was slightly larger than that in the training set and internal test set.Table 1Patient demographic characteristics in the data setsCharacteristicTraining setInternal test setExternal test setP valueAge (y)49.15 ± 14.8148.15 ± 15.9049.57 ± 13.440.386Sex0.925 Male1693(54.04%)183(53.67%)231(53.11%) Female1440(45.96%)158(46.33%)204(46.89%)MRI machine<0.001 GE1130(36.07%)111(32.55%)191(43.91%) Siemens1298(41.43%)120(35.19%)126(28.97%) Philips587(18.74%)75(22.00%)64(14.71%) Others118(3.76%)35(10.26%)54(12.41%)Field strength<0.001 1.5 T457(14.7%)6(1.76%)35(8.05%) 3.0 T2676(85.3%)335(98.24%)400(91.95%)Types0.024 HCs1081(32.0%)137(40.18%)163(37.47%) Gliomas1049(33.5%)102(29.91%)143(33.87%) BMs1003(34.5%)102(29.91)129(29.66%)Number of Lesions4305286367<0.0012D Maximum Diameter of MAR in the Axial Plane (mm)69.02 ± 42.4570.15 ± 38.6772.37 ± 39.110.483Total3133341435-Data are numbers of examinations, with percentages in parentheses.HCs healthy controls, BMs brain metastases, MAR maximum abnormal region.Data are means ± SD.One-way ANOVA test and Kruskal-Wallis tests were used to compare continuous variables, and the χ2 test was used to compare categorical variables, with corresponding P values calculated. P < 0.05 was considered statistically significant. Table 1Patient demographic characteristics in the data setsCharacteristicTraining setInternal test setExternal test setP valueAge (y)49.15 ± 14.8148.15 ± 15.9049.57 ± 13.440.386Sex0.925 Male1693(54.04%)183(53.67%)231(53.11%) Female1440(45.96%)158(46.33%)204(46.89%)MRI machine<0.001 GE1130(36.07%)111(32.55%)191(43.91%) Siemens1298(41.43%)120(35.19%)126(28.97%) Philips587(18.74%)75(22.00%)64(14.71%) Others118(3.76%)35(10.26%)54(12.41%)Field strength<0.001 1.5 T457(14.7%)6(1.76%)35(8.05%) 3.0 T2676(85.3%)335(98.24%)400(91.95%)Types0.024 HCs1081(32.0%)137(40.18%)163(37.47%) Gliomas1049(33.5%)102(29.91%)143(33.87%) BMs1003(34.5%)102(29.91)129(29.66%)Number of Lesions4305286367<0.0012D Maximum Diameter of MAR in the Axial Plane (mm)69.02 ± 42.4570.15 ± 38.6772.37 ± 39.110.483Total3133341435-Data are numbers of examinations, with percentages in parentheses.HCs healthy controls, BMs brain metastases, MAR maximum abnormal region.Data are means ± SD.One-way ANOVA test and Kruskal-Wallis tests were used to compare continuous variables, and the χ2 test was used to compare categorical variables, with corresponding P values calculated. P < 0.05 was considered statistically significant. Patient demographic characteristics in the data sets Data are numbers of examinations, with percentages in parentheses. HCs healthy controls, BMs brain metastases, MAR maximum abnormal region. Data are means ± SD. One-way ANOVA test and Kruskal-Wallis tests were used to compare continuous variables, and the χ2 test was used to compare categorical variables, with corresponding P values calculated. P < 0.05 was considered statistically significant. Detection of gliomas and brain metastases At the lesion level for tumor detection, segmentation results of BTSC-Net are presented in Fig. 1a, and model performance is summarized in Table 2. The internal test set achieved a Dice coefficient of 0.888 (95% CI: 0.869, 0.907) and the 95th percentile Hausdorff distance (HD95) of 20.9 mm (95% CI: 16.2 mm, 25.6 mm). The external test set showed comparable performance, with a mean Dice coefficient of 0.872 (95% CI: 0.831, 0.913) and an HD95 of 22.7 mm (95% CI: 14.7 mm, 30.8 mm), reflecting only minor decreases of 1.6% and 1.8 mm, respectively.Fig. 1Segmentation results.a Visualized segmentation results of BTSC-Net. The first panel shows CE-T1WI; the second panel shows T2-FLAIR; the third panel shows the ground truth of MAR in T2-FLAIR (red indicates ground truth); the fourth panel shows the segmentation result of MAR by BTSC-Net (white lines indicate BTSC-Net segmentation); the fifth panel shows the gradient-weighted class activation mapping of the decoder in BTSC-Net; and the sixth panel shows the surface distance between the segmentation result and ground truth (color scale from blue to red indicates increasing surface distance). b Visualization of the surface distance of MAR in T2-FLAIR (color scale from pink to red indicates increasing surface distance). CE-T1WI contrast-enhanced T1-weighted sequences, T2-FLAIR T2-weighted fluid-attenuated inversion recovery sequences, BMs brain metastasis, MAR maximal abnormal region, GT ground truth, BTSC-Net multi-task network for segmentation and classification of gliomas and brain metastases, Grad-CAM gradient-weighted class activation mapping, SVD surface distance visualization, HD95 Hausdorff distance 95th percentile (mm).Table 2Performance of BTSC-Net in the data setsTasksVariableTraining setInternal test setExternal test setDetection of brain tumorNo. of participants3133341435No. of tumor patients2052204272Dice coefficient0.898 (0.895, 0.902)0.888 (0.869, 0.907)0.872 (0.831,0.913)Gliomas0.9020.9000.883BMs0.8970.8760.861HD95 (mm)22.423 (16.135, 28.709)20.912 (16.211, 25.613)22.736(14.712,30.760)Gliomas24.13222.51525.047BMs20.71419.30920.425Sensitivity0.998 (0.993, 0.999)0.995 (0.973, 0.999)0.993 (0.986, 1.000)Specificitya0.995 (0.991, 0.997)0.985 (0.948, 0.996)0.975 (0.939, 0.990)≤5 mm0.9980.9860.9655 mm~30 mm0.9950.9920.972≥30 mm0.9911.0000.979PPV(mm)a0.999 (0.996, 1.000)0.990 (0.965, 0.997)0.986 (0.963, 0.994)≤5 mm0.9920.9860.9815 mm~30 mm0.9980.9860.985≥30 mm0.9951.0000.991NPV0.991 (0.983, 0.995)0.993 (0.960, 0.999)0.988 (0.976, 1.000)Accuracy0.996 (0.993, 0.998)0.991 (0.974, 0.997)0.990 (0.977, 0.996)F1-score0.997 (0.994, 1.000)0.993 (0.986, 1.000)0.993 (0.986, 1.000)AUC0.997 (0.995, 0.998)0.990 (0.975, 1.000)0.988 (0.975, 0.998)Diagnosis of gliomas and BMsNo. of glioma/BMs1049/1003102/102143/129Sensitivity0.953 (0.938, 0.964)0.961 (0.903, 0.985)0.944 (0.893, 0.971)Specificitya0.979 (0.968, 0.986)0.921 (0.851, 0.959)0.921 (0.861, 0.957)≤5 mm0.9810.9590.9045 mm~30 mm0.9600.8570.892≥30 mm0.9770.9710.955PPV†0.956 (0.942, 0.967)0.925 (0.858, 0.961)0.931 (0.878, 0.962)≤5 mm0.9960.9320.9335 mm~30 mm0.9950.9600.929≥30 mm0.9450.8570.915NPV0.977 (0.966, 0.985)0.959 (0.899, 0.984)0.936 (0.879, 0.967)Accuracy0.966 (0.957, 0.973)0.941 (0.900, 0.966)0.933 (0.897, 0.957)F1-score0.967 (0.934, 1.000)0.942 (0.884, 1.000)0.938 (0.875, 1.000)AUC0.966 (0.958, 0.974)0.941 (0.906, 0.972)0.933 (0.900, 0.962)Data in parentheses are 95% CIs.HD95 Hausdorff distance 95th percentile (mm), BMs brain metastases, PPV positive predictive value, NPV negative predictive value, AUC area under the receiver operating characteristic curve.aSpecificity and PPV were calculated based on the 2D maximum diameter of the maximum abnormal region (using thresholds of ≤5 mm, 5~30 mm, and ≥30 mm). Segmentation results. a Visualized segmentation results of BTSC-Net. The first panel shows CE-T1WI; the second panel shows T2-FLAIR; the third panel shows the ground truth of MAR in T2-FLAIR (red indicates ground truth); the fourth panel shows the segmentation result of MAR by BTSC-Net (white lines indicate BTSC-Net segmentation); the fifth panel shows the gradient-weighted class activation mapping of the decoder in BTSC-Net; and the sixth panel shows the surface distance between the segmentation result and ground truth (color scale from blue to red indicates increasing surface distance). b Visualization of the surface distance of MAR in T2-FLAIR (color scale from pink to red indicates increasing surface distance). CE-T1WI contrast-enhanced T1-weighted sequences, T2-FLAIR T2-weighted fluid-attenuated inversion recovery sequences, BMs brain metastasis, MAR maximal abnormal region, GT ground truth, BTSC-Net multi-task network for segmentation and classification of gliomas and brain metastases, Grad-CAM gradient-weighted class activation mapping, SVD surface distance visualization, HD95 Hausdorff distance 95th percentile (mm). Table 2Performance of BTSC-Net in the data setsTasksVariableTraining setInternal test setExternal test setDetection of brain tumorNo. of participants3133341435No. of tumor patients2052204272Dice coefficient0.898 (0.895, 0.902)0.888 (0.869, 0.907)0.872 (0.831,0.913)Gliomas0.9020.9000.883BMs0.8970.8760.861HD95 (mm)22.423 (16.135, 28.709)20.912 (16.211, 25.613)22.736(14.712,30.760)Gliomas24.13222.51525.047BMs20.71419.30920.425Sensitivity0.998 (0.993, 0.999)0.995 (0.973, 0.999)0.993 (0.986, 1.000)Specificitya0.995 (0.991, 0.997)0.985 (0.948, 0.996)0.975 (0.939, 0.990)≤5 mm0.9980.9860.9655 mm~30 mm0.9950.9920.972≥30 mm0.9911.0000.979PPV(mm)a0.999 (0.996, 1.000)0.990 (0.965, 0.997)0.986 (0.963, 0.994)≤5 mm0.9920.9860.9815 mm~30 mm0.9980.9860.985≥30 mm0.9951.0000.991NPV0.991 (0.983, 0.995)0.993 (0.960, 0.999)0.988 (0.976, 1.000)Accuracy0.996 (0.993, 0.998)0.991 (0.974, 0.997)0.990 (0.977, 0.996)F1-score0.997 (0.994, 1.000)0.993 (0.986, 1.000)0.993 (0.986, 1.000)AUC0.997 (0.995, 0.998)0.990 (0.975, 1.000)0.988 (0.975, 0.998)Diagnosis of gliomas and BMsNo. of glioma/BMs1049/1003102/102143/129Sensitivity0.953 (0.938, 0.964)0.961 (0.903, 0.985)0.944 (0.893, 0.971)Specificitya0.979 (0.968, 0.986)0.921 (0.851, 0.959)0.921 (0.861, 0.957)≤5 mm0.9810.9590.9045 mm~30 mm0.9600.8570.892≥30 mm0.9770.9710.955PPV†0.956 (0.942, 0.967)0.925 (0.858, 0.961)0.931 (0.878, 0.962)≤5 mm0.9960.9320.9335 mm~30 mm0.9950.9600.929≥30 mm0.9450.8570.915NPV0.977 (0.966, 0.985)0.959 (0.899, 0.984)0.936 (0.879, 0.967)Accuracy0.966 (0.957, 0.973)0.941 (0.900, 0.966)0.933 (0.897, 0.957)F1-score0.967 (0.934, 1.000)0.942 (0.884, 1.000)0.938 (0.875, 1.000)AUC0.966 (0.958, 0.974)0.941 (0.906, 0.972)0.933 (0.900, 0.962)Data in parentheses are 95% CIs.HD95 Hausdorff distance 95th percentile (mm), BMs brain metastases, PPV positive predictive value, NPV negative predictive value, AUC area under the receiver operating characteristic curve.aSpecificity and PPV were calculated based on the 2D maximum diameter of the maximum abnormal region (using thresholds of ≤5 mm, 5~30 mm, and ≥30 mm). Performance of BTSC-Net in the data sets Data in parentheses are 95% CIs. HD95 Hausdorff distance 95th percentile (mm), BMs brain metastases, PPV positive predictive value, NPV negative predictive value, AUC area under the receiver operating characteristic curve. aSpecificity and PPV were calculated based on the 2D maximum diameter of the maximum abnormal region (using thresholds of ≤5 mm, 5~30 mm, and ≥30 mm). At the patient level for tumor detection (Table 2), the internal test set showed an accuracy of 0.991 (95% CI: 0.974, 0.997), with sensitivity and specificity of 0.995 (95% CI: 0.973, 0.999) and 0.985 (95% CI: 0.948, 0.996), respectively. The external test set showed an accuracy of 0.990 (95% CI: 0.977, 0.996), sensitivity of 0.993 (95% CI: 0.986, 1.000), and specificity of 0.975 (95% CI: 0.939, 0.990). Subgroup analysis (Table 2) showed that tumors with a maximum 2D diameter of MAR < 5 mm achieved specificity and PPV of 0.986 and 0.986 in the internal test set, and 0.965 and 0.981 in the external test set. In Tables S1 and S2, BTSC-Net demonstrated robust detection across internal and external test sets. Sensitivity and accuracy remained high across age and sex, indicating minimal demographic influence. Performance was stable across the MRI machine, with slightly lower external accuracy for GE (0.850). Imaging sequences, field strength, and acquisition Matrix of T2-FLAIR had limited impact, while thinner CE-T1CE slices (≤1.5 mm) achieved the highest metrics. Detection of healthy controls, gliomas, and BMs remained uniformly strong. The Kolmogorov–Smirnov curves and Precision–Recall curves are provided in Figs. S1 and S2. Diagnosis of gliomas and brain metastases For the diagnosis of gliomas and BMs, confusion matrices, ROC curves, and evaluation metrics are shown in Fig. 2 and Table 2. In the internal test set, accuracy was 0.941 (95% CI: 0.900, 0.966), with an AUC of 0.941 (95% CI: 0.906, 0.972) as shown in Fig. 2a. In the external test set, accuracy was 0.933 (95% CI: 0.897, 0.957), representing a 0.8% decrease compared with the internal test set, while the AUC decreased slightly to 0.933 (95% CI: 0.900, 0.962) as shown in Fig. 2b.Fig. 2Confusion matrices and receiver operating characteristic (ROC) curves for the internal and external test sets.a Internal test set. b External test set. BMs brain metastasis, HCs healthy controls, AUC area under the receiver operating characteristic curve. Confusion matrices and receiver operating characteristic (ROC) curves for the internal and external test sets. a Internal test set. b External test set. BMs brain metastasis, HCs healthy controls, AUC area under the receiver operating characteristic curve. In the 2D Maximum diameter subgroup analysis, BTSC-Net showed better diagnostic performance for gliomas with MAR ≥ 30 mm. In the internal test set, specificity and PPV were 0.971 and 0.960, respectively; in the external test set, they were 0.955 and 0.929, respectively. BTSC-Net demonstrated robust diagnosis performance across internal and external test sets, as shown in Tables S1 and S2. Accuracy and sensitivity were stable across age and sex groups. Across the MRI machine, internal accuracy ranged from 0.866 to 1.000 and external from 0.922 to 0.958, with minor machine for Philips. Imaging sequence, field strength, and acquisition Matrix of T2-FLAIR had a limited impact, while thinner CE-T1CE slices (≤1.5 mm) achieved the highest metrics, and a thinner slice thickness further improved the accuracy. Gliomas and BMs were consistently well classified. The Kolmogorov–Smirnov curves and Precision–Recall curves are provided in Figs. S1 and S2. Comparison with other methods BTSC-Net was compared with SOTA segmentation methods (3D Unet26, Mamba-Unet27, TransBTSV228), as shown in Figs. 1b and 3. In the detection task (Fig. 3a), TransBTSV2 was the suboptimal method in the internal test set and external test set, with Dice coefficient of 0.841 (95% CI: 0.714, 0.968) and 0.811 (95% CI: 0.697, 0.925), AUCs of 0.962 (95% CI: 0.925, 0.980; P = 0.022) and 0.947 (95% CI: 0.932, 0.962; P < 0.001), respectively. Compared with TransBTSV2, BTSC-Net increased the mean Dice coefficient by 4.7% and 6.1% and improved AUC by 2.8% and 4.1% in the internal and external test sets, respectively.Fig. 3Detection and diagnosis results of different methods.a Internal test set. b External test set. BM brain metastasis, BTSC-Net multi-task network for segmentation and classification of gliomas and brain metastases, HD95 Hausdorff distance 95th percentile (cm), PPV positive predictive value, NPV negative predictive value, AUC area under the receiver operating characteristic curve; The “-” indicates that the smaller the absolute value, the better the model performance. Detection and diagnosis results of different methods. a Internal test set. b External test set. BM brain metastasis, BTSC-Net multi-task network for segmentation and classification of gliomas and brain metastases, HD95 Hausdorff distance 95th percentile (cm), PPV positive predictive value, NPV negative predictive value, AUC area under the receiver operating characteristic curve; The “-” indicates that the smaller the absolute value, the better the model performance. In the diagnosis task (Fig. 3b), BTSC-Net was compared with SOTA classification methods (Radiomics29 (Table S5), EfficientNetV230, and T2T-ViT31). T2T-ViT was the best-performing comparator, with accuracies of 0.887 (95% CI: 0.837, 0.924) and 0.896 (95% CI: 0.856, 0.933) in the internal test set and external test set. BTSC-Net improved accuracy by 5.4% and 3.7% in the internal test set and external test set, and improved AUC by 5.4% (P = 0.043) and 3.8% (P = 0.015), respectively. The calibration curves, decision curves, and Brier scores of different methods further demonstrated the superior performance of BTSC-Net, as shown in Fig. S3. Clinical performance of BTSC-CAD A senior radiologist and four junior radiologists conducted a two-phase crossover study (independent diagnosis and BTSC-CAD-assisted diagnosis) based on the external test set to evaluate the clinical efficacy of BTSC-CAD (Table 3 and Fig. 4). Additional results comparing BTSC-CAD-assisted detection and diagnoses across radiologists are provided in Table S3 and Fig. S4.Fig. 4Results of the senior radiologist, junior radiologists, and BTSC-CAD in the external test set.a ROC curves of BTSC-CAD and the senior radiologist. b ROC curves of junior radiologists with and without BTSC-CAD. c Examples of difficult detection and diagnosis cases. BMs brain metastasis, BTSC-CAD computer-aided diagnosis system for segmentation and classification of gliomas and brain metastases, w/ with, w/o without, AUC area under the receiver operating characteristic curve, ROC receiver operating characteristic curve.Table 3Results of BTSC-CAD-assisted radiologists in gliomas and BMsTaskReadersSensitivitySpecificityAccuracyAUCNRITime (s)Detection of brain tumorBTSC-CAD0.9930.9750.9900.9880.004(P = 0.323)25 ± 0.35(0.986, 1.000)(0.939, 0.990)(0.977, 0.996)(0.975, 0.998)Senior radiologist0.9890.9880.9930.99163 ± 28.31(0.968, 0.996)(0.977, 1.000)(0.980, 0.998)(0.982, 1.000)Radiologist 1w/o BTSC-CAD0.8640.9940.9130.9050.092(P < 0.001)128 ± 40.29(0.818, 0.900)(0.966, 0.999)(0.882, 0.936)(0.861, 0.949)w/ BTSC-CAD0.9560.9940.9700.96485 ± 33.32(0.924, 0.975)(0.966, 0.999)(0.950, 0.982)(0.940, 0.988)Radiologist 2w/o BTSC-CAD0.8380.9880.8940.8880.085(P = 0.001)135 ± 35.89(0.790, 0.877)(0.956, 0.997)(0.862, 0.920)(0.840, 0.936)w/BTSC-CAD0.9230.9880.9470.93872 ± 30.27(0.885, 0.949)(0.956, 0.997)(0.922, 0.965)(0.904, 0.972)Radiologist 3w/o BTSC-CAD0.8900.9820.9240.9150.051(P = 0.026)175 ± 40.36(0.847, 0.922)(0.947, 0.994)(0.895, 0.945)(0.875, 0.955)w/ BTSC-CAD0.9340.9880.9540.946102 ± 27.44(0.898, 0.958)(0.956, 0.997)(0.930, 0.970)(0.929, 0.977)Radiologist 4w/o BTSC-CAD0.8490.9880.9010.8940.088(P < 0.001)183 ± 48.75(0.802, 0.887)(0.956, 0.997)(0.869, 0.926)(0.846, 0.942)w/ BTSC-CAD0.9380.9880.9560.948147 ± 30.27(0.902, 0.961)(0.956, 0.997)(0.933, 0.972)(0.918, 0.978)Diagnosis of gliomas and BMsBTSC-CAD0.9440.9210.9330.9330.029(P = 0.057)25 ± 0.35(0.893, 0.971)(0.861, 0.957)(0.897, 0.957)(0.900, 0.962)Senior radiologist0.9440.9530.9420.94883 ± 33.17(0.893, 0.971)(0.902, 0.979)(0.915, 0.969)(0.919, 0.974)Radiologist 1w/o BTSC-CAD0.7060.7050.7060.7050.335(P < 0.001)195 ± 51.22(0.627, 0.775)(0.622, 0.777)(0.649, 0.757)(0.652, 0.758)w/ BTSC-CAD0.9790.7670.8790.897136 ± 24.73(0.940, 0.993)(0.687, 0.832)(0.835, 0.912)(0.833, 0.910)Radiologist 2w/o BTSC-CAD0.8320.6250.7320.7390.209(P < 0.001)182 ± 56.67(0.762, 0.885)(0.534, 0.699)(0.676, 0.781)(0.662, 0.776)w/BTSC-CAD0.9790.6820.8380.870129 ± 20.32(0.940, 0.993)(0.598, 0.756)(0.790, 0.877)(0.866, 0.874)Radiologist 3w/o BTSC-CAD0.8810.6650.6840.7130.276(P = 0.001)242 ± 55.74(0.818, 0.924)(0.581, 0.751)(0.626, 0.736)(0.704, 0.722)w/ BTSC-CAD0.9790.6430.8200.859163 ± 22.15(0.940, 0.993)(0.558, 0.721)(0.770, 0.861)(0.827, 0.891)Radiologist 4w/o BTSC-CAD0.8880.3330.6250.6630.397(P < 0.001)225 ± 66.71(0.826, 0.930)(0.258, 0.418)(0.566, 0.680)(0.657, 0.669)w/ BTSC-CAD0.9730.7150.8600.888157 ± 20.68(0.950, 0.996)(0.638, 0.791)(0.814, 0.896)(0.881, 0.895)Data in parentheses are 95% CIs; Data following ± are standard deviations; AUC area under the receiver operating characteristic curve, BTSC-CAD computer-aided diagnosis system for segmentation and classification of gliomas and brain metastases, NRI Net reclassification improvement, w/ With, w/o Without. Results of the senior radiologist, junior radiologists, and BTSC-CAD in the external test set. a ROC curves of BTSC-CAD and the senior radiologist. b ROC curves of junior radiologists with and without BTSC-CAD. c Examples of difficult detection and diagnosis cases. BMs brain metastasis, BTSC-CAD computer-aided diagnosis system for segmentation and classification of gliomas and brain metastases, w/ with, w/o without, AUC area under the receiver operating characteristic curve, ROC receiver operating characteristic curve. Table 3Results of BTSC-CAD-assisted radiologists in gliomas and BMsTaskReadersSensitivitySpecificityAccuracyAUCNRITime (s)Detection of brain tumorBTSC-CAD0.9930.9750.9900.9880.004(P = 0.323)25 ± 0.35(0.986, 1.000)(0.939, 0.990)(0.977, 0.996)(0.975, 0.998)Senior radiologist0.9890.9880.9930.99163 ± 28.31(0.968, 0.996)(0.977, 1.000)(0.980, 0.998)(0.982, 1.000)Radiologist 1w/o BTSC-CAD0.8640.9940.9130.9050.092(P < 0.001)128 ± 40.29(0.818, 0.900)(0.966, 0.999)(0.882, 0.936)(0.861, 0.949)w/ BTSC-CAD0.9560.9940.9700.96485 ± 33.32(0.924, 0.975)(0.966, 0.999)(0.950, 0.982)(0.940, 0.988)Radiologist 2w/o BTSC-CAD0.8380.9880.8940.8880.085(P = 0.001)135 ± 35.89(0.790, 0.877)(0.956, 0.997)(0.862, 0.920)(0.840, 0.936)w/BTSC-CAD0.9230.9880.9470.93872 ± 30.27(0.885, 0.949)(0.956, 0.997)(0.922, 0.965)(0.904, 0.972)Radiologist 3w/o BTSC-CAD0.8900.9820.9240.9150.051(P = 0.026)175 ± 40.36(0.847, 0.922)(0.947, 0.994)(0.895, 0.945)(0.875, 0.955)w/ BTSC-CAD0.9340.9880.9540.946102 ± 27.44(0.898, 0.958)(0.956, 0.997)(0.930, 0.970)(0.929, 0.977)Radiologist 4w/o BTSC-CAD0.8490.9880.9010.8940.088(P < 0.001)183 ± 48.75(0.802, 0.887)(0.956, 0.997)(0.869, 0.926)(0.846, 0.942)w/ BTSC-CAD0.9380.9880.9560.948147 ± 30.27(0.902, 0.961)(0.956, 0.997)(0.933, 0.972)(0.918, 0.978)Diagnosis of gliomas and BMsBTSC-CAD0.9440.9210.9330.9330.029(P = 0.057)25 ± 0.35(0.893, 0.971)(0.861, 0.957)(0.897, 0.957)(0.900, 0.962)Senior radiologist0.9440.9530.9420.94883 ± 33.17(0.893, 0.971)(0.902, 0.979)(0.915, 0.969)(0.919, 0.974)Radiologist 1w/o BTSC-CAD0.7060.7050.7060.7050.335(P < 0.001)195 ± 51.22(0.627, 0.775)(0.622, 0.777)(0.649, 0.757)(0.652, 0.758)w/ BTSC-CAD0.9790.7670.8790.897136 ± 24.73(0.940, 0.993)(0.687, 0.832)(0.835, 0.912)(0.833, 0.910)Radiologist 2w/o BTSC-CAD0.8320.6250.7320.7390.209(P < 0.001)182 ± 56.67(0.762, 0.885)(0.534, 0.699)(0.676, 0.781)(0.662, 0.776)w/BTSC-CAD0.9790.6820.8380.870129 ± 20.32(0.940, 0.993)(0.598, 0.756)(0.790, 0.877)(0.866, 0.874)Radiologist 3w/o BTSC-CAD0.8810.6650.6840.7130.276(P = 0.001)242 ± 55.74(0.818, 0.924)(0.581, 0.751)(0.626, 0.736)(0.704, 0.722)w/ BTSC-CAD0.9790.6430.8200.859163 ± 22.15(0.940, 0.993)(0.558, 0.721)(0.770, 0.861)(0.827, 0.891)Radiologist 4w/o BTSC-CAD0.8880.3330.6250.6630.397(P < 0.001)225 ± 66.71(0.826, 0.930)(0.258, 0.418)(0.566, 0.680)(0.657, 0.669)w/ BTSC-CAD0.9730.7150.8600.888157 ± 20.68(0.950, 0.996)(0.638, 0.791)(0.814, 0.896)(0.881, 0.895)Data in parentheses are 95% CIs; Data following ± are standard deviations; AUC area under the receiver operating characteristic curve, BTSC-CAD computer-aided diagnosis system for segmentation and classification of gliomas and brain metastases, NRI Net reclassification improvement, w/ With, w/o Without. Results of BTSC-CAD-assisted radiologists in gliomas and BMs 0.004 (P = 0.323) 0.092 (P < 0.001) 0.085 (P = 0.001) 0.051 (P = 0.026) 0.088 (P < 0.001) 0.029 (P = 0.057) 0.335 (P < 0.001) 0.209 (P < 0.001) 0.276 (P = 0.001) 0.397 (P < 0.001) Data in parentheses are 95% CIs; Data following ± are standard deviations; AUC area under the receiver operating characteristic curve, BTSC-CAD computer-aided diagnosis system for segmentation and classification of gliomas and brain metastases, NRI Net reclassification improvement, w/ With, w/o Without. For brain tumor detection shown in Fig. 4a, BTSC-CAD achieved an accuracy of 0.990 (95% CI: 0.977, 0.996) and an AUC of 0.988 (95% CI: 0.975, 0.998), without significant net reclassification improvement (NRI = 0.004, P = 0.323) compared with the senior radiologist, whose AUC was 0.991 (95% CI: 0.982, 1.000). The senior radiologist also demonstrated sensitivity of 0.989 (95% CI: 0.968, 0.996) and specificity of 0.988 (95% CI: 0.977, 1.000), with the highest overall accuracy of 0.993 (95% CI: 0.980, 0.998). As shown in Fig. 4b, the four junior radiologists exhibited significantly lower baseline detection AUCs (0.888 to 0.915). After BTSC-CAD assistance, their detection sensitivity increased by 4.4%–9.2%, and AUCs improved by 0.031–0.059 (all P < 0.05), largely due to BTSC-CAD’s ability to help identify small, otherwise difficult-to-detect lesions (Fig. 4c). For the diagnosis of gliomas versus BMs, BTSC-CAD achieved an accuracy of 0.933 (95% CI: 0.897, 0.957) and an AUC of 0.933 (95% CI: 0.900, 0.962), with no significant NRI compared with the senior radiologist (NRI = 0.029, P = 0.057). The senior radiologist reached a slightly higher AUC of 0.948 (95% CI: 0.919, 0.974) and sensitivity of 0.944 (95% CI: 0.893, 0.971). Without BTSC-CAD, junior radiologists showed diagnostic AUCs from 0.663 to 0.739, with considerable variability in specificity from 0.333 to 0.705. With BTSC-CAD assistance, AUCs improved substantially (by 0.131–0.225, all P < 0.001), with the greatest benefit observed in radiologists with lower baseline performance. This demonstrates BTSC-CAD’s value in supporting accurate differentiation of gliomas and BMs (Fig. 4c). In addition, BTSC-CAD reduced junior radiologists’ reading times by 53–79 s (29% ~ 33%), with an average reduction of 64.8 s (30.8%). Discussion This study presents a multi-task deep learning framework, termed BTSC-Net, for the simultaneous detection and diagnosis of gliomas and BMs. By integrating pixel-level segmentation with high-level regional features, the unified framework enhances both segmentation and diagnostic accuracy for these two malignant brain tumors. More importantly, a clinically oriented computer-aided diagnosis system based on BTSC-Net was further developed and integrated into the hospital PACS environment, representing a concrete step toward an end-to-end detection-segmentation-diagnosis-triage-treatment paradigm in neuro-oncology practice. For brain tumor detection, BTSC-Net showed strong overall performance in both the internal and external test sets. However, it still missed several lesions, resulting in sensitivity decreases of 0.3% and 0.5% compared with the training set. Performance in detecting brain tumors <5 mm was somewhat limited. T2-FLAIR has high sensitivity for lesions, with a typical lower detection limit of about 5 mm32,33. Conversely, CE-T1WI can identify smaller enhancing lesions (<3 mm in 2D maximum diameter) but has limited ability to detect low-grade gliomas without blood-brain barrier disruption. Notably, BTSC-Net outperformed Transformer-based architectures, attributable to the SBAM (Table S6) and the adaptive weighted Dice-cross-entropy loss. These design choices facilitate more precise delineation of anatomical characteristics typical of BMs to improve boundary localization and sensitivity. The feature-importance visualization technique based on gradient-weighted class activation mapping (Grad-CAM) delineates the imaging characteristics of lesions34 delineates the imaging characteristics of lesions. Grad-CAM facilitates a more intuitive understanding of the correspondence between image features and segmentation results, enhancing model interpretability and improving differentiation of tumor tissues. For small brain tumors, Grad-CAM visualizations generated by BTSC-Net can help radiologists better identify such lesions. Furthermore, Grad-CAMs of the encoder showed hotspots with distinct patterns (e.g., circular rings or vertical stripes) in extracerebral regions, possibly due to prominent non-zero gradient responses in background voxels after normalization. To exclude potential confounding effects of skull-related features, we reran BTSC-Net on skull-stripped MRIs. The results remained comparable to those obtained with the original MRIs (Table S8), indicating that skull-related signals do not drive the BTSC-Net’s predictions. In the differential diagnosis of gliomas versus BMs, the relatively low incidence of gliomas and their overlapping imaging features pose diagnostic challenges for both BTSC-Net and radiologists, especially for large-volume tumors (≥30 mm in 2D maximum diameter)4,35. BTSC-Net showed relatively low PPVs of 0.945 and 0.857 in the internal and external test sets, respectively. This difficulty arises because large gliomas and BMs can both exhibit central necrosis with ring enhancement, peritumoral edema, and diffusion restriction on conventional MRI13, thereby reducing the algorithm’s differential diagnostic performance. Compared with radiomics-based approaches and global feature-dependent models such as T2T-ViT, BTSC-Net achieved substantial improvements in diagnostic accuracy and discriminative performance. These results indicate that, through the multitask learning framework and the FFCM, BTSC-Net effectively integrates enhanced shared representations with re-encoded features, thereby jointly optimizing auxiliary diagnostic performance. The diagnostic performance of BTSC-Net remains limited by slice thickness and in-plane resolution, highlighting the need for further research on image quality enhancement methods to improve model generalization. Grad-CAM facilitates a more intuitive understanding of the correspondence between image features and diagnostic outcomes, enhancing model interpretability and improving differentiation of tumor tissues. BTSC-CAD, based on a C/S architecture, was developed and clinically evaluated, a step not reported in previous studies36,37. BTSC-CAD can reduce reading time to a clinically feasible range (<30 s per case) through heterogeneous computing optimization, such as GPU acceleration, enabled by robust parallel processing. BTSC-CAD showed accuracy differences of only 0.003 for detection and 0.009 for diagnosis compared with senior radiologists. It achieved lesion detection sensitivity and accuracy comparable to those of senior radiologists, although diagnostic specificity was slightly lower. Importantly, BTSC-CAD assistance significantly improved the diagnostic performance of junior radiologists, as evidenced by a positive and statistically significant NRI. Moreover, BTSC-CAD reduced reading times by approximately 25%–30% across all readers, highlighting its potential to enhance workflow efficiency in high-volume clinical settings. Nevertheless, the modest reduction in specificity observed in some radiologists raises concerns about potential overdiagnosis, underscoring that AI should serve as an adjunct to, rather than a replacement for, expert interpretation. This combined approach may provide a generalized solution for applying deep learning algorithms to disease screening and diagnosis, thereby reducing workload and improving accuracy38. This study has several limitations. First, the advent of targeted therapies and immunotherapies has produced complex imaging patterns, such as pseudoprogression and pseudoremission, which complicate assessment and differential diagnosis. Because of the limited available data, some patients in this study may have received systemic therapies or corticosteroids; these treatments can alter brain tumor imaging features, and we could not assess their potential impact on model performance. Second, BTSC-CAD was designed for gliomas and BMs; it requires manual screening of diagnostic results before clinical application and does not apply to other intracranial lesions. Third, reported performance metrics (e.g., AUC, sensitivity, specificity) may not directly generalize to clinical practice, as the balanced 1:1:1 cohort does not reflect real-world prevalence. Prospective validation under natural prevalence and rigorous assessment in open-set scenarios are therefore essential before clinical deployment. Future work will evaluate the influence of extracerebral MRI signals by excluding background and skull regions. We will also assemble larger, more diverse brain lesion cohorts reflecting real-world populations, followed by cross-cohort evaluation and prospective multicenter validation to establish the generalizability and clinical robustness of BTSC-CAD. In conclusion, the multi-task model developed using multimodal MRI improved the detection and diagnosis of gliomas and BMs. The performance of the computer-aided diagnosis system with a user-friendly interface was not statistically different from that of senior radiologists in the external test set. Moreover, the system provided significant assistance to junior radiologists in both detection and diagnosis, supporting the integration of this technology into the clinical workflows and improving diagnostic efficiency while informing therapeutic decision-making for brain tumors. Methods Participants The study participants were recruited from seven centers, namely Huashan Hospital, Fudan University, Affiliated Hospital of Xuzhou Medical University, Xuanwu Hospital Capital Medical University, Hainan General Hospital, Fourth Hospital of Harbin Medical University, Second Hospital & Clinical Medical School, Lanzhou University, and Fifth Hospital Affiliated to Wenzhou Medical University. This study has received approval from the Ethics Review Committee of Huashan Hospital, Fudan University (KY2023-543). An English version of the ethics approval is available upon request. As this was a retrospective study, the requirement for informed consent was waived. All procedures conformed to the Declaration of Helsinki and relevant national regulations. A total of 3909 participants were included from seven centers between January 2015 and January 2024 (Fig. 5) by a stratified random sampling method. Data from 3474 cases from centers 1 to 3 were randomly divided by stratified sampling into a training set (n = 3133) and an internal test set (n = 341) at a 9:1 ratio. An external test set was obtained from centers 4 to 7 (n = 435). Each case included both CE-T1WI and T2-FLAIR sequences. Details of the MRI examinations are provided in Table S4.Fig. 5Flowchart of the multi-center datasets.CE-T1WI contrast-enhanced T1-weighted sequences, T2-FLAIR T2-weighted fluid-attenuated inversion recovery sequences, BMs brain metastases, HCs healthy controls. Flowchart of the multi-center datasets. CE-T1WI contrast-enhanced T1-weighted sequences, T2-FLAIR T2-weighted fluid-attenuated inversion recovery sequences, BMs brain metastases, HCs healthy controls. Annotation of gliomas and brain metastases The annotation workflow followed a collaborative, distributed review-refine paradigm. Initially, the maximum abnormal regions (MARs) of all visible tumors based on T2-FLAIR were manually annotated in a blinded manner by four radiologists (K.L., P.D., X.C., and H.L.), each with over 10 years of experience, using ITK-SNAP (version 3.8; http://www.itksnap.org). Then, a senior radiologist (D.G.) with 30 years of experience reviewed all annotations, provided modification suggestions, and oversaw iterative revisions performed by the four radiologists until consensus was achieved. The diagnostic labels (gliomas or BM) were determined by experts with reference to pathology reports. Model construction and performance evaluation The workflow of this study consisted of four stages, as shown in Fig. 6a. In the image preprocessing stage, CE-T1WI images were registered to T2-FLAIR images using rigid registration implemented with the SimpleITK toolkit (version 2.4.1; https://simpleitk.org/). All MRIs were then resampled to a uniform voxel spacing of 1 × 1 × 1 mm3 via linear interpolation, and grayscale values were normalized to the range 0–255 using min-max normalization.Fig. 6Overview of study approach.a The pipeline of this study. b Architecture of BTSC-Net. BTSC-Net comprises a BTS-Net and a BTC-Net. BTSC-Net adopts a U-shaped CNN-Transformer hybrid architecture. The mirror encoder and SBAM are introduced to emphasize boundary-background differences and enhance high-level tumor representations. CNN features, Transformer features, and visual difference features are concatenated and decoded for MAR segmentation. BTC-Net leverages shared segmentation features and is re-encoded to get the Transformer features. Then, the Transformer features and the shared features weighted by the symbolic distance map are activated through FFCM to obtain fusion features. After undergoing convolution and global average pooling operations, fusion features finally generate the predicted classification result in the Softmax layer. The adaptive weighted Dice loss is combined with the cross-entropy loss to minimize the composite loss function for training the network. CE-T1WI contrast-enhanced T1-weighted sequences, T2-FLAIR T2-weighted fluid-attenuated inversion recovery sequences, DL deep learning, BTSC-Net multi-task network for segmentation and classification of gliomas and brain metastases, BTS-Net brain tumor segmentation network, BTC-Net brain tumor classification network, HC healthy control, BM brain metastasis, BTSC-CAD computer-aided diagnosis system for segmentation and classification of gliomas and brain metastases, CNN convolutional neural network, SBAM symmetric boundary attention module, FFCM feature fusion and correction module, RSU residual U-block, GAP global average pooling. Overview of study approach. a The pipeline of this study. b Architecture of BTSC-Net. BTSC-Net comprises a BTS-Net and a BTC-Net. BTSC-Net adopts a U-shaped CNN-Transformer hybrid architecture. The mirror encoder and SBAM are introduced to emphasize boundary-background differences and enhance high-level tumor representations. CNN features, Transformer features, and visual difference features are concatenated and decoded for MAR segmentation. BTC-Net leverages shared segmentation features and is re-encoded to get the Transformer features. Then, the Transformer features and the shared features weighted by the symbolic distance map are activated through FFCM to obtain fusion features. After undergoing convolution and global average pooling operations, fusion features finally generate the predicted classification result in the Softmax layer. The adaptive weighted Dice loss is combined with the cross-entropy loss to minimize the composite loss function for training the network. CE-T1WI contrast-enhanced T1-weighted sequences, T2-FLAIR T2-weighted fluid-attenuated inversion recovery sequences, DL deep learning, BTSC-Net multi-task network for segmentation and classification of gliomas and brain metastases, BTS-Net brain tumor segmentation network, BTC-Net brain tumor classification network, HC healthy control, BM brain metastasis, BTSC-CAD computer-aided diagnosis system for segmentation and classification of gliomas and brain metastases, CNN convolutional neural network, SBAM symmetric boundary attention module, FFCM feature fusion and correction module, RSU residual U-block, GAP global average pooling. In the multi-task deep learning model stage (BTSC-Net, as shown in Fig. 6b), the segmentation subnet (BTS-Net) employs parallel CNN and Transformer encoders, a symmetric boundary attention module (SBAM), and a deep-supervision decoder. These components work together to extract and analyze high-level semantic features from MRIs for tumor segmentation. The classification subnet (BTC-Net) utilizes a feature fusion and correction module (FFCM) to enhance tumor classification features by integrating segmentation-shared features and re-encoded features. Final diagnostic outputs were produced by a global average pooling layer followed by a Softmax layer. BTSC-Net was trained with a composite loss function combining adaptive weighted Dice loss and cross-entropy loss. In the CAD system stage, BTSC-CAD was developed for the detection and diagnosis of gliomas and BMs based on a client-server (C/S) architecture and BTSC-Net, as shown in Fig. 6a. The functionalities of BTSC-CAD include user authentication, local image data storage and management, AI-assisted detection and diagnosis of brain tumors, 3D segmentation visualization, and formatted report generation and printing. The interface and functional details of BTSC-CAD are shown in Figs. S5 and S6. In the clinical performance stage, radiologist evaluation was conducted on the external test set by one senior radiologist (Y.L., 20 years of experience) and four junior radiologists (D.Z., H.F., Y.Z., and Q.H., 3 ~ 5 years of experience). The senior radiologist independently performed the detection and diagnosis of gliomas and BMs. The junior radiologists were randomly assigned in a 1:1 ratio to either the software-assisted diagnosis followed by independent diagnosis group or the independent diagnosis followed by software-assisted diagnosis group. They interpreted the external test set in a crossover design, and all results were recorded for statistical analysis. A washout period of 4 weeks was implemented between assisted and unassisted reads to minimize recall bias. Additional details of the evaluation protocol are provided in Fig. S7. Statistical analysis Model performance was evaluated using the Dice coefficient for segmentation, HD95, diagnostic accuracy, sensitivity, specificity, F1-score, and area under the receiver operating characteristic curve. Statistical analysis was conducted using Python 3.9 and GraphPad Prism 8. Differences between hospitals were analyzed using independent t-tests and chi-square tests. One-way ANOVA and Kruskal–Wallis tests were used to compare three or more groups. AUCs were calculated for each model, and pairwise comparisons were conducted using the DeLong test. NRI was used to assess the improvement in classification by BTSC-CAD-assisted radiologists relative to radiologists alone, with statistical significance determined using a two-sided z-test. A P < 0.05 was considered statistically significant. Supplementary information Supplementary_0421 Supplementary_0421