Radiomics for differential diagnosis of Bosniak II-IV renal masses via CT imaging Authors: Xun Zhao, Ye Yan, Wanfang Xie, Zijian Qin, Litao Zhao, Cheng Liu, Shudong Zhang, Jiangang Liu, Lulin Ma Published: 2024 ABSTRACT AbstractRationale and ObjectivesThe management of complex renal cysts is guided by the Bosniak classification system, which may be inadequate for risk stratification of patients to determine the appropriate intervention. Radiomics models based on CT imaging may provide additional useful information.Materials and MethodsA total of 322 patients with Bosniak II-IV cysts were included in the study from January 2010 to December 2019. Contrast-enhanced CT scans were performed on all patients. ITK-snap was used for segmentation, and the PyRadiomics 3.0.1 package was used for feature extraction. The radiomics features were screened via the least absolute shrinkage and selection operator (LASSO) regression method. After feature selection, a logistic regression (LR) model, support vector machine (SVM) model and random forest (RF) model were constructed.ResultsIn the present study, 217 benign renal cysts (67.4%) and 105 cystic renal cell carcinomas (32.6%) were identified. According to the Bosniak classification, the sample included 179 (55.6%) Bosniak II cysts, 38 (11.8%) Bosniak IIF cysts, 44 (13.7%) Bosniak III cysts and 61 (18.9%) Bosniak IV cysts. A total of 1334 radiomics features were extracted from both unenhanced and cortical CT scans. After LASSO regression, all the models (LR, SVM and RF) showed satisfactory discrimination and reliability in both unenhanced and cortical CT scans (AUC > 0.950). In the Bosniak IIF-III subgroup analysis, the diagnostic accuracy of the LR model was very low for both the unenhanced and cortical scans. In contrast, the SVM model and RF model showed excellent and stable performance in classifying Bosniak IIF-III cysts. The AUCs of the models were all > 0.85, with a maximum of 0.941. The sensitivity, specificity, accuracy, and AUC of the RF model were 0.889, 0.913, 0.902, and 0.941, respectively.ConclusionOur data indicate that radiomics models can effectively distinguish between cystic renal cell carcinoma (cRCC) and complex renal cysts (Bosniak II-IV). Radiomics models may still have high diagnostic accuracy even for Bosniak IIF-III cysts that are clinically difficult to distinguish. However, external validation of these findings is still needed.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12885-024-13283-6.Keywords: Renal cysts, Cystic renal cell carcinoma, Radiomics, Differential diagnosis Introduction With improvements in imaging technology in recent years, an increasing number of cystic renal lesions can be detected and treated at an early stage. Approximately 8–15% of cystic renal lesions have complex manifestations [1], such as bleeding, calcification, septa, thickening of the capsule wall, irregular walls, enhancing walls or septa and nodules, which complicate the differentiation of benign cystic renal lesions and cystic renal cell carcinoma (cRCC). Bosniak first proposed the CT-based classification of cystic renal lesions in 1986; this classification method has been developed for more than 30 years and has become the most widely used classification system for cystic renal lesions [2–6]. Bosniak classified cystic renal lesions into five classes: I, II, IIF, III and IV. Each class has different malignant probabilities and different suggestions for clinical management. A Bosniak I mass is considered a benign simple renal cyst and does not require follow-up, whereas Bosniak II-IV cysts are complex cystic renal lesions that may require urological consultation for further treatment. However, the accuracy of the Bosniak classification is insufficient. A meta-analysis revealed that the Bosniak classification had a moderate specificity of only 74% [7]. An incorrect dependence on the Bosniak classification may lead to unexpected consequences, such as excessive surgical treatment and potential neoplastic transplantation. According to Schoots’ study [8], almost 50% of patients with Bosniak III cysts undergo unnecessary surgeries. Therefore, developing a better method to distinguish between cRCC and benign complex renal cysts (Bosniak II-IV) is crucial for clinical management. Radiomics was proposed by Lambin in 2012 [9] and is defined as the process of converting medical images into high-dimensional, mineable data via high-throughput extraction of quantitative features, followed by subsequent data analysis for decision support. Radiomics has been widely applied in renal cell carcinoma (RCC) [10] but is rarely used in the identification of complex renal cysts. Therefore, the aim of this study was to develop a radiomics model to differentiate between cRCC and complex renal cysts with the hope of improving the accuracy of differentiation. Materials and methods Patients This retrospective study included consecutive patients diagnosed with complex renal cysts who underwent renal cyst unroofing, nephron-sparing surgery and radical nephrectomy from January 2010 to December 2019. This study protocol was approved by the ethics committee of our hospital (IRB00006761-M2022169). The inclusion criteria were as follows: (1) unenhanced and cortical CT images acquired within 2 weeks before surgery; (2) Bosniak II-IV cysts; and (3) completed surgery with definite postoperative pathology. The exclusion criteria were as follows: (1) incomplete medical records; (2) low-quality CT images; and (3) Bosniak I cysts. Pathology was based on the 2016 WHO classification of renal cell carcinoma and reviewed by two experienced pathologists [11]. The flowchart on the patient selection and study design were shown in Supplementary Materials 1. A total of 322 patients were ultimately included in the study. The patients were randomly divided into 4 sets for 4-fold cross-validation. During the feature selection and model construction in each fold, three sets were used as a training set, and one set was used as a testing set. Each testing set was totally independent from each training set. Additionally, subgroup analysis was performed in 82 patients with Bosniak IIF-III cysts. Bosniak classification The Bosniak classification used in this study was based on the updated Bosniak classification of 2019 [6], which was evaluated and reviewed by experienced radiologists and urologists. Examples of Bosniak I-IV cysts are shown in Fig. 1. Examples of Bosniak I-IV cysts. A: Bosniak I; B: Bosniak II; C: Bosniak IIF; D: Bosniak III; E Bosniak IV CT acquisition Abdominal contrast-enhanced CT scans were obtained from all participants using a 64-slice scanner (GE Discovery HD 750, GE Healthcare). The participants were asked to abstain from food for 6–8 h prior to the examination. The fixed acquisition parameters were as follows: 120 kV, automatic tube current modulation technique; detector collimation: 64 × 0.625 mm; matrix: 512 × 512; reconstruction slice thickness: 5 mm; and standard convolution kernel. Nonionic contrast medium (iopromide, Ultravist 370 mg/mL; Bayer) was intravenously administered at a dose of 1.5 mL/kg body weight and a rate of 3 mL/s. The scan was sequenced into three phases, namely, the cortical, medullary, and excretory phases, which were initiated at 12, 37, and 170 s after injection, respectively. Segmentation Digital Imaging and Communications in Medicine (DICOM) images were exported from the picture archiving and communication system (PACS) after desensitization. The segmentation of the cysts was performed by a resident with 3 years of experience and confirmed by a professor with 30 years of experience via ITK-Snap (version 3.8.0). A fully manual slice-by-slice segmentation was performed without the use of any threshold or semiautomatic tools. The segmentation was guided by the following principles: (1) All segmentations were executed on the axial plane, progressing slice by slice from the initial to the final cyst image to delineate the volume of interest (VOI). (2) Segmentation was performed along the edge of the cyst as much as possible. (3) The cortical images served as a reference for the unenhanced scan images, with an effort to ensure consistency between the segmentation areas of unenhanced and cortical CT scans. Radiomics feature extraction Radiomics features were extracted from both the original CT images and the transformed CT images with the PyRadiomics 3.0.1 package. The extracted features are similar to those used in our previous research [12]. In total, 1334 features were extracted from each patient. The calculation feature list is presented in Supplementary Table 1. Feature selection and model construction Before feature selection, the z-score normalization was used to normalize the radiomics features of training set and then transform those of the testing set. Then least absolute shrinkage and selection operator (LASSO) regression was subsequently used for feature selection. The LassoCV function from scikit-learn (version 1.0.1) was implemented for lasso feature selection. The lambda value of LASSO model was selected from a geometric sequence which contained 50 numbers on a range of [10 − 3, 103]. The selected lambda value was shown in Supplementary Materials 2. The Lasso model was optimized using leave-one-out-cross-validation (LOOCV) on the training dataset. Logistic regression (LR), support vector machine (SVM) and random forest (RF) models were constructed. The construction of the models was performed with the Scikit-Learn 1.0.1 package. A grid search method was used to select the best hyperparameters of the machine learning models in this study. For the optimization of LR models, a grid search approach with 5-fold cross-validation on the training set was applied for selecting the optimal C from 2i, where i was an integer from the range of [-8, 11]. For the optimization of SVM models, a grid search approach with 5-fold cross-validation on the training set was applied for selecting the optimal C from a geometric sequence which contained 10 numbers on a range of [2 − 1, 23], and gamma from a geometric sequence which contained 50 numbers on a range of [2–4, 2]. For the optimization of RF models, a grid search approach with 5-fold cross-validation on the training set was applied for selecting the optimal estimator from {200, 250, 300}, criterion from {gini, entropy}, max features from {3, 5}, max depth from {10, 20}, and min samples split from {2, 4}. The selected hyperparameters of the models can be seen in Supplementary Materials 2. The accuracy, sensitivity, specificity and area under the receiver operating characteristic curve (AUC) of the models were calculated separately. Statistical analysis Statistical analysis was performed with SPSS version 24.0 and Python version 3.8. For clinical data, categorical variables were analysed via the chi-square test, and continuous variables were analysed via the t test or Mann‒Whitney U test. The average AUC, accuracy, sensitivity, and specificity of each model were calculated to assess the diagnostic efficacy of the model. The AUC, accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) with 95% confidence intervals (CI) were provided. A two-tailed P value < 0.05 was considered to indicate statistical significance. Results Clinical characteristics A total of 322 patients were analysed, including 242 patients in the training cohort and 80 patients in the test cohort. The clinical characteristics of the patients are shown in Table 1. Among the 322 patients, there were 209 (64.9%) males and 113 (35.1%) females; 217 had benign cysts (67.4%) and 105 had cystic renal cell carcinoma (32.6%); and 179 (55.6%) had Bosniak II cysts, 38 (11.8%) had Bosniak IIF cysts, 44 (13.7%) had Bosniak III cysts and 61 (18.9%) had Bosniak IV cysts. There were significant differences in age (P < 0.001), smoking history (P = 0.004), and Bosniak classification (P < 0.001) between cRCC patients and those with benign renal cysts. There were no significant differences in other characteristics between the two groups of patients. Table 1Characteristics of 322 patients with Bosniak II-IV massesCharacteristicsPatients with benign renal cysts(n = 217)Patients with cystic renal cell carcinoma(n = 105)P valueAge (years)60.00(50.00–68.00)52.00(46.00–61.00)<0.001*BMI (kg/m2)24.72(22.57–27.10)24.44(22.66–26.30)0.484*WBC (109/L)5.80(4.90–6.82)5.86(5.10–7.09)0.364*HGB (g/L)141.00(133.00-152.00)142.00(128.00-153.00)0.861*PLT (109/L)215.00(175.00-255.00)212.00(180.00-253.00)0.709*Cr (µmol/L)78.00(71.00–89.00)82.00(69.00–88.00)0.709*Sex0.299Male137(63.1%)72(68.6%)Female80(36.9%)33(31.4%)Symptom0.076Yes75(34.9%)26(24.8%)No142(65.1%)79(75.2%)Smoking history0.004Yes25(11.5%)25(23.8%)No192(88.5%)80(76.2%)Side0.239Left104(48.2%)43(41.0%)Right113(51.8%)62(59.0%)Bosniak classification<0.001II166(76.5)13(12.4%)IIF33(15.2)5(4.8%)III13(6.0)31(29.5%)IV5(2.3%)56(53.3%) Characteristics of 322 patients with Bosniak II-IV masses Radiomics features and models After LASSO regression, 40, 37, 40 and 37 features in the unenhanced scan images were selected (Supplementary Table 2). The results of the three models were very good (Table 2). In the LR model, the average AUC was 0.961 ± 0.016; the average accuracy was 0.919 ± 0.040; the average sensitivity was 0.943 ± 0.033; and the average specificity was 0.908 ± 0.057. In the SVM model, the average AUC was 0.957 ± 0.017, the average accuracy was 0.913 ± 0.019, the average sensitivity was 0.849 ± 0.062, and the average specificity was 0.945 ± 0.023. In the RF model, the average AUC was 0.973 ± 0.009, the average accuracy was 0.928 ± 0.024, the average sensitivity was 0.924 ± 0.059, and the average specificity was 0.931 ± 0.040. Table 2Diagnostic results of the prediction models based on unenhanced imagesIndicators40 features37 features40 features37 featuresmean ± SDLRAccuracy0.963[0.914-1.000]0.951[0.901–0.988]0.863[0.787–0.925]0.900[0.838–0.963]0.919 ± 0.040Sensitivity0.962[0.864-1.000]0.963[0.875-1.000]0.962[0.870-1.000]0.885[0.741-1.000]0.943 ± 0.033Specificity0.964[0.902-1.000]0.944[0.872-1.000]0.815[0.709–0.911]0.907[0.824–0.979]0.908 ± 0.057PPV0.926[0.800-1.000]0.897[0.759-1.000]0.714[0.553–0.853]0.821[0.667–0.955]0.839 ± 0.082NPV0.981[0.800-1.000]0.981[0.759-1.000]0.978[0.553–0.853]0.942[0.667–0.955]0.970 ± 0.016AUC0.968[0.907-1.000]0.978[0.900-0.991]0.936[0.820–0.944]0.962[0.816–0.959]0.961 ± 0.016SVMAccuracy0.938[0.877–0.988]0.889[0.840–0.963]0.900[0.825–0.963]0.925[0.863–0.975]0.913 ± 0.019Sensitivity0.885[0.750-1.000]0.741[0.600-0.926]0.885[0.727-1.000]0.885[0.750-1.000]0.849 ± 0.062Specificity0.964[0.902-1.000]0.963[0.907-1.000]0.907[0.830–0.981]0.944[0.880-1.000]0.945 ± 0.023PPV0.920[0.774-1.000]0.909[0.773-1.000]0.821[0.655–0.96]0.885[0.750-1.000]0.884 ± 0.038NPV0.946[0.774-1.000]0.881[0.773-1.000]0.942[0.655–0.96]0.944[0.750-1.000]0.928 ± 0.027AUC0.946[0.847–0.983]0.938[0.781–0.945]0.962[0.810–0.967]0.982[0.839–0.974]0.957 ± 0.017RFAccuracy0.951[0.901–0.988]0.938[0.877–0.988]0.887[0.812–0.95]0.925[0.863–0.975]0.928 ± 0.024Sensitivity0.962[0.864-1.000]0.852[0.692–0.967]0.962[0.870-1.000]0.885[0.750-1.000]0.924 ± 0.059Specificity0.945[0.875-1.000]0.981[0.940-1.000]0.852[0.754–0.938]0.944[0.877-1.000]0.931 ± 0.040PPV0.893[0.765-1.000]0.958[0.850-1.000]0.758[0.600-0.889]0.885[0.741-1.000]0.873 ± 0.072NPV0.981[0.765-1.000]0.930[0.850- 1.000]0.979[0.600-0.889]0.944[0.741-1.000]0.958 ± 0.022AUC0.973[0.895–0.992]0.986[0.834–0.980]0.961[0.841–0.959]0.973[0.839–0.976]0.973 ± 0.009 Diagnostic results of the prediction models based on unenhanced images 0.963 [0.914-1.000] 0.951 [0.901–0.988] 0.863 [0.787–0.925] 0.900 [0.838–0.963] 0.962 [0.864-1.000] 0.963 [0.875-1.000] 0.962 [0.870-1.000] 0.885 [0.741-1.000] 0.964 [0.902-1.000] 0.944 [0.872-1.000] 0.815 [0.709–0.911] 0.907 [0.824–0.979] 0.926 [0.800-1.000] 0.897 [0.759-1.000] 0.714 [0.553–0.853] 0.821 [0.667–0.955] 0.981 [0.800-1.000] 0.981 [0.759-1.000] 0.978 [0.553–0.853] 0.942 [0.667–0.955] 0.968 [0.907-1.000] 0.978 [0.900-0.991] 0.936 [0.820–0.944] 0.962 [0.816–0.959] 0.938 [0.877–0.988] 0.889 [0.840–0.963] 0.900 [0.825–0.963] 0.925 [0.863–0.975] 0.885 [0.750-1.000] 0.741 [0.600-0.926] 0.885 [0.727-1.000] 0.885 [0.750-1.000] 0.964 [0.902-1.000] 0.963 [0.907-1.000] 0.907 [0.830–0.981] 0.944 [0.880-1.000] 0.920 [0.774-1.000] 0.909 [0.773-1.000] 0.821 [0.655–0.96] 0.885 [0.750-1.000] 0.946 [0.774-1.000] 0.881 [0.773-1.000] 0.942 [0.655–0.96] 0.944 [0.750-1.000] 0.946 [0.847–0.983] 0.938 [0.781–0.945] 0.962 [0.810–0.967] 0.982 [0.839–0.974] 0.951 [0.901–0.988] 0.938 [0.877–0.988] 0.887 [0.812–0.95] 0.925 [0.863–0.975] 0.962 [0.864-1.000] 0.852 [0.692–0.967] 0.962 [0.870-1.000] 0.885 [0.750-1.000] 0.945 [0.875-1.000] 0.981 [0.940-1.000] 0.852 [0.754–0.938] 0.944 [0.877-1.000] 0.893 [0.765-1.000] 0.958 [0.850-1.000] 0.758 [0.600-0.889] 0.885 [0.741-1.000] 0.981 [0.765-1.000] 0.930 [0.850- 1.000] 0.979 [0.600-0.889] 0.944 [0.741-1.000] 0.973 [0.895–0.992] 0.986 [0.834–0.980] 0.961 [0.841–0.959] 0.973 [0.839–0.976] After LASSO regression, 12, 31, 23 and 28 features in the cortical phase images were selected (Supplementary Table 3). The results of these three models were better than those of the models based on unenhanced scan images (Table 3). In the LR model, the average AUC was 0.977 ± 0.001, the average accuracy was 0.922 ± 0.010, the average sensitivity was 0.914 ± 0.032, and the average specificity was 0.926 ± 0.000. In the SVM model, the average AUC was 0.969 ± 0.005, the average accuracy was 0.904 ± 0.013, the average sensitivity was 0.848 ± 0.044, and the average specificity was 0.931 ± 0.015. In the RF model, the average AUC was 0.979 ± 0.005, the average accuracy was 0.916 ± 0.006, the average sensitivity was 0.857 ± 0.042, and the average specificity was 0.945 ± 0.023. Table 3Diagnostic results of the prediction models based on cortical imagesIndicators12 features31 features23 features28 featuresmean ± SDLRAccuracy0.914[0.840–0.975]0.926[0.864–0.975]0.938[0.875–0.988]0.912[0.850–0.963]0.922 ± 0.010Sensitivity0.885[0.739-1.000]0.926[0.808-1.000]0.962[0.870-1.000]0.885[0.739-1.000]0.914 ± 0.032Specificity0.927[0.854–0.984]0.926[0.845–0.983]0.926[0.850–0.983]0.926[0.847–0.983]0.926 ± 0.000PPV0.852[0.696–0.967]0.862[0.714–0.967]0.862[0.720–0.966]0.852[0.696–0.968]0.857 ± 0.005NPV0.944[0.696–0.967]0.962[0.714–0.967]0.980[0.720–0.966]0.943[0.696–0.968]0.957 ± 0.015AUC0.978[0.821–0.972]0.975[0.852–0.982]0.975[0.887–0.983]0.981[0.827–0.97]0.977 ± 0.001SVMAccuracy0.901[0.827–0.963]0.889[0.815–0.951]0.900[0.825–0.963]0.925[0.863–0.975]0.904 ± 0.013Sensitivity0.846[0.684–0.964]0.778[0.600–0.92]0.885[0.737-1.000]0.885[0.737-1.000]0.848 ± 0.044Specificity0.927[0.854–0.984]0.944[0.878-1.000]0.907[0.825–0.981]0.944[0.875-1.000]0.931 ± 0.015PPV0.846[0.688–0.966]0.875[0.708-1.000]0.821[0.667–0.957]0.885[0.742-1.000]0.857 ± 0.025NPV0.927[0.688–0.966]0.895[0.708-1.000]0.942[0.667–0.957]0.944[0.742-1.000]0.927 ± 0.020AUC0.961[0.796–0.962]0.974[0.769–0.941]0.974[0.811–0.963]0.967[0.837–0.975]0.969 ± 0.005RFAccuracy0.926[0.864–0.975]0.914[0.852–0.975]0.912[0.850–0.963]0.912[0.850–0.963]0.916 ± 0.006Sensitivity0.846[0.682–0.963]0.815[0.643–0.957]0.923[0.808-1.000]0.846[0.680–0.964]0.857 ± 0.042Specificity0.964[0.902-1.000]0.963[0.895-1.000]0.907[0.830–0.98]0.944[0.875-1.000]0.945 ± 0.023PPV0.917[0.773-1.000]0.917[0.773-1.000]0.828[0.667–0.955]0.880[0.727-1.000]0.885 ± 0.036NPV0.930[0.773-1.000]0.912[0.773-1.000]0.961[0.667–0.955]0.927[0.727-1.000]0.932 ± 0.018AUC0.979[0.820–0.972]0.982[0.800-0.963]0.970[0.847–0.973]0.984[0.804–0.961]0.979 ± 0.005 Diagnostic results of the prediction models based on cortical images 0.914 [0.840–0.975] 0.926 [0.864–0.975] 0.938 [0.875–0.988] 0.912 [0.850–0.963] 0.885 [0.739-1.000] 0.926 [0.808-1.000] 0.962 [0.870-1.000] 0.885 [0.739-1.000] 0.927 [0.854–0.984] 0.926 [0.845–0.983] 0.926 [0.850–0.983] 0.926 [0.847–0.983] 0.852 [0.696–0.967] 0.862 [0.714–0.967] 0.862 [0.720–0.966] 0.852 [0.696–0.968] 0.944 [0.696–0.967] 0.962 [0.714–0.967] 0.980 [0.720–0.966] 0.943 [0.696–0.968] 0.978 [0.821–0.972] 0.975 [0.852–0.982] 0.975 [0.887–0.983] 0.981 [0.827–0.97] 0.901 [0.827–0.963] 0.889 [0.815–0.951] 0.900 [0.825–0.963] 0.925 [0.863–0.975] 0.846 [0.684–0.964] 0.778 [0.600–0.92] 0.885 [0.737-1.000] 0.885 [0.737-1.000] 0.927 [0.854–0.984] 0.944 [0.878-1.000] 0.907 [0.825–0.981] 0.944 [0.875-1.000] 0.846 [0.688–0.966] 0.875 [0.708-1.000] 0.821 [0.667–0.957] 0.885 [0.742-1.000] 0.927 [0.688–0.966] 0.895 [0.708-1.000] 0.942 [0.667–0.957] 0.944 [0.742-1.000] 0.961 [0.796–0.962] 0.974 [0.769–0.941] 0.974 [0.811–0.963] 0.967 [0.837–0.975] 0.926 [0.864–0.975] 0.914 [0.852–0.975] 0.912 [0.850–0.963] 0.912 [0.850–0.963] 0.846 [0.682–0.963] 0.815 [0.643–0.957] 0.923 [0.808-1.000] 0.846 [0.680–0.964] 0.964 [0.902-1.000] 0.963 [0.895-1.000] 0.907 [0.830–0.98] 0.944 [0.875-1.000] 0.917 [0.773-1.000] 0.917 [0.773-1.000] 0.828 [0.667–0.955] 0.880 [0.727-1.000] 0.930 [0.773-1.000] 0.912 [0.773-1.000] 0.961 [0.667–0.955] 0.927 [0.727-1.000] 0.979 [0.820–0.972] 0.982 [0.800-0.963] 0.970 [0.847–0.973] 0.984 [0.804–0.961] Bosniak IIF and Bosniak III cysts are more difficult to distinguish from cRCC than cysts of other Bosniak classifications are. We conducted a subgroup analysis of Bosniak IIF and Bosniak III cysts. After LASSO regression, 21, 12, 28 and 60 features in the unenhanced scan images were screened (Supplementary Table 4). The accuracy of the LR model in classifying Bosniak IIF-III cysts was lower than that for classifying Bosniak II-IV cysts, and the results fluctuated greatly during cross-validation, indicating that the model was unstable (Table 4). In contrast, the SVM model and RF model also showed excellent and stable performance in classifying Bosniak IIF-III cysts, indicating that the radiomics models could effectively classify Bosniak IIF-III cysts, which are difficult to distinguish clinically. Table 4Diagnostic results of the prediction models based on unenhanced images in the Bosniak IIF-III subgroupIndicators21 features12 features28 features60 featuresmean ± SDLRAccuracy0.429[0.190–0.619]0.810[0.619–0.952]0.450[0.200–0.65]0.900[0.750-1.000]0.647 ± 0.210Sensitivity1.000[1.000–1.000]0.667[0.250–0.909]1.000[1.000–1.000]1.000[1.000–1.000]0.917 ± 0.144Specificity0.000[0.000–0.000]0.917[0.750-1.000]0.000[0.000–0.000]0.818[0.583-1.000]0.434 ± 0.435PPV0.429[0.190–0.619]0.857[0.500-1.000]0.450[0.200–0.65]0.818[0.545-1.000]0.638 ± 0.200NPVNan0.786[0.500-1.000]Nan1.000[0.545-1.000]NanAUC0.500[0.500–0.5]0.898[0.582–0.938]0.500[0.500–0.5]0.960[0.792-1.000]0.715 ± 0.216SVMAccuracy0.905[0.857-1.000]0.810[0.619–0.952]0.800[0.600–0.95]0.900[0.750-1.000]0.854 ± 0.049Sensitivity0.889[1.000–1.000]0.778[0.429-1.000]0.667[0.273-1.000]1.000[1.000–1.000]0.833 ± 0.124Specificity0.917[0.727-1.000]0.833[0.600-1.000]0.909[0.714-1.000]0.818[0.583-1.000]0.869 ± 0.044PPV0.889[0.643-1.000]0.778[0.400-1.000]0.857[0.500-1.000]0.818[0.545-1.000]0.835 ± 0.042NPV0.917[0.643-1.000]0.833[0.400-1.000]0.796[0.500-1.000]1.000[0.545-1.000]0.886 ± 0.079AUC0.880[0.864-1.000]0.898[0.575–0.962]0.838[0.567–0.938]0.970[0.792-1.000]0.897 ± 0.048RFAccuracy0.905[0.857-1.000]0.905[0.762-1.000]0.900[0.750-1.000]0.900[0.750-1.000]0.902 ± 0.002Sensitivity0.889[1.000–1.000]0.889[0.600-1.000]0.889[0.615-1.000]0.889[0.571-1.000]0.889 ± 0.000Specificity0.917[0.727-1.000]0.917[0.750-1.000]0.909[0.714-1.000]0.909[0.727-1.000]0.913 ± 0.004PPV0.889[0.643-1.000]0.889[0.600-1.000]0.889[0.600-1.000]0.889[0.600-1.000]0.889 ± 0.000NPV0.917[0.643-1.000]0.917[0.600-1.000]0.909[0.600-1.000]0.909[0.600-1.000]0.913 ± 0.004AUC0.940[0.864-1.000]0.944[0.738-1.000]0.909[0.747-1.000]0.970[0.729-1.000]0.941 ± 0.022Note: Nan in the table means not a number Diagnostic results of the prediction models based on unenhanced images in the Bosniak IIF-III subgroup 0.429 [0.190–0.619] 0.810 [0.619–0.952] 0.450 [0.200–0.65] 0.900 [0.750-1.000] 1.000 [1.000–1.000] 0.667 [0.250–0.909] 1.000 [1.000–1.000] 1.000 [1.000–1.000] 0.000 [0.000–0.000] 0.917 [0.750-1.000] 0.000 [0.000–0.000] 0.818 [0.583-1.000] 0.429 [0.190–0.619] 0.857 [0.500-1.000] 0.450 [0.200–0.65] 0.818 [0.545-1.000] 0.786 [0.500-1.000] 1.000 [0.545-1.000] 0.500 [0.500–0.5] 0.898 [0.582–0.938] 0.500 [0.500–0.5] 0.960 [0.792-1.000] 0.905 [0.857-1.000] 0.810 [0.619–0.952] 0.800 [0.600–0.95] 0.900 [0.750-1.000] 0.889 [1.000–1.000] 0.778 [0.429-1.000] 0.667 [0.273-1.000] 1.000 [1.000–1.000] 0.917 [0.727-1.000] 0.833 [0.600-1.000] 0.909 [0.714-1.000] 0.818 [0.583-1.000] 0.889 [0.643-1.000] 0.778 [0.400-1.000] 0.857 [0.500-1.000] 0.818 [0.545-1.000] 0.917 [0.643-1.000] 0.833 [0.400-1.000] 0.796 [0.500-1.000] 1.000 [0.545-1.000] 0.880 [0.864-1.000] 0.898 [0.575–0.962] 0.838 [0.567–0.938] 0.970 [0.792-1.000] 0.905 [0.857-1.000] 0.905 [0.762-1.000] 0.900 [0.750-1.000] 0.900 [0.750-1.000] 0.889 [1.000–1.000] 0.889 [0.600-1.000] 0.889 [0.615-1.000] 0.889 [0.571-1.000] 0.917 [0.727-1.000] 0.917 [0.750-1.000] 0.909 [0.714-1.000] 0.909 [0.727-1.000] 0.889 [0.643-1.000] 0.889 [0.600-1.000] 0.889 [0.600-1.000] 0.889 [0.600-1.000] 0.917 [0.643-1.000] 0.917 [0.600-1.000] 0.909 [0.600-1.000] 0.909 [0.600-1.000] 0.940 [0.864-1.000] 0.944 [0.738-1.000] 0.909 [0.747-1.000] 0.970 [0.729-1.000] Note: Nan in the table means not a number After LASSO regression, 8, 25, 10 and 19 features in the cortical phase CT images were screened (Supplementary Table 5). Similar to the results with the unenhanced scan images, the LR model did not have discrimination ability, whereas both the SVM model and the RF model had good discriminative ability (Table 5). Table 5Diagnostic results of the prediction models based on cortical images in the Bosniak IIF-III subgroupIndicators8 features25 features10 features19 featuresmean ± SDLRAccuracy0.429[0.190–0.619]0.429[0.190–0.619]0.450[0.200–0.650]0.450[0.200–0.650]0.439 ± 0.010Sensitivity1.000[1.000–1.000]1.000[1.000–1.000]1.000[1.000–1.000]1.000[1.000–1.000]1.000 ± 0.000Specificity0.000[0.000–0.000]0.000[0.000–0.000]0.000[0.000–0.000]0.000[0.000–0.000]0.000 ± 0.000PPV0.429[0.190–0.619]0.429[0.190–0.619]0.450[0.200–0.65]0.450[0.200–0.65]0.439 ± 0.010NPV-----AUC0.500[0.500–0.5]0.500[0.500–0.5]0.500[0.500–0.5]0.500[0.500–0.5]0.500 ± 0.000SVMAccuracy0.810[0.667-1.000]0.810[0.619–0.952]0.950[0.850-1.000]0.850[0.650-1.000]0.855 ± 0.057Sensitivity0.778[0.600-1.000]0.667[0.250–0.917]1.000[1.000–1.000]0.889[0.571-1.000]0.834 ± 0.124Specificity0.833[0.583-1.000]0.917[0.750-1.000]0.909[0.714-1.000]0.818[0.583-1.000]0.869 ± 0.044PPV0.778[0.500-1.000]0.857[0.500-1.000]0.900[0.636-1.000]0.800[0.462-1.000]0.834 ± 0.048NPV0.833[0.500-1.000]0.786[0.500-1.000]1.00000[0.636-1.000]0.900[0.462-1.000]0.880 ± 0.080AUC0.861[0.679-1.000]0.935[0.571–0.938]0.939[0.857-1.000]0.960[0.656-1.000]0.924 ± 0.037RFAccuracy0.905[0.857-1.00000]0.905[0.762-1.000]0.800[0.700-1.000]0.900[0.750-1.000]0.877 ± 0.045Sensitivity0.778[0.600-1.000]0.889[0.600-1.000]0.667[0.400-1.000]0.889[0.571-1.000]0.806 ± 0.092Specificity1.000[1.000–1.000]0.917[0.750-1.000]0.909[0.727-1.000]0.909[0.714-1.000]0.934 ± 0.038PPV1.000[1.000–1.000]0.889[0.600-1.000]0.857[0.571-1.000]0.889[0.583-1.000]0.909 ± 0.054NPV0.857[1.000–1.000]0.917[0.600-1.000]0.769[0.571-1.000]0.909[0.583-1.000]0.865 ± 0.059AUC0.907[0.800- 1.000]0.972[0.738-1.000]0.929[0.637-1.000]0.949[0.726-1.000]0.939 ± 0.024 Diagnostic results of the prediction models based on cortical images in the Bosniak IIF-III subgroup 0.429 [0.190–0.619] 0.429 [0.190–0.619] 0.450 [0.200–0.650] 0.450 [0.200–0.650] 1.000 [1.000–1.000] 1.000 [1.000–1.000] 1.000 [1.000–1.000] 1.000 [1.000–1.000] 0.000 [0.000–0.000] 0.000 [0.000–0.000] 0.000 [0.000–0.000] 0.000 [0.000–0.000] 0.429 [0.190–0.619] 0.429 [0.190–0.619] 0.450 [0.200–0.65] 0.450 [0.200–0.65] 0.500 [0.500–0.5] 0.500 [0.500–0.5] 0.500 [0.500–0.5] 0.500 [0.500–0.5] 0.810 [0.667-1.000] 0.810 [0.619–0.952] 0.950 [0.850-1.000] 0.850 [0.650-1.000] 0.778 [0.600-1.000] 0.667 [0.250–0.917] 1.000 [1.000–1.000] 0.889 [0.571-1.000] 0.833 [0.583-1.000] 0.917 [0.750-1.000] 0.909 [0.714-1.000] 0.818 [0.583-1.000] 0.778 [0.500-1.000] 0.857 [0.500-1.000] 0.900 [0.636-1.000] 0.800 [0.462-1.000] 0.833 [0.500-1.000] 0.786 [0.500-1.000] 1.00000 [0.636-1.000] 0.900 [0.462-1.000] 0.861 [0.679-1.000] 0.935 [0.571–0.938] 0.939 [0.857-1.000] 0.960 [0.656-1.000] 0.905 [0.857-1.00000] 0.905 [0.762-1.000] 0.800 [0.700-1.000] 0.900 [0.750-1.000] 0.778 [0.600-1.000] 0.889 [0.600-1.000] 0.667 [0.400-1.000] 0.889 [0.571-1.000] 1.000 [1.000–1.000] 0.917 [0.750-1.000] 0.909 [0.727-1.000] 0.909 [0.714-1.000] 1.000 [1.000–1.000] 0.889 [0.600-1.000] 0.857 [0.571-1.000] 0.889 [0.583-1.000] 0.857 [1.000–1.000] 0.917 [0.600-1.000] 0.769 [0.571-1.000] 0.909 [0.583-1.000] 0.907 [0.800- 1.000] 0.972 [0.738-1.000] 0.929 [0.637-1.000] 0.949 [0.726-1.000] Discussion The differentiation of benign complex renal cysts and cystic renal cell carcinoma (cRCC) is a clinical challenge and is highly important for clinical treatment decision making. In recent years, radiomics has been widely used in RCC and has been shown to improve clinical diagnostic effectiveness [13]. In this study, we used a radiomics method combined with a machine learning method to identify benign complex renal cysts and cRCC and constructed three models: the LR, SVM, and RF models. These three models performed well in the classification of benign cysts and cRCCs, and the SVM and RF models performed well in the classification of Bosniak IIF-III cysts, which are difficult to distinguish clinically (ROC curves of all models are shown in Supplementary Figs. 1–12). The routine examinations used in clinical practice for detecting renal cystic masses include ultrasound, CT and MRI. Among these examinations, CT is considered the standard for the evaluation of renal cysts. Although the Bosniak classification for renal cysts has been widely accepted, it still has several limitations, such as poor differentiation of Bosniak IIF and III cysts. The updated Bosniak classification in 2019 included a limited number of image features and did not include indicators such as size and texture features [6]. Benign cysts, particularly Bosniak III cysts, are often overtreated when using the Bosniak classification. Approximately 50% of Bosniak III masses that were treated with surgery were found to be pathologically benign lesions. Although several studies [14, 15] have suggested that a multimodality approach (CT combined with contrast-enhanced ultrasound or MRI) is useful for diagnosing malignancy in patients with cystic renal masses, increasing the number of examinations can impose a greater burden on patients. Therefore, we attempted to improve the identification accuracy of complex renal cysts on CT images by using radiomics. The proportions of features extracted and selected in this study were essentially consistent with those in a previous study [16]. After LASSO regression, the ratio of the features included in the model to the total features did not exceed 1:5. The features in the model included fewer shape features and more first-order features and texture features, reflecting the heterogeneity of masses [17]. Among them, GLCM features were relatively common, accounting for the majority of cortical phase image features screened by LASSO regression. GLCM characterizes the texture of an image by calculating how often pairs of pixels with specific values and in a specified spatial relationship occur in an image. GLCM is closely related to tumour heterogeneity and can be used in the diagnosis of skin cancer [18], the differentiation of breast masses [19, 20], the diagnosis of osteosarcoma and the differentiation of brain tumour subtypes [21]. Our study also revealed that, compared with the LR model, the SVM and RF models are more effective at identifying benign complex renal cysts and cRCCs. For Bosniak IIF-III cysts, which are difficult to distinguish from cRCC in clinical practice, the LF model performs poorly, whereas the SVM and RF models perform well. There may be two reasons for this difference: a smaller sample size and similar image features with fewer differences between benign renal cysts and cRCCs; these factors may have a greater impact on the diagnostic performance of the LR model than on that of the other models. In other words, the SVM and RF models may be more suitable for this complex diagnosis. At present, studies on the application of radiomics in renal cysts are limited. Kim [22] et al. evaluated the utility of CT texture analysis in differentiating low-attenuation RCC from renal cysts on unenhanced CT images and reported that high entropy revealed via CT texture analysis may be used to differentiate low-attenuation RCC from cysts on unenhanced CT images. Miskin [23] et al. created a CT texture-based machine learning algorithm that distinguishes benign cysts from potential cRCCs. A total of 257 cystic renal masses were included in the study cohort. The AUCs for the RF, LR and SVM models were 0.88, 0.90 and 0.89, respectively. However, the purpose of this model is to reduce interreader variability when the Bosniak classification is applied without involving the differentiation of benign cysts and cRCC via pathology. He et al. [24] used deep learning combined with radiomics features to differentiate renal cysts, and the AUC of the model was 0.934. However, the number of patients was small (n = 159), and many Bosniak I cysts (n = 59) were included. Yu et al. [25] constructed a radiomics nomogram for the preoperative differentiation of benign and malignant cystic renal lesions and achieved good results (AUC = 0.962). However, they used medullary phase images, which is different from our method. Our data suggest that the application of radiomics in the differentiation of renal cysts has great potential. In this study, the results of cross-validation of the same model were basically consistent. For example, for the RF model, the AUC was > 0.95 for all cysts and > 0.9 for the Bosniak IIF-III cysts subgroup, indicating that the stability and diagnostic accuracy of the model are very high, which may help improve clinical practice. Our study has several limitations, such as its relatively small sample size, single-centre design without external validation, retrospective study design, and lack of inclusion of features from other imaging examinations, such as MRI and PET-CT, in the prediction models. The small number of rare cases in some classifications, such as Bosniak II cRCCs and Bosniak IV benign cysts, may have introduced bias. Furthermore, the inclusion of many features in the model poses a potential risk of overfitting. Conclusion Radiomics models can effectively distinguish between cystic renal cell carcinoma (cRCC) and benign complex renal cysts (Bosniak II-IV). The radiomics model still had high diagnostic efficacy even for Bosniak IIF-III cysts, which are commonly considered difficult to differentiate (AUC, 0.939 ± 0.024). Electronic supplementary material Below is the link to the electronic supplementary material. Supplementary Material 1 Supplementary Material 2 Supplementary Material 3 Supplementary Material 4 Acknowledgements We acknowledge Jing-Han Dong and Zheng-Hui Sun for their kind help with data collection, all the nurses in our center for their services, and the patients involved in this study. Abbreviations Volume of interest Logistic regression Support vector machine Random forest Area under the receiver operating characteristic curve Cystic renal cell carcinoma Author contributions Conception and design: X.Z., Y.Y., W.X. Provision of study material or patients: C.L., S.Z., L.M. Collection and assembly of data: X.Z., Y.Y., C.L., S.Z., L.M. Data analysis and interpretation: W.X., L.Z., J.L. Manuscript writing: X.Z., Y.Y., W.X., J.L., S.Z., L.M. Final approval of manuscript: All authors. Funding This work was supported by the National Nature Science Foundation of China (No. 82173385) and the National Science Foundation of Beijing (Z200027). Data availability The datasets used and analyzed during the study are available from the corresponding author on reasonable request. The ROC curves of the models could be found in supplementary materials. Declarations Ethical approval All procedures performed in studies involving human participants were in accordance with the Declaration of Helsinki. This study was approved by the Ethics Committee of Peking University Third Hospital (ethics approval number: IRB00006761-M2022169). Written informed consent was obtained from individual participants or their guardians. Consent for publication Not applicable. Competing interests The authors declare no competing interests. Consent to participate All the authors declare their participation in the study and have no conflicts of interest. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Xun Zhao, Ye Yan and Wanfang Xie contributed equally to this work. Contributor Information Shudong Zhang, Email: shootong@163.com. Jiangang Liu, Email: jgliu@buaa.edu.cn. Lulin Ma, Email: malulinpku@163.com. References Associated Data Supplementary Materials Supplementary Material 1 Supplementary Material 2 Supplementary Material 3 Supplementary Material 4 Data Availability Statement The datasets used and analyzed during the study are available from the corresponding author on reasonable request. The ROC curves of the models could be found in supplementary materials.