Diagnostic performance of deep learning models on ultrasound images for distinguishing benign from malignant ovarian cysts Authors: Weimei Li, Yuhua Xia, Yongzhao Li, Xing Wu, Lin Shi Published: 2026 ABSTRACT AbstractBackgroundOvarian cysts are a common pelvic disorder in women, and accurate differentiation between benign and malignant types is essential for guiding treatment decisions and prognostic evaluations. However, traditional ultrasound examinations heavily depend on the operator’s experience, introducing subjectivity and diagnostic inconsistencies. In recent years, deep learning technologies have demonstrated strong potential in intelligent medical imaging diagnostics, offering innovative solutions for automated and precise classification of ovarian cysts.ResultsCompared to subjective evaluations by senior ultrasound physicians (accuracy: 76.5%) and the O-RADS classification system (accuracy: 87.8%), the DenseNet121 model demonstrated a superior Area Under the Curve (AUC: 0.913 vs. 0.858, P < 0.05), indicating stronger overall discriminative ability.ConclusionsDeep learning models based on ultrasound images can effectively address noise and feature complexity in such imaging, enabling high-precision classification of benign and malignant ovarian cysts. These models hold strong potential for clinical adoption, providing physicians with objective and reliable decision-making support.Keywords: Ovarian cyst, Deep learning, O-RADS, Feature extraction Background In recent years, deep learning technologies have demonstrated strong potential in intelligent medical imaging diagnostics [1], offering innovative solutions for automated and precise classification of ovarian cysts [2].Ovarian cysts represent a prevalent gynecological condition in women [3], and accurate differentiation between benign and malignant types is critical for developing individualized treatment strategies and improving patient prognosis [4]. However, clinical diagnosis remains challenging, as some patients are preoperatively suspected of malignancy but postoperatively confirmed as benign, leading to overtreatment and psychological burden. Ultrasound examination [5], due to its non-invasive, convenient, and cost-effective nature, serves as the preferred imaging modality for initial screening and follow-up of ovarian cysts. Nevertheless, diagnostic outcomes are heavily influenced by the operator’s experience and subjective judgment, resulting in substantial inter-observer variability. To enhance diagnostic consistency and accuracy, the Ovarian-Adnexal Reporting and Data System (O-RADS) has been internationally proposed [6], standardizing ultrasound features into risk stratification categories to provide clinical references [7]. However, O-RADS exhibits uncertainty in diagnosing category 4 ovarian masses [8], with approximately half of those classified as malignant later confirmed as benign postoperatively, indicating a potential issue of overdiagnosis. Additionally, while serum tumor markers can serve as supplementary indicators [9], their sensitivity and specificity are limited when used in isolation. In recent years, deep learning has achieved remarkable advancements in medical image analysis [10], enabling automated learning and extraction of high-dimensional image features while minimizing biases from manual feature selection [11]. This offers novel approaches for intelligent diagnosis of complex images. Existing studies have demonstrated that convolutional neural networks (CNNs) and their variants achieve excellent classification performance in lesion identification from ultrasound images of the breast, thyroid, and prostate [12].However, research on deep learning for differentiating benign and malignant ovarian cysts remains relatively limited [13], with a lack of direct comparative studies against O-RADS and clinical expert assessments. Ovarian cysts are a common pelvic disorder in women [14].This study, based on ultrasound images from pathologically confirmed ovarian cysts [15], constructed multiple end-to-end deep learning models [16, 17], and Vision Transformer (ViT). Results indicated that the DenseNet121 model outperformed the others, achieving area under the curve (AUC) values of 0.914 and 0.913 on the training and validation sets, respectively. It also demonstrated superior accuracy, sensitivity, and specificity on the validation set. Grad-CAM visualizations further validated the model’s reliability in extracting features from lesion regions, showing high consistency with clinical lesion distributions. Furthermore, the diagnostic efficacy of this model was compared with subjective assessments by senior ultrasound physicians and O-RADS grading results [18], aiming to explore the application value of deep learning in differentiating benign and malignant ovarian cysts and to provide clinicians with more objective and reliable decision-support tools [19]. Compared to subjective evaluations by senior ultrasound physicians (accuracy: 76.5%) and the O-RADS classification system [20] (accuracy: 87.8%), the DenseNet121 model exhibited significant advantages in overall diagnostic efficacy (P < 0.05). Methods Study population This retrospective study included 327 patients with ovarian cysts who underwent surgery and received pathological confirmation at our institution from January 2023 to January 2025. All patients had complete preoperative two-dimensional ultrasound images and clinical data, comprising 249 benign cases and 78 malignant cases. The dataset was randomly divided into a training set (196 cases) and a validation set (131 cases) in a 6:4 ratio. The training set (n = 196) was used for model parameter learning and hyperparameter optimization, while the validation set (n = 131) was used to evaluate the final generalization performance of the model.Patients who were pregnant, lactating, or had a history of severe cardiac disease were excluded. Terminology Note: It should be noted that in this study, the independent dataset used for final performance evaluation is referred to as the “validation set,” which aligns with the conventional terminology in medical imaging diagnostic research. In some technical tables (e.g., model performance comparison tables), this same dataset is labeled as “Test.” Both terms refer to the identical set of independent samples that were not involved in model training or parameter tuning. Data collection and preprocessing All patients underwent pelvic ultrasound examinations using the GE Voluson E8 color Doppler ultrasound system, equipped with a transvaginal probe (RIC5-9-D) and an abdominal probe (RAB2-5-D). Images were acquired in transverse, sagittal, and magnified lesion views, documenting details such as lesion number, size, morphology, borders, wall thickness, papillary projections, septations, blood flow signals, resistance index, and ascites. Images were stored in DICOM format, anonymized by removing patient information, and imported into the deep learning framework [21]. Preprocessing involved resizing to 224 × 224 pixels, pixel value normalization, and data augmentation techniques including random rotation, translation, and flipping to enhance model generalization. Gold Standard: The final diagnosis for all cases was confirmed by surgical pathology, which served as the gold standard. In accordance with clinical practice, for the purpose of binary classification, borderline ovarian tumors (BOTs) were analyzed as malignant lesions. This approach was adopted because these tumors require surgical staging and follow-up management similar to malignant tumors, and including them in the malignant group helps the model learn features from more clinically challenging cases. Model construction and training Four end-to-end deep learning models—DenseNet121, ResNet34, GoogLeNet [22], and Vision Transformer (ViT)—were selected as candidate networks. All models were initialized with ImageNet pre-trained weights and fine-tuned via transfer learning on the training set. Binary cross-entropy was used as the loss function, with Adam as the optimizer, an initial learning rate of 1e-4, a batch size of 32, and training for 100 epochs. AUC was monitored on the validation set, with an early stopping strategy implemented to prevent overfitting. Performance evaluation metrics Postoperative pathological diagnosis served as the gold standard. Receiver operating characteristic (ROC) curves were plotted, and the area under the curve (AUC) with 95% confidence intervals was calculated. Additional metrics, including accuracy, sensitivity, specificity, precision, recall, and F1 score, were computed to assess classification performance. Grad-CAM was applied to the optimal model for visualization analysis, examining the consistency between the model’s attention regions and clinical lesion distributions. Statistical analysis Statistical analyses were performed using SPSS 26.0 and Python. Continuous data are presented as mean ± standard deviation (x ± s), and categorical data as percentages (%). AUC comparisons between models were conducted using the DeLong test, while classification metrics such as accuracy were compared via chi-square or McNemar tests [23]. A p-value < 0.05 was considered statistically significant. Results Patient baseline characteristics This study included a total of 327 patients with pathologically confirmed ovarian cysts, comprising 249 benign cases (76.15%) and 78 malignant cases (23.85%). The cohort was randomly divided into a training set of 196 cases (149 benign, 47 malignant) and a validation set of 131 cases (100 benign, 31 malignant) in a 6:4 ratio. Among them, benign lesions encompassed 14 pathological subtypes, with endometriomas (71 cases), serous/mucinous cystadenomas (55 cases), and dermoid cysts (44 cases) being the most common. Malignant lesions encompassed 5 pathological subtypes, predominantly serous/mucinous carcinomas (43 cases) and adenocarcinomas (20 cases) (Tables 1 and 2). This distribution is consistent with the epidemiological characteristics of ovarian lesions in our institution. No statistically significant differences were observed between the two groups in terms of age, lesion size, or pathological type distribution (P > 0.05), ensuring comparability. Table 1Subtype distribution and case numbers of benign lesionsSubtype Distribution of Benign CasescasesChocolate cyst71Serous/mucinous cystadenoma55Dermoid cyst44Simple/non-simple cyst25Follicle/corpus luteum22Ovarian fibroma6Borderline papillary tumor5Fibroblast proliferation4Thecomatous hyperplasia4Hemorrhagic cyst4Theca fibroma4Epidermoid cyst2Ovarian Brenner tumor with borderline mucinous cystadenoma2Adult granulosa cell tumor1 Subtype distribution and case numbers of benign lesions Table 2Subtype distribution and case numbers of malignant lesionsSubtype Distribution of Malignant CasescasesSerous/mucinous carcinoma43Adenocarcinoma20Clear cell carcinoma7Metastatic carcinoma6Metastatic endometrioid carcinoma2 Subtype distribution and case numbers of malignant lesions Based on the above distribution, we provide an objective description of the imaging features of different pathological subtypes: Endometriomas (71 cases): Often present with intra-cystic punctate echoes, thickened cyst walls, or multilocular structures, with relatively complex sonographic appearances. Endometriomas (71 cases): Often present with intra-cystic punctate echoes, thickened cyst walls, or multilocular structures, with relatively complex sonographic appearances. Serous/mucinous cystadenomas (55 cases): Typically appear as unilocular or multilocular cystic structures with smooth cyst walls and good internal transparency. Serous/mucinous cystadenomas (55 cases): Typically appear as unilocular or multilocular cystic structures with smooth cyst walls and good internal transparency. Dermoid cysts (44 cases): May exhibit characteristic features such as fat-fluid levels, calcifications, or hairball signs. Dermoid cysts (44 cases): May exhibit characteristic features such as fat-fluid levels, calcifications, or hairball signs. Serous/mucinous carcinomas (43 cases): Often present as cystic-solid masses with irregularly thickened cyst walls and papillary projections. Serous/mucinous carcinomas (43 cases): Often present as cystic-solid masses with irregularly thickened cyst walls and papillary projections. Clear cell carcinomas (7 cases) and borderline tumors (7 cases): Have limited case numbers and exhibit imaging features that fall between benign and malignant categories. Clear cell carcinomas (7 cases) and borderline tumors (7 cases): Have limited case numbers and exhibit imaging features that fall between benign and malignant categories. Comparison of classification performance among models The classification performance of the four deep learning models is presented in Table 3. The DenseNet121 model demonstrated the highest AUC and overall classification performance in both the training and validation sets, with a training AUC of 0.914 (95% CI: 0.8733–0.9548) and a validation AUC of 0.913 (95% CI: 0.8584–0.9677). The validation accuracy, sensitivity, and specificity reached 87.8%, 82.6%, and 88.9%, respectively. Compared with ResNet34, GoogLeNet, and ViT, DenseNet121 exhibited significantly superior performance across all metrics (P < 0.05) (Table 4). Table 3Classification performance of the four deep learning modelsModelNameAccAUC95% CISensitivitySpecificityPrecisionRecallCohort0densenet1210.8370.9140.8733–0.95480.9070.810.6450.907Train1densenet1210.8780.9130.8584–0.96770.8260.8890.6130.826Test2resnet340.8370.9030.8605–0.94500.8330.8380.6620.833Train3resnet340.6490.8440.7537–0.93510.9570.5830.3280.957Test4googlenet0.3930.4910.4005–0.58240.8330.2250.290.833Train5googlenet0.4350.5240.3949–0.65380.7390.370.20.739Test6ViT0.2760.4070.3185–0.4954100.2761Train7ViT0.8170.6280.4786–0.77740.4350.8980.4760.435TestOI-RADS0.8780.8580.7741–0.93690.8260.8890.6130.826TestHuman prediction0.7630.7370.6220–0.83830.6960.7780.40.737Test Classification performance of the four deep learning models Table 4Overall performance of the DenseNet121 model in differentiating benign and malignant tumorsGroupBenign CasesMalignant CasesP-valueTrain141550.034Test108230.034 Overall performance of the DenseNet121 model in differentiating benign and malignant tumors ROC curve analysis Figures 1 illustrate the ROC curves for the GoogLeNet and ViT models on the training and validation sets, respectively, where their AUC values approached random classification levels, indicating poor discriminative ability for benign versus malignant ovarian cysts. In contrast, the ROC curves for DenseNet121 and ResNet34 shifted markedly toward the upper-left corner, with DenseNet121 exhibiting the largest area under the curve and the optimal diagnostic performance. ROC curves of the four models on the training and validation sets Grad-CAM visualization results Several representative cases from the validation set were selected for Grad-CAM visualization of the DenseNet121 model’s attention regions. The results revealed that the model primarily focused on lesion wall thickness, papillary projections, and septations, which aligned closely with key pathological signs. This suggests strong consistency between the model’s decision-making basis and clinical observations (see Fig. 2). Grad-CAM Visualization Results for the DenseNet121 Model: Red areas indicate the model’s attention regions, showing high overlap with lesion distributions Comparative analysis with O-RADS and expert assessments The DenseNet121 model’s results were compared with subjective assessments by senior ultrasound physicians and the O-RADS grading system [24]. On the validation set, the DenseNet121 model achieved an accuracy of 87.8%, which was significantly higher than that of human assessments (76.5%). As shown in Table 3, compared with the O-RADS system, although both achieved the same accuracy (87.8%), the DenseNet121 model demonstrated a superior Area Under the Curve (AUC: 0.913 vs. 0.858, P < 0.05), indicating stronger overall discriminative ability. Furthermore, the DenseNet121 model exhibited a narrower 95% confidence interval with a higher lower bound (0.8584–0.9677 vs. 0.7741–0.9369), suggesting more stable and reliable performance estimates. This advantage holds significant clinical implications: the higher AUC indicates that the model maintains superior discriminative performance across different diagnostic thresholds, providing clinicians with greater decision-making flexibility—by adjusting thresholds to increase sensitivity and reduce missed malignancies in screening scenarios, or maintaining high specificity to avoid unnecessary surgical interventions in preoperative assessments—thereby achieving an optimal balance between reducing overdiagnosis and underdiagnosis risks. Summary Overall, the DenseNet121 model exhibited the highest diagnostic performance and generalization capability in classifying benign and malignant ovarian cysts, performing well on both the training and validation sets. It effectively focused on key imaging features relevant to pathological diagnosis, outperforming traditional O-RADS classification and manual assessments [25]. This model holds substantial value for clinical promotion and as an auxiliary decision-making tool. Discussion This study, based on ultrasound images from 327 pathologically confirmed ovarian cyst cases, systematically compared the performance of four deep learning models—DenseNet121, ResNet34, GoogLeNet [26], and Vision Transformer (ViT)—in classifying benign and malignant ovarian cysts. These models were also benchmarked against O-RADS grading and subjective assessments by senior ultrasound physicians [27]. The results demonstrated that DenseNet121 outperformed the other models in key metrics such as AUC, sensitivity, specificity, and accuracy. Its diagnostic performance not only surpassed manual evaluations but also exceeded O-RADS grading, indicating that deep learning-based automated diagnostic models offer significant advantages in differentiating benign and malignant ovarian cysts [28]. First, the superior performance of DenseNet121 may be closely linked to its architectural features. DenseNet facilitates feature reuse and efficient gradient propagation through dense cross-layer connections, enabling the capture of fine-grained features in ultrasound images. This is particularly well-suited for handling medical images with complex textures and subtle boundaries. In contrast, while ResNet34 incorporates residual connections, its shallower network depth limits feature representation capabilities, resulting in reduced accuracy on the validation set. The poorer performance of GoogLeNet and ViT in this study could be attributed to the relatively limited ultrasound image dataset, ViT’s strong dependence on large-scale data, and GoogLeNet’s outdated network design. These findings underscore the importance of selecting architectures optimized for small-sample learning in medical imaging scenarios. Compared to O-RADS grading, the DenseNet121 model achieved a better balance between sensitivity and specificity. Although O-RADS provides a standardized grading system that effectively reduces diagnostic variability among operators [29], it exhibits ambiguity in risk assessment for category 4 lesions, often leading to overtreatment in clinical practice. The findings of this study suggest that deep learning models, through automated extraction of imaging features, may facilitate a more nuanced differentiation between high-risk and low-risk subtypes within O-RADS category 4 lesions. This potential warrants further investigation and validation. Future work should prospectively collect ultrasound data containing 4a/4b subcategory information to evaluate the practical value of these models in reducing diagnostic misinterpretations within this clinically challenging lesion group.Relative to subjective assessments by experienced ultrasound physicians, the model not only delivered higher accuracy but also enabled rapid, stable, and reproducible diagnoses, mitigating the impact of inter-individual experience variations [11]. Furthermore, the Grad-CAM visualization results provided additional evidence of the DenseNet121 model’s interpretability. The model’s attention regions were primarily focused on lesion wall thickness, papillary projections, and areas with abundant blood flow—key indicators in the clinical diagnosis of malignant ovarian tumors. These findings suggest that deep learning models can go beyond “black-box” classifications to offer reasonable visual explanations, enhancing clinicians’ trust in the model and facilitating its integration into real-world clinical workflows. Although the results of this study hold certain clinical reference value, several limitations remain. First, as a single-center retrospective study with a relatively limited sample size—particularly with a low proportion of malignant cases—it may affect the model’s generalizability. Second, the study relied solely on two-dimensional static ultrasound images as input, excluding three-dimensional ultrasound or dynamic video data, which could overlook spatial or temporal features. Third, this study primarily evaluated the diagnostic performance of the DenseNet121 model as an independent system and compared it with the baseline level of senior ultrasound physicians (accuracy: 76.3%, AUC: 0.737). However, the model was only compared with experts from a single institution and O-RADS classification results; its incremental value in real-world clinical workflows—namely, whether and to what extent it can improve physicians’ diagnostic performance as a decision support tool—remains to be validated.Fourth, borderline ovarian tumors (BOTs) were classified into the malignant group for analysis in this study. Although this approach aligns with clinical management principles, BOTs possess unique biological behavior and imaging characteristics that fall between clearly benign and clearly malignant categories. This may introduce confounding factors when evaluating the model’s performance in distinguishing between typical benign and highly malignant tumors.Fifth, given the limited sample size of certain pathological subtypes—with only 7 cases each for clear cell carcinoma and borderline tumors—subgroup analysis would likely yield unstable performance estimates, such as excessively wide confidence intervals. Therefore, rather than conducting a stratified quantitative evaluation of the DenseNet121 model, this study focused on validating its overall performance. Future research could explore the following directions: (1) expanding sample sizes and incorporating multi-center datasets, particularly by increasing the number of cases with rare subtypes (such as clear cell carcinoma and borderline tumors), to further enhance model robustness and generalization; (2) investigating multimodal deep learning models that integrate diverse data sources, such as ultrasound video sequences, serum tumor markers [30], and patient clinical information, to achieve superior diagnostic efficacy; (3) conducting systematic subgroup analyses once the sample size is sufficient to investigate the model’s dignostic performance across different ovarian tumor subtypes and stages, thereby evaluating its potential in early ovarian cancer screening.Additionally, conduct systematic evaluations specifically targeting the challenging O-RADS category 4 lesions to more comprehensively validate the incremental clinical value of deep learning models in intermediate-risk categories.; (4) combining explainable artificial intelligence techniques to improve model transparency and clinical acceptability; and (5) conducting prospective multi-reader, multi-case studies to quantify the model’s impact on physicians’ diagnostic accuracy, confidence, and decision-making time, thereby providing more robust evidence for the implementation of the model in clinical decision support systems; and (6) conducting more detailed analyses specifically targeting borderline ovarian tumors as a special category, enabling a more comprehensive evaluation of the model’s performance across different pathological types. Conclusions In summary, the DenseNet121 deep learning model developed in this study exhibited excellent performance in classifying benign and malignant ovarian cysts, significantly improving diagnostic accuracy and consistency while reducing subjective errors. It holds strong potential as a clinical auxiliary diagnostic tool. With further advancements in multi-center validation and multimodal integration, deep learning is poised to become a vital complement to ovarian tumor risk stratification and treatment decision-making, supporting the evolution of precision medicine. Acknowledgements We are grateful to Lichang Zhong and Kaihua Fan for their valuable contributions to subject recruitment and eligibility assessment, patient liaison and longitudinal follow-up, and the management of study documentation. Authors’ contributions W.M.L. and Y.H.X. designed the project.W.M.L. ,Y.H.X., Y.Z.L. ,X.W. and L.S. performed experiments.W.M.L wrote the manuscript.W.M.L., Y.H.X. ,Y.Z.L., X.W., and L.S. collected the samples.W.M.L., Y.H.X. , Y.Z.L., X.W., and L.S. analyzed the data.W.M.L., Y.H.X. , Y.Z.L. reviewed and edited the manuscript.W.M.L.acquired funding. Funding This work was supported by the Shanghai Key Clinical Research Center Fund[grant number 2023ZZ02006]. Data availability The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. Declarations Ethics approval and consent to participate This study was approved by the Institutional Review Board (IRB) of the Sixth People’s Hospital of Shanghai, China (Approval No.:2025-KY-105(K) Date of Review:2025-03-27). The study was conducted in accordance with the Good Clinical Practice (GCP) guidelines and the Ethical Review Guidelines for Drug Clinical Trials issued by the National Medical Products Administration (NMPA) of the People’s Republic of China, as well as the principles of ICH-GCP. It also adheres to the requirements of the Declaration of Helsinki and complies with relevant laws and regulations in China.Written informed consent was obtained from all individual participants included in the study. Consent for publication Not applicable. Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Weimei Li, Yuhua Xia and Yongzhao Li contributed equally to this work. Contributor Information Xing Wu, Email: xing_apple@163.com. Lin Shi, Email: shilin_love@163.com. References Associated Data Data Availability Statement The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.