A “calcification”-enhanced deep learning approach for precise differentiation of thyroid nodules Authors: Xinying Zhu, Chen Chen, Yahan Zhou, Lingyan Zhou, Qingquan He, Xiao Zhang, Jincao Yao, Chenke Xu, Dong Xu Published: 2025 ABSTRACT BackgroundCalcification is one of the most valuable imaging features in the ultrasound diagnosis of thyroid nodules. A calcification-enhanced deep learning (DL) approach was developed in this study for the automatic detection of thyroid nodules and their intranodular calcifications from ultrasound images. Calcification features were integrated into the modeling process to improve the accuracy of benign-malignant differentiation for thyroid nodules.MethodsA total of 6886 thyroid nodules from 3 hospitals, collected between January 2014 and March 2024, were retrospectively included in this study. These nodules were partitioned into training, validation, and test sets at a ratio of 7:1:2. All nodules had a clearly documented final clinical diagnosis of benign or malignant status. A DL model that integrates intranodular calcification features and nodule imaging features was constructed. The model was trained using the training set, hyperparameters were optimized using the validation set, and final evaluation was conducted on an independent test set. The area under the receiver operating characteristic curve (AUC) was used as the primary evaluation metric.ResultsAmong the 6886 thyroid nodules included in this study, 4433 were benign and 2453 were malignant. DLAM−CFF—a DL model that integrates intranodular calcification features and nodule imaging features—exhibited excellent performance in differentiating benign from malignant thyroid nodules within the independent test cohort. Its sensitivity, specificity, and accuracy were 0.863, 0.864, and 0.864, respectively, with an AUC of 0.925. DLAM−CFF was compared with DLCFF (a DL model relying solely on nodule features) and traditional DL models, including Xception, InceptionResNetV2, DenseNet121, and ResNet50. The results indicated that the AUC values of these comparative models were 0.832, 0.805, 0.821, 0.813, and 0.798, respectively—all of which were significantly lower than those of DLAM−CFF (P < 0.05).ConclusionsThe “calcification”-enhanced DL model proposed in this study not only enables the automatic detection of thyroid nodules and their intranodular calcifications in ultrasound images but also demonstrates excellent diagnostic performance in predicting the benignity or malignancy of thyroid nodules by integrating the overall features of nodules with calcification features. Introduction Thyroid nodules are a common disease, and their detection rate has gradually increased with the application of high-resolution ultrasound in recent years, with a trend toward younger age at diagnosis [1, 2]. However, the vast majority of thyroid nodules are pathologically confirmed to be benign, with only 7–15% being malignant [3]. The key point in the diagnosis and management of thyroid nodules is the differentiation between benign and malignant lesions. Fine-needle aspiration biopsy (FNA) is regarded as the gold standard for the preoperative diagnosis of thyroid nodules; however, it is an invasive procedure that increases patients’ psychological and economic burdens [4]. Additionally, some patients experience significant psychological distress due to the inability to confirm the pathological nature of their nodules, leading to numerous unnecessary diagnostic and therapeutic procedures. Ultrasonography (US) examination is characterized by simplicity, real-time imaging, non-invasiveness, and high cost-effectiveness; thus, it serves as the primary imaging modality for evaluating thyroid nodules [1, 5]. The accurate detection rate of thyroid nodules using US has been reported to range from 81.5% to 99.1% [6]. Among the ultrasound findings of thyroid nodules, calcification is recognized as a critical sign with high diagnostic significance; however, comprehensive evaluation is required based on its specific type and size, as the correlation between different calcification subtypes and malignant risk differs. For instance, macrocalcifications (e.g., complete annular calcification and coarse calcification) are more prevalent in benign nodules, whereas specific subtypes of microcalcifications (e.g., psammoma bodies [PBs], which are highly suggestive of papillary thyroid carcinoma (PTC)) confer greater predictive value for malignant lesions [7–9]. Numerous prior studies have demonstrated that the incidence of microcalcifications is significantly higher in malignant nodules than in benign counterparts [10]. Consequently, among all calcification subtypes, microcalcifications have garnered the greatest attention in clinical and research settings [11]. However, the overall clinical significance of calcifications in differentiating benign from malignant thyroid nodules remains incompletely elucidated [12, 13]. One of the core reasons is the lack of a unified standard for radiologists to define various bright spots (e.g., microcalcifications, colloid crystals, backwall enhancement, or comet tail artifacts) on ultrasound images, which often hinders precise correlation with pathological calcifications [14–16]. For instance, structures identified as “microcalcifications” on ultrasound may not only represent pathologically defined PBs but also correspond to benign components such as concentrated colloid crystals and stromal microcalcifications. Additionally, they must be differentiated from benign artifacts, including backwall enhancement and comet tail artifacts. This often makes it difficult to accurately identify pathologically defined microcalcifications (i.e., PBs) via ultrasound in clinical practice. Consequently, accurately distinguishing the nature of these bright spots has emerged as a key challenge in thyroid ultrasound diagnosis [17]. Therefore, there is an urgent need for a more objective method to explore and utilize the malignant predictive value of “calcifications” in ultrasound images of thyroid nodules. In recent years, deep learning (DL) has achieved remarkable success in the field of image classification by learning patterns directly from raw data through neural networks [18]. Multiple studies on thyroid nodule differentiation using DL have demonstrated that DL methods can effectively learn diverse features from thyroid nodules, assist radiologists in improving diagnostic capabilities, and reduce unnecessary FNA procedures [19–22]. However, DL research focusing on “calcification”—a critical imaging feature—remains limited. Traditional DL models largely rely on the input of single global features of nodules, with local calcification features not incorporated separately. This may result in insufficient recognition of subtle calcification features associated with malignancy. Additionally, data acquired from different centers and ultrasound devices exhibits heterogeneity, which imposes heightened demands on the generalization ability of models. Wang et al. [23] aimed to detect and quantify calcifications in thyroid nodules based on ultrasound images and further investigate the value of “calcification” on ultrasound images in predicting the risk of lymph node metastasis (LNM) in PTC. Our team has also conducted relevant studies previously [24–26], in which preliminary exploration of “calcification” features in ultrasound images of nodules was performed, providing early evidence for imaging analysis in this field. Consequently, an innovative “calcification”-enhanced DL approach is proposed in this study. Based on large-scale multicenter ultrasound data, a dual-branch DL model (DLAM−CFF) was developed to automatically detect thyroid nodules and their associated calcifications, with calcification-related features automatically extracted and modeled. The model leverages two separate branches to learn global imaging features of nodules and local calcification features, respectively. An attention mechanism is incorporated during the fusion stage to dynamically weight high-value features (e.g., calcification location and morphology), thereby enhancing the correlation between the two feature types, effectively suppressing noise interference, and more efficiently capturing interactive information between calcification patterns and nodule backgrounds. This facilitates the accurate identification of subtle calcification structures highly associated with malignant risk. Additionally, ultrasound data from multiple centers and devices was incorporated in this study to introduce data heterogeneity, thereby further improving the model’s generalization ability. Through the objective extraction and fusion analysis of “calcifications” via DL, this study aims to improve the accuracy of differentiating benign from malignant thyroid nodules and provide a more reliable and objective diagnostic aid for clinical practice. Materials and methods This study was approved by the Ethics Committee of Zhejiang Cancer Hospital. For the retrospectively collected ultrasound images, a waiver of informed consent was granted (IRB-2024-580 (IIT)). Study design and datasets This was a multicenter retrospective diagnostic study that utilized an ultrasound image dataset from 3 hospitals in China, collected between January 2014 and March 2024. Patients with ultrasound-confirmed thyroid nodules and clearly documented definitive clinical diagnoses of benign or malignant status in the electronic medical record system were included. This definitive clinical diagnosis was established based on clinical pathways, either via postoperative histopathology or Bethesda II/VI FNA cytology. All preoperative ultrasound images were initially included, but low-quality or low-resolution images were excluded following screening. In this study, all thyroid nodule images from each center were acquired using various devices manufactured by multiple vendors, including Philips, Toshiba, GE Healthcare, and Siemens. A two-stage DL model (DLAM−CFF) based on a calcification enhancement strategy was proposed for the differential diagnosis of benign and malignant thyroid nodules with calcifications. The overall workflow of the model is presented in Fig. 1. In the first stage, the thyroid nodule region and its internal calcification regions were extracted using a semantic segmentation method, and two independent regions of interest (ROIs) were constructed based on these segmented regions. In the second stage, a dual-branch convolutional neural network (CNN) was designed to extract features from the calcification regions and the entire nodule regions separately. Final classification decisions were made through feature fusion and an attention mechanism. It is important to emphasize that the automatic extraction of calcification regions in this study was not intended to independently classify calcifications or determine their histological properties, but rather to provide locally high-risk features with diagnostic value for differentiating benign from malignant nodules. Since the imaging interpretation of calcifications cannot be separated from the overall structure and context of the nodule, and the diagnosis of nodule benignancy or malignancy also relies on the integrated information of both, the entire nodule ROI was simultaneously extracted, and the two types of features were fused in subsequent network layers for the final benign-malignant classification. For the development and testing of the DLAM−CFF model (for differentiating benign and malignant thyroid nodules), a total of 11,634 ultrasound images from 6886 thyroid nodules were included in this study from the ultrasound image dataset. Ultrasound images were divided into mutually independent training, validation, and test sets at a ratio of 7:1 :2 based on the unique ID of each nodule. All nodules from the same patient were assigned to the same dataset. Stratified sampling based on the benign-malignant ratio was employed to ensure balanced distribution of baseline characteristics across all datasets, with results presented in Table 1. Fig. 1The flowchart of this study includes four components. (a) Multi-center data collection and dataset partitioning; (b) Segmentation of thyroid nodules and calcifications, and region of interest (ROI) extraction; (c) Malignant risk prediction via calcification-nodule dual-domain feature enhancement and attention-guided fusion; (d) Model performance evaluation. Xception: Short for “Extreme Inception”, a DL model proposed by Chollet in 2017. It employs depthwise separable convolutions to achieve efficient feature extraction and is well-suited for image classification tasks [28] The flowchart of this study includes four components. (a) Multi-center data collection and dataset partitioning; (b) Segmentation of thyroid nodules and calcifications, and region of interest (ROI) extraction; (c) Malignant risk prediction via calcification-nodule dual-domain feature enhancement and attention-guided fusion; (d) Model performance evaluation. Xception: Short for “Extreme Inception”, a DL model proposed by Chollet in 2017. It employs depthwise separable convolutions to achieve efficient feature extraction and is well-suited for image classification tasks [28] Table 1Baseline characteristicsTraining datasetValidation datasetTest datasetFinal clinical diagnosisBenignMalignantBenignMalignantBenignMalignantFNA cytology (%)2013 (65)1057 (62)288 (65)149 (61)567 (64)273 (56)Postoperative histopathology (%)1090 (35)660 (38)155 (35)96 (39)320 (36)218 (44)No. of nodules31031717443245887491No. of images545226827923951556757Mean age (range)51 ± 12 (9–83)44 ± 12 (15–78)49 ± 12 (10–81)44 ± 12 (20–80)50 ± 12 (17–86)44 ± 12 (17–79)Male (%)690 (22)418 (24)98 (22)63 (26)206 (23)118 (24)Female (%)2413 (78)1299 (76)345 (78)182 (74)681 (77)373 (76)Mean size (mm)14.27 ± 11.03 (1.20–69.67)8.99 ± 6.29 (1.70–58.97)14.27 ± 10.93 (1.60–56.40)9.15 ± 6.16 (2.57–39.00)14.01 ± 10.56 (1.16–60.00)9.31 ± 6.64 (2.10–63.33)Echogenic focilarge comet-tail artifacts (%)2203 (80.5)759 (56.2)313 (78.4)107 (56.9)613 (78.0)218 (55.3)Macrocaicifications (%)152 (5.6)60 (4.4)24 (6.0)11 (5.9)56 (7.1)22 (5.6)Peripheral (rim) calcifications (%)47 (1.7)21 (1.6)12 (3.0)1 (0.5)13 (1.7)6 (1.5)Punctate echogenic foci (%)334 (12.2)511 (37.8)50 (12.5)69 (36.7)104 (13.2)148 (37.6)Note. Unless otherwise indicated, data in parentheses are percentages. Mean data are ± SDs with ranges in parentheses Table 1Baseline characteristicsTraining datasetValidation datasetTest datasetFinal clinical diagnosisBenignMalignantBenignMalignantBenignMalignantFNA cytology (%)2013 (65)1057 (62)288 (65)149 (61)567 (64)273 (56)Postoperative histopathology (%)1090 (35)660 (38)155 (35)96 (39)320 (36)218 (44)No. of nodules31031717443245887491No. of images545226827923951556757Mean age (range)51 ± 12 (9–83)44 ± 12 (15–78)49 ± 12 (10–81)44 ± 12 (20–80)50 ± 12 (17–86)44 ± 12 (17–79)Male (%)690 (22)418 (24)98 (22)63 (26)206 (23)118 (24)Female (%)2413 (78)1299 (76)345 (78)182 (74)681 (77)373 (76)Mean size (mm)14.27 ± 11.03 (1.20–69.67)8.99 ± 6.29 (1.70–58.97)14.27 ± 10.93 (1.60–56.40)9.15 ± 6.16 (2.57–39.00)14.01 ± 10.56 (1.16–60.00)9.31 ± 6.64 (2.10–63.33)Echogenic focilarge comet-tail artifacts (%)2203 (80.5)759 (56.2)313 (78.4)107 (56.9)613 (78.0)218 (55.3)Macrocaicifications (%)152 (5.6)60 (4.4)24 (6.0)11 (5.9)56 (7.1)22 (5.6)Peripheral (rim) calcifications (%)47 (1.7)21 (1.6)12 (3.0)1 (0.5)13 (1.7)6 (1.5)Punctate echogenic foci (%)334 (12.2)511 (37.8)50 (12.5)69 (36.7)104 (13.2)148 (37.6)Note. Unless otherwise indicated, data in parentheses are percentages. Mean data are ± SDs with ranges in parentheses Baseline characteristics Note. Unless otherwise indicated, data in parentheses are percentages. Mean data are ± SDs with ranges in parentheses Calcification and nodule region segmentation First, a publicly available thyroid nodule segmentation network based on the Swin-Unet architecture was used for the initial segmentation of thyroid nodules in ultrasound images [27]. This network has demonstrated favorable performance across multiple medical image segmentation tasks and is particularly suitable for lesion regions with ill-defined boundaries or complex structures. Precise localization of nodule regions was achieved using its pre-trained weights. Building on the nodule segmentation results, potential intranodular calcifications were further identified, and a calcification segmentation method based on local gray-level thresholds was proposed. Calcifications in ultrasound images typically appear as local hyperechoic regions; however, their absolute gray-level values are susceptible to variations in imaging parameters and equipment. To address this issue, an adaptive thresholding algorithm based on intranodular gray-level distribution was proposed. This algorithm first calculates the quartiles of pixel intensity within each nodule region, followed by the determination of a calcification identification threshold based on the third quartile and interquartile range (IQR). This strategy effectively mitigates interference from inter-image gray-level variations and focuses on identifying echo points within each nodule that are significantly brighter relative to their surrounding background; these points are preliminarily designated as potential calcification regions. To further improve the specificity of calcification segmentation, morphological processing was applied to the initially extracted potential regions to remove scattered noise points and preserve the morphology of clinically relevant calcifications. Finally, based on the nodule segmentation results and calcification segmentation results, the overall nodule ROI (224 × 224 pixels) and calcification region ROI (96 × 96 pixels) were cropped separately and used as inputs for the second-stage classification network. Design of the dual-branch classification network A dual-branch structure was employed in the second stage to process the calcification ROI and overall nodule ROI separately, enabling the simultaneous capture of micro-scale calcification morphological features and macro-scale nodule structural features. For the calcification feature extraction branch, the cropped calcification region images (96 × 96 × 3) were used as input. Given the typically small size and diverse morphologies of calcification regions, the network was required to have strong local feature reuse and detail capture capabilities. Consequently, DenseNet121 was selected as the backbone network. DenseNet facilitates feature reuse and enhances gradient flow through its dense connection mechanism, making it particularly suitable for feature extraction of microcalcifications. Subsequently, an UpSampling2D layer and Conv2D layer were further incorporated to expand the feature maps to higher resolutions, restoring detailed information that might have been lost during downsampling. Finally, a multi-head self-attention module was designed to model long-range dependencies within calcification regions, emphasizing important features and suppressing redundant information. For the nodule global feature extraction branch, the input was the overall nodule ROI (224 × 224 × 3), which was designed to extract macro-scale features of nodules, such as overall morphology, edge regularity, and internal echogenicity. The Extreme Inception (Xception) network was selected as the backbone network for this branch. Xception [28] is a deep convolutional neural network proposed by Chollet. Its core innovation lies in the replacement of traditional convolutions with depthwise separable convolutions, which reduces the number of parameters while preserving robust feature representation capabilities. Notably, it is particularly well-suited for medium-scale image classification tasks. Its multi-scale feature extraction capability also contributes to capturing structural information of nodules at different levels. The features output by the two branches were concatenated to form a combined feature representation. This fused feature not only contains local detailed information of the calcification region but also integrates the global contextual information of the entire nodule. Subsequently, the fused features were downsampled using a global average pooling layer to reduce the risk of overfitting, followed by connection to fully connected layers for outputting the final benign/malignant probabilities. A Sigmoid activation function was used in the classification layer to achieve binary classification output. Training details and implementation The ultrasound images used for training, validation, and testing of the DLAM−CFF model in this study were randomly partitioned from the ultrasound image dataset at a predefined ratio. For fair comparison, the DLAM−CFF model and four classical CNN models (Xception, InceptionResNetV2, DenseNet121, and ResNet50) were all trained, validated, and tested using the same training, validation, and test sets. Additionally, a single-branch classification network (DLCFF) that retains only the nodule global feature extraction branch was designed to validate the effectiveness of the calcification feature extraction branch. Furthermore, all models were implemented on the Ubuntu 20.04 operating system using the Keras 2.5.0 and TensorFlow 2.5.0 frameworks. The host machine used was equipped with an Intel® Core™ i9-13900 K @ 3.0 GHz and an NVIDIA GeForce RTX 3070 Ti 8 GB GPU. During the training process, the batch size was set to 16, and the training was conducted for 60 epochs. Statistical analysis All statistical analyses were performed using the Python language. The area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were used as evaluation metrics in this study. Additionally, 95% confidence intervals (CIs) were used to estimate the ranges of these values. Comparisons of AUC values between models were conducted using the DeLong test. The F1-score was used to comprehensively evaluate the precision and recall of model classification, while the Kappa coefficient was used to quantitatively analyze the classification agreement between model diagnostic results and the pathological gold standard. A two-tailed P-value < 0.05 was considered statistically significant. Results Patient characteristics A total of 6886 thyroid nodule cases were retrospectively collected from 3 hospitals in this study. Among these cases, 4433 were benign nodules (64.38%) and 2453 were malignant nodules (35.62%). The training set consisted of 8134 images from 4820 cases, the validation set comprised 1187 images from 688 cases, and the test set included 2313 images from 1378 cases. The baseline characteristics of the cases are presented in Table 1. Predictive accuracy of DL models for discriminating malignant and benign nodules The performance of six DL models—including DLAM−CFF, DLCFF, Xception, InceptionResNetV2, DenseNet121, and ResNet50—in the classification task was compared in this study. Table 2 presents the diagnostic accuracy, sensitivity, and specificity of each DL model in the test set. Figure 2 illustrates the confusion matrices for benign-malignant thyroid nodule classification of each model, which further reveal the specific distribution characteristics of classification errors. Table 2Diagnostic performance of multiple DL models in differentiating benign and malignant thyroid nodulesDLAM−CFFDLCFFXceptionInceptionResNetV2DenseNet121ResNet50Accuracy (95% CI)0.864(0.843, 0.884)0.846(0.824, 0.868)0.829(0.806, 0.852)0.803(0.779, 0.828)0.750(0.724, 0.777)0.746(0.720, 0.773)Sensitivity (95% CI)0.863(0.827, 0.898)0.844(0.807, 0.881)0.801(0.760, 0.841)0.712(0.665, 0.758)0.717(0.671, 0.763)0.682(0.635, 0.729)Specificity (95% CI)0.864(0.838, 0.890)0.847(0.820, 0.875)0.844(0.817, 0.872)0.855(0.828, 0.882)0.769(0.737, 0.801)0.782(0.751, 0.814)PPV (95% CI)0.780(0.740, 0.821)0.756(0.715, 0.797)0.743(0.700, 0.785)0.733(0.688, 0.779)0.635(0.589, 0.681)0.637(0.590, 0.685)NPV (95% CI)0.918(0.897, 0.940)0.906(0.883, 0.929)0.883(0.858, 0.908)0.841(0.813, 0.869)0.829(0.799, 0.859)0.814(0.784, 0.845)AUROC (95% CI)0.925(0.909, 0.941)0.905(0.887, 0.923)0.883(0.863, 0.903)0.855(0.834, 0.877)0.822(0.799, 0.845)0.797(0.772, 0.821)F10.8190.7970.7700.7220.6730.659K (Kappa)0.7100.6740.6340.5700.4720.457AUROC, The area under the receiver operating characteristic curve; PPV, positive predictive value; NPV, negative predictive value; F1, F1 score Table 2Diagnostic performance of multiple DL models in differentiating benign and malignant thyroid nodulesDLAM−CFFDLCFFXceptionInceptionResNetV2DenseNet121ResNet50Accuracy (95% CI)0.864(0.843, 0.884)0.846(0.824, 0.868)0.829(0.806, 0.852)0.803(0.779, 0.828)0.750(0.724, 0.777)0.746(0.720, 0.773)Sensitivity (95% CI)0.863(0.827, 0.898)0.844(0.807, 0.881)0.801(0.760, 0.841)0.712(0.665, 0.758)0.717(0.671, 0.763)0.682(0.635, 0.729)Specificity (95% CI)0.864(0.838, 0.890)0.847(0.820, 0.875)0.844(0.817, 0.872)0.855(0.828, 0.882)0.769(0.737, 0.801)0.782(0.751, 0.814)PPV (95% CI)0.780(0.740, 0.821)0.756(0.715, 0.797)0.743(0.700, 0.785)0.733(0.688, 0.779)0.635(0.589, 0.681)0.637(0.590, 0.685)NPV (95% CI)0.918(0.897, 0.940)0.906(0.883, 0.929)0.883(0.858, 0.908)0.841(0.813, 0.869)0.829(0.799, 0.859)0.814(0.784, 0.845)AUROC (95% CI)0.925(0.909, 0.941)0.905(0.887, 0.923)0.883(0.863, 0.903)0.855(0.834, 0.877)0.822(0.799, 0.845)0.797(0.772, 0.821)F10.8190.7970.7700.7220.6730.659K (Kappa)0.7100.6740.6340.5700.4720.457AUROC, The area under the receiver operating characteristic curve; PPV, positive predictive value; NPV, negative predictive value; F1, F1 score Diagnostic performance of multiple DL models in differentiating benign and malignant thyroid nodules AUROC, The area under the receiver operating characteristic curve; PPV, positive predictive value; NPV, negative predictive value; F1, F1 score Fig. 2Examples of segmentation results using DLAM−CFF and different models Examples of segmentation results using DLAM−CFF and different models DLCFF performs classification solely based on nodule features. Its diagnostic accuracy, sensitivity, and specificity were 0.846 (95% CI: 0.824, 0.868), 0.844 (95% CI: 0.807, 0.881), and 0.847 (95% CI: 0.820, 0.875), respectively. As observed from the confusion matrix results, 58 false negatives (missed diagnosis rate: 15.6%) and 101 false positives (misdiagnosis rate: 15.3%) were reported for DLCFF. The diagnostic performance of DLAM−CFF was improved by integrating nodule features with calcification features, optimized through the introduction of an attention mechanism. Its diagnostic accuracy reached 0.864 (95% CI: 0.843, 0.884), with a sensitivity of 0.863 (95% CI: 0.827, 0.898) and specificity of 0.864 (95% CI: 0.838, 0.890). As observed from the confusion matrix results, only 51 false negatives (missed diagnosis rate: 13.7%) and 88 false positives (misdiagnosis rate: 13.3%) were reported for DLAM−CFF, with both the number of missed malignant cases and misdiagnosed benign cases lower than those of all comparative models. Among other commonly used models, Xception achieved an accuracy of 0.821 (95% CI: 0.798, 0.845), sensitivity of 0.815 (95% CI: 0.778, 0.852), and specificity of 0.826 (95% CI: 0.799, 0.853). InceptionResNetV2 exhibited an accuracy of 0.793 (95% CI: 0.768, 0.818), sensitivity of 0.789 (95% CI: 0.751, 0.827), and specificity of 0.796 (95% CI: 0.769, 0.823). DenseNet121 and ResNet50 demonstrated poorer performance, with ResNet50 achieving the lowest accuracy of 0.746 (95% CI: 0.720, 0.773), a sensitivity of 0.682 (95% CI: 0.635, 0.729), and a specificity of 0.769 (95% CI: 0.737, 0.801). Additionally, 118 false negatives (missed diagnosis rate: 31.8%) were reported for ResNet50, indicating an extremely high clinical risk of missed diagnosis. In summary, the confusion matrix of DLAM−CFF is consistent with its quantitative metrics, demonstrating that DLAM−CFF achieves the best performance in terms of overall classification accuracy and malignant nodule detection. Receiver-operating characteristic (ROC) curve analysis of DL models The ROC curves and AUC values of each model are presented in Fig. 3. DLAM−CFF exhibited the best performance, with an AUC of 0.925 (95% CI: 0.909, 0.941), which indicated the strongest comprehensive ability to differentiate benign from malignant thyroid nodules. DLCFF ranked second, with an AUC of 0.905 (95% CI: 0.887, 0.923). A DeLong test was conducted to compare the AUC values of the two models, with the difference identified as statistically significant (P < 0.05; Fig. 4). This result confirmed that the overall discriminative performance of DLAM−CFF was significantly superior to that of DLCFF. Fig. 3Receiver operating characteristic (ROC) curves for assessing the diagnostic performance of multiple DL Models in differentiating benign and malignant thyroid nodules. AUC: Area under the receiver operating characteristic curve Receiver operating characteristic (ROC) curves for assessing the diagnostic performance of multiple DL Models in differentiating benign and malignant thyroid nodules. AUC: Area under the receiver operating characteristic curve Fig. 4DeLong test heatmap illustrating pairwise significance comparisons (via p-values) among multiple DL models for differentiating benign and malignant thyroid nodules DeLong test heatmap illustrating pairwise significance comparisons (via p-values) among multiple DL models for differentiating benign and malignant thyroid nodules Among other commonly used models, the AUC of Xception was 0.883 (95% CI: 0.863, 0.903). For InceptionResNetV2 and DenseNet121, the AUC values were 0.855 (95% CI: 0.834, 0.877) and 0.822 (95% CI: 0.799, 0.845), respectively. ResNet50 exhibited the weakest performance, with an AUC of only 0.797 (95% CI: 0.772, 0.821). All these AUC values were significantly lower than those of DLAM−CFF and DLCFF (both developed in this study), with the differences deemed statistically significant (P < 0.05). Visualization of model attention regions Grad-CAM heatmaps were used to visualize the key regions of interest for the model during decision-making (Fig. 5). Red-yellow regions indicate a higher degree of model attention. Fig. 5Transverse and longitudinal ultrasound images with corresponding Grad-CAM heatmaps for various thyroid nodule cases. Panels a–d and e–h depict malignant nodules, while panels i–l and m–p depict benign nodules. For each case, four image types are presented: (a, e, i, m) transverse grayscale images; (b, f, j, n) Grad-CAM heatmaps overlaid on transverse images; (c, g, k, o) longitudinal grayscale images; (d, h, l, p) Grad-CAM heatmaps overlaid on longitudinal images Transverse and longitudinal ultrasound images with corresponding Grad-CAM heatmaps for various thyroid nodule cases. Panels a–d and e–h depict malignant nodules, while panels i–l and m–p depict benign nodules. For each case, four image types are presented: (a, e, i, m) transverse grayscale images; (b, f, j, n) Grad-CAM heatmaps overlaid on transverse images; (c, g, k, o) longitudinal grayscale images; (d, h, l, p) Grad-CAM heatmaps overlaid on longitudinal images For malignant nodules (Panels a–d, e–h), the DLAM–CFF model consistently focused on malignant ultrasound signs associated with thyroid cancer. For nodules with microcalcifications (Panels a–d), the model focused primarily on intranodular microcalcifications and the surrounding irregular hypoechoic structures. For calcification-free nodules with prominent malignant features (Panels e–h), the model focused on regions with ill-defined borders and internal heterogeneity. For benign nodules (Panels i–l, m–p), the heatmaps exhibited a low-intensity, diffuse activation pattern. In calcification-free benign nodules (Panels i–l), the model only showed mild attention to uniform, regular internal echoes. In nodules with macrocalcifications (Panels m–p), the model reduced its attention to macrocalcifications and their acoustic shadows, concentrating activation on the periphery of calcifications rather than the nodule parenchyma—this is consistent with their low malignant potential. Discussion The results of this study showed that the “calcification”-enhanced DL model (DLAM−CFF) exhibited the best performance in thyroid nodule classification tasks, with an AUC of 0.925, a sensitivity of 0.863, and a specificity of 0.864. This performance was significantly superior to that of DLCFF (which relies solely on nodule features) and other classical models, and this advantage was closely associated with the model’s design that integrates overall nodule features with “calcification” features. Clinically, the interpretation of calcifications cannot be separated from the overall information of the nodule, and the diagnostic significance of calcifications depends on their location, distribution within the nodule, and accompanying imaging features. It was observed that relying solely on local features of calcification regions is insufficient to support accurate benign-malignant differentiation, while relying solely on overall structural information of the nodule fails to capture key details such as microcalcifications. Therefore, this study proposes treating the calcification ROI as a supplementary feature that provides local high-risk cues. Through fusion with overall nodule features via an attention mechanism, enhanced representation of malignant signs is achieved, which constitutes the “calcification”-enhanced nodule risk assessment strategy. A workflow framework of “nodule segmentation-calcification segmentation-dual-branch feature extraction” was employed in this study. This framework first precisely segmented thyroid nodule regions from raw ultrasound images, followed by the identification and extraction of intranodular calcifications, thereby eliminating interference from glandular tissue and surrounding structures for subsequent feature extraction. Notably, the integration of a dual-branch structure and an attention mechanism facilitates addressing the challenges associated with identifying thyroid microcalcifications, including their small size, scattered distribution, and propensity for confusion with colloid and other structures. Therefore, compared with the DLCFF model (which does not incorporate calcification features), DLAM−CFF learns the overall nodule context and local calcification details in parallel. By enhancing key calcification features with dynamic weights, it not only reduces the omission of microcalcifications but also suppresses non-specific noise—this is the key reason for the simultaneous improvement in the model’s sensitivity and specificity. Furthermore, Grad-CAM visualizations further confirmed that the regions attended to by the DLAM–CFF model are consistent with clinically high-risk features, including microcalcifications, intranodular heterogeneity, and border abnormalities—thus supporting the interpretability and clinical acceptability of the DLAM–CFF model. Thyroid nodules with microcalcifications are associated with a higher malignant risk and frequently indicate malignant lesions, such as PTC. In contrast, macrocalcifications and marginal arc-shaped calcifications are mostly associated with benign lesions (e.g., nodular goiter, adenoma) and have a lower malignant probability. However, in clinical practice, accurate identification of calcification types still faces several challenges: microcalcifications are prone to misdiagnosis due to their small size and easy confusion with colloid crystals and artifacts; mixed calcifications are difficult to differentiate due to their complex morphological feature; subtle features are hard to quantify; and the consistency of manual diagnosis is low due to variations in radiologists’ experience and equipment differences. Park et al. [29] analyzed ultrasound findings of 854 thyroid nodules with macrocalcifications and found that the presence of interrupted calcifications, irregular thickness, and soft tissue margins was associated with malignancy in thyroid nodules with macrocalcifications. Chen et al. [25] reported that radiologists demonstrated significantly insufficient diagnostic consistency in differentiating between calcifications and colloid within echogenic foci of thyroid nodules. Specifically, the mean Kappa value was only 0.2122 in the echogenic foci subgroup and only 0.5853 in the punctate echogenic foci subgroup. This finding indicates that manual interpretation has significant subjective variability in the differentiation of such subtle imaging features. Furthermore, DL-based auxiliary assessment for calcification features of thyroid nodules has not yet been fully explored, making it difficult to accurately analyze the malignant risk associated with this critical feature. The effectiveness of DL in the auxiliary diagnosis of thyroid nodules with echogenic foci has been demonstrated in our previous studies [25, 26]. However, these studies utilized datasets from only one or two centers, with a limited sample size, and failed to fully cover the diverse calcification subtypes and imaging heterogeneity encountered in clinical practice. Consequently, they may be insufficient for evaluating the generalization performance of models across multiple centers, devices, and diverse populations. In light of this, a DLAM−CFF model that integrates features from different imaging devices and covers various calcification types was developed in this study based on a multicenter ultrasound dataset cohort. This model is designed to achieve accurate identification and benign/malignant classification of thyroid nodules by fusing calcification features, provide objective evidence for the assessment of thyroid nodule malignancy through the analysis of calcification features, and enhance the clinical utility of DL in calcification-related diagnosis of thyroid nodules. The results showed that the DLAM−CFF model, which fuses nodule and intranodular calcification features, exhibited good performance in thyroid nodule classification tasks and has the potential to provide reliable auxiliary diagnosis for radiologists. This study has the following limitations: First, the included malignant nodules are predominantly classic PTC, with a small sample size of rare subtypes such as follicular carcinoma and medullary carcinoma. This may compromise the model’s performance in identifying such nodules and limit the contributory value of calcification features across different pathological subtypes; Second, the model was constructed based on static ultrasound images, which can only capture information from a single frame and fail to acquire multi-planar morphology, mobility, hemodynamic features of nodules, and their relationships with adjacent structures during dynamic scanning. This results in discrepancies from real clinical diagnostic scenarios; Finally, calcification segmentation was not quantitatively evaluated due to the absence of standardized annotations, although ablation results indirectly support its value. Future research will advance in three aspects: First, prospective multicenter studies will be conducted to expand the sample size of rare subtypes such as follicular carcinoma and medullary carcinoma, and subgroup analyses will be performed to validate the model’s generalization ability; Second, dynamic ultrasound videos will be incorporated to capture richer spatial features, and a spatial-feature-based DL framework will be developed to align with clinical dynamic diagnostic workflows; Finally, standardized pixel-level calcification annotations will be incorporated to enable quantitative evaluation of the segmentation module and comparison with existing segmentation networks. Conclusion In conclusion, this study demonstrated that the “calcification”-enhanced DL approach significantly outperforms strategies relying solely on single nodule features in the differentiation of benign and malignant thyroid nodules, and can effectively improve diagnostic accuracy. This approach has demonstrated unique advantages in identifying calcification-related malignant features and holds promise for development as a reliable auxiliary tool for the accurate clinical diagnosis of thyroid nodules in the future.