Evaluation of a YOLOv5x-based deep learning model for interproximal caries segmentation on bitewing radiographs across primary and permanent teeth Authors: Hatice Selin Güçlü, Akif Demirel, Kaan Orhan Published: 2025 ABSTRACT AbstractBackgroundThis study aimed to develop and evaluate a YOLOv5x-based instance segmentation model for automatic interproximal caries segmentation on bitewing radiographs across primary and permanent teeth and to summarize performance using clinically interpretable metrics.MethodsThe current research had a retrospective study design. 1000 primary teeth and 1000 permanent teeth bitewing radiographs were analysed for interproximal caries, which were marked using polygonal segmentation in the Craniocatch labelling module, resulting in a total of 5704 caries lesions being annotated. The images were divided into 80% training, 10% validation, and 10% testing, and the Yolov5x model was trained in the PyTorch environment. Performance was reported using metrics such as confusion matrix, precision, recall, F1 score and ROC-AUC.ResultsPrecision value in the test dataset of primary tooth radiographs was determined as 0.80, recall as 0.78, F1 score as 0.79 and ROC-AUC as 0.87. In the permanent tooth group, precision was 0.78, recall 0.75, F1 score 0.76 and ROC-AUC value 0.76.ConclusionsWithin the limitations of the study, confusion matrices of primary and permanent tooth radiographs showed that the false positive and false negative rates were at acceptable levels. Yolov5x model is considered to have potential for use in children as a diagnostic tool, treatment planning, and clinical decision support system. However, the single-centre study design and the training process based solely on radiographic images may limit generalisability. Therefore, future studies with multi-centre, age-balanced, and multimodal data, including external validation, are recommended.Keywords: Artificial intelligence, Caries segmentation, Dental caries, Permanent teeth, Primary teeth Background Dental caries exerts a considerable burden on children and adolescents, manifesting in various forms, including pain, infection, nutritional and sleep disorders, as well as school absenteeism and impaired academic performance. Recent studies demonstrate that decay has a detrimental effect on daily life activities and quality of life related to oral health. Furthermore, it is associated with significant deterioration in school performance and attendance. This situation underscores the pivotal role of dependable screening and precise diagnosis in the initial phases, with the potential to enhance clinical and societal outcomes [1–6]. Clinical examination is the basis for caries diagnosis; however, early lesions on approximal surfaces may be overlooked. Therefore, individualized radiographic evaluation -particularly bitewing radiographs, which are considered the gold standard for detecting approximal lesions- increases diagnostic accuracy. American Academy of Pediatric Dentistry (AAPD) “Best Practice” document recommends that the timing, selection, and frequency of radiographs in pediatric patients be planned individually based on caries risk and clinical findings [7]. Over the past decade, deep learning-based artificial intelligence (AI) applications have become a promising decision support area for detecting cavities in dental radiographs [8–10]. Recent studies emphasize that models trained specifically on bitewing images can demonstrate high sensitivity/specificity in identifying approximal caries; however, data set diversity, methodological heterogeneity, and lack of external validation remain significant limitations [8–12]. Studies using different architectures, including the current You Only Look Once (YOLO) family, on bitewing radiographs report competitive performance compared to clinicians [13]. A study published in 2025 detected both enamel and dentin lesions with high accuracy using an improved YOLOv9c backbone and directly compared it with clinicians on the same test set [13]. However, it is clear that external center validations and strengthened reporting standards are necessary to translate reported successes into clinical generalizability. The clinical value of AI is not limited to accuracy metrics; its impact on cost-effectiveness and treatment decisions is also important. In randomized studies, AI support generally increases sensitivity, but it can also change the rate of more invasive decisions by affecting treatment thresholds; in economic models, context-sensitive, cost-saving results have been reported in some scenarios, while uncertain results have been reported in others. Therefore, it is recommended that AI be positioned as a decision support tool and integrated into clinical pathways using a “responsible integration” approach [14, 15]. The application of AI-supported systems in pediatric dental radiology is expanding and these technologies offer promising results in the automatic diagnosing of carious lesions. However, the vast majority of studies in the literature focus on permanent teeth, and research on caries diagnosis in primary teeth in the early age group remains limited. Considering that caries progresses more rapidly in primary teeth, coupled with the vulnerability stemming from the structural characteristics of enamel and dentin, the importance of early and accurate diagnosis becomes even more apparent. Furthermore, the limited level of cooperation in pediatric patients reduces the effectiveness of traditional diagnostic methods and can lead to delayed detection of dental caries. This situation complicates the treatment process and increases the need for invasive interventions. In this context, the need for technologies that can save time and increase accuracy in the diagnostic processes for pediatric patients is growing day by day. At this point, this study aims to make a concrete contribution to this need by comparing the performance of AI model developed using radiographs obtained from both primary and permanent teeth. Thus, it is important to provide significant data for the development of reliable and effective digital diagnostic support tools that can be used in pediatric dentistry practice. The results obtained from such studies are expected to fill the gap in the literature regarding the integration of AI-based solutions in pediatric dentistry. In light of these above-mentioned gaps, the present study does not aim to propose a new detection or segmentation architecture. Instead, it uses a conventional YOLOv5x instance-segmentation pipeline and focuses on an application- and validation-oriented contribution with direct clinical relevance. Specifically, the main aims of the present study were: to construct a curated bitewing radiograph dataset including both primary and permanent teeth, in which interproximal carious lesions are delineated using polygonal segmentation and to train and evaluate a YOLOv5x model separately in primary and permanent dentitions using lesion-level, clinically interpretable metrics. Methods The study design, ethical approval, and all other methodological steps relating to the study investigating the performance of the Yolov5x deep learning architecture/model in diagnosing interproximal caries in primary and permanent teeth were presented in detail below. Study design This study was conducted on bitewing dental radiographic images and had a retrospective design. Within the scope of the study, the diagnostic performance of the model developed with the Yolov5x algorithm in diagnosing the caries located at the approximal areas of primary and permanent teeth was evaluated. Ethical approval and ethical principles Ethical approval for the current study was obtained from Ankara University Faculty of Dentistry Clinical Research Ethics Committee (Date: 06.11.2023, Decision: 12/11). Furthermore, the research protocol in this study was implemented in accordance with the ethical principles outlined in World Medical Association’s Declaration of Helsinki (https://www.wma.net/policies-post/wma-declaration-of-helsinki/) [16]. The study was also conducted retrospectively using radiographs retrieved from our institution’s Picture Archiving and Communication System (PACS). In accordance with the routine institutional workflow, consent for radiographic imaging was obtained from patients and/or their parents at the time of image acquisition. Given the retrospective design and the use of fully anonymized images obtained under routine clinical consent, the ethics committee confirmed that no additional informed consent specific to research use was required. Furthermore, the use of included radiographs for research purposes of this study was approved by the institutional ethics committee and supported by the necessary institutional permissions. Provision of image data Bitewing radiographic images included in the study protocol were obtained with the consent of the patients/parents at Ankara University Faculty of Dentistry Department of Pediatric Dentistry Clinic and Ankara University Faculty of Dentistry Department of Oral and Maxillofacial Radiology Department Clinics, were selected from images recorded in the hospital’s PACS system, after obtaining the consent of the patients/parents, as previously required for different indications. All bitewing radiographs included in the study were obtained using the Belmont Phot-X II (Belmont Takara Company, Japan) device, which has a high-frequency DC system. During radiographic imaging using a digital sensor (Gendex GX PS-500, Gendex Dental Systems, Hatfield, PA, USA), low-dose protocols specific to pediatric patients were applied; the tube voltage was set to 60–70 kV, and the tube current was set to 4–5 mA. Exposure times were set to be between 0.10 and 0.25 s. Accordingly, the criteria for including previously acquired radiographic images in this study were listed below. Additionally, for patients within the permanent dentition age range, bitewing radiographs had been acquired previously using the institution’s standard exposure protocols for adult patients, as determined by routine clinical practice. These images were not obtained specifically for this research; rather, eligible radiographs recorded in the PACS under the adult protocol were retrospectively identified and included according to the criteria described below. Eligibility criteria Inclusion criteria The presence of radiographically detectable dental caries in at least one interproximal surface (distal and/or mesial) on both primary and permanent bitewing radiographs, The presence of radiographically detectable dental caries in at least one interproximal surface (distal and/or mesial) on both primary and permanent bitewing radiographs, For bitewing radiographs obtained during the primary or mixed dentition period, the corresponding patient age should range between 4 and 9 years, For bitewing radiographs obtained during the primary or mixed dentition period, the corresponding patient age should range between 4 and 9 years, For bitewing radiographs representing the permanent dentition, where only permanent teeth were present, the corresponding patient age should range between 12 and 30 years. For bitewing radiographs representing the permanent dentition, where only permanent teeth were present, the corresponding patient age should range between 12 and 30 years. Exclusion criteria The absence of radiographically detectable dental caries at the interproximal surface (distal and/or mesial) in both primary and permanent bitewing radiographs, The absence of radiographically detectable dental caries at the interproximal surface (distal and/or mesial) in both primary and permanent bitewing radiographs, For bitewing radiographs obtained during the primary or mixed dentition period, the patient must be outside the 4–9 age range, For bitewing radiographs obtained during the primary or mixed dentition period, the patient must be outside the 4–9 age range, For radiographs pertaining to permanent dentition, the patient’s age must be outside the 12–30 range, For radiographs pertaining to permanent dentition, the patient’s age must be outside the 12–30 range, Presence of distinct radiographic artifacts, Presence of distinct radiographic artifacts, Presence of dental anomalies involving the tooth crown and root. Presence of dental anomalies involving the tooth crown and root. For both groups, an inclusion requirement/criteria was that each bitewing radiograph contained at least one radiographically detectable interproximal carious lesion. Consequently, the dataset was enriched for positive (with caries lesion/s) cases and does not include completely caries-free bitewings. This design was chosen to ensure a sufficient number of positive examples for lesion-level segmentation and model training, but it did not reflect the full spectrum of real-world screening scenarios, in which many bitewing radiographs show no interproximal caries. Study groups and the distribution of included radiographic images Radiographic images including primary teeth and permanent teeth were included in the study and the characteristics of these groups were presented below. In total, 2000 bitewing radiographs (1000 primary dentition images in Group 1 and 1000 permanent dentition images in Group 2) were included. Across these images, 5704 interproximal carious lesions were annotated (2901 lesions in primary teeth and 2803 lesions in permanent teeth), as summarised in Tables 1 and 2. Table 1Number of radiographic images associated with training, validation and test datasetsDatasetPrimary tooth radiographic images (Group 1), nPermanent tooth radiographic images (Group 2), nTotal, nTraining, n8008001600Validation, n100100200Test, n100100200Total100010002000 Number of radiographic images associated with training, validation and test datasets Table 2Number of labels associated with training, validation and test datasetsDatasetPrimary tooth labels (Group 1), nPermanent tooth labels (Group 2), nTotal, nTraining, n237122814652Validation, n271263534Test, n259259518Total290128035704 Number of labels associated with training, validation and test datasets The primary dentition group comprised 946 patients contributing 1000 radiographs (mean age ± standard deviation: 6.43 ± 1.23 years; range: 4–9 years), of whom 562 (59.41%) were female and 384 (40.59%) were male. The permanent dentition group included 921 patients contributing 1000 radiographs (mean age ± standard deviation: 14.35 ± 3.44 years; range: 12–30 years), with 425 (46.15%) females and 496 (53.85%) males. Group 1: primary tooth radiographic images This group included a total of 1000 images previously taken for different indications from patients aged 4–9 years who were in the primary or mixed dentition period. In the radiographic images in this group, if permanent teeth were also present, the procedures were performed only based on the primary teeth. The distribution of the 1000 radiographic images included was planned as follows: 800 for training, 100 for validation and 100 for the test dataset (Fig. 1). Bitewing radiographic images of primary teeth (a and b) and permanent teeth (c and d) from the included dataset Group 2: permanent tooth radiographic images This group included a total of 1000 images taken from patients aged 12–30 years who were in the permanent dentition period. The distribution of the 1000 radiographic images included was planned as follows: 800 for training, 100 for validation and 100 for the test dataset (Fig. 1). Image anonymization and data security All digital radiographic images included in the current study have been anonymized via PACS so that the images cannot be linked to personal identification information. In this context, the patient’s name and surname, hospital system protocol number, date of birth and similar personally identifiable information have been deleted from the image files. The anonymized bitewing radiographs have been stored on a password-protected computer accessible only to researchers and closed to access by other individuals. AI model included in the study In this study, Yolov5x model, an extended variant of the Yolov5 architecture, was used. Yolov5x is a deep learning model that aims to detect small and ambiguous objects more successfully thanks to its deeper and wider layer structure. Model training was performed using the Ultralytics Yolov5 PyTorch library. The epoch count was set to 800 and the learning rate to 0.01 as the primary hyperparameters used during the training process. Labeling process All reference annotations were performed by a dentist working in the department of pediatric dentistry (H.S.G.). For each caries lesion, interproximal carious lesions were delineated using polygonal masks following a predefined annotation protocol that specified radiographic criteria for enamel and dentin involvement. Because the present project was conducted as a single-center thesis study, no additional clinicians were recruited as independent annotators, and formal inter- or intra-rater reliability analysis (e.g., κ statistics) was not performed. Also, no formal multi-reader calibration session was conducted. By design, there were no inter-rater disagreements to resolve; borderline cases were re-examined by the annotator according to the same predefined protocol. In all digital bitewing radiographic images of both primary and permanent teeth used in the current study, labeling procedures were completed manually to segment interproximal caries. Labeling was performed to cover all interproximal caries lesions in all teeth present in the radiographic images. In this context, even if permanent teeth were included in the bitewing radiographs of primary or mixed dentition radiographs (Group 1), the labeling were performed only for primary teeth and the permanent teeth in these images were excluded. However, all permanent teeth in Group 2 were included in the labeling procedures. Bitewing radiographs were uploaded to Craniocatch (Eskişehir, Turkey) labeling module and the boundaries of each carious lesion in each image were manually determined using a polygonal segmentation method. As a result of this process, output files in a data format suitable for training AI model were obtained. Segmentation results were recorded to include the relevant class labels and segmentation coordinates of the structures for each image. Throughout the labeling process, images were processed in random order; this approach was adopted to ensure that the model was not affected by systematic bias during the evaluation phase (Fig. 2). Images showing the labeling process of interproximal caries lesions on bitewing radiographic images of a primary tooth (Group 1, a) and a permanent tooth (Group 2, b) Procedure and datasets Three datasets were created for Yolov5x architecture used in this study. The datasets used consisted of a training dataset, a validation dataset and a test dataset. For each dentition group (primary and permanent teeth), 1000 bitewing radiographs were included. For both groups, the images were randomly partitioned at the radiograph level into training (80%), validation (10%), and test (10%) subsets (800, 100, and 100 images, respectively). The training set was used to fit the YOLOv5x-based instance segmentation model, the validation set was used for hyperparameter tuning and early stopping, and the test set was kept held out and only used once to obtain the final performance metrics. Because the dataset was organised and annotated at the radiograph level, the study did not enforce a strict patient-level separation. The number of radiographs and labels for the training, validation and test datasets for the primary (Group 1) and permanent tooth (Group 2) bitewing radiography groups included in the study were shown in detail in Tables 1 and 2. Model training and hyperparameters The training procedure and the dataset associated with this process were used to train AI model for caries diagnosis and segmentation. Accordingly, 800 radiographic images from each of primary tooth and permanent tooth bitewing radiographic image groups (Group 1 and Group 2, respectively) were randomly assigned to the training dataset. Accordingly, a total of 1600 bitewing radiographic images (n = 800 for Group 1, n = 800 for Group 2) were used for the training process of Yolov5x deep learning-based AI architecture. Yolov5x was trained for caries segmentation in primary and permanent teeth with a total of 4652 labels (n = 2371 for Group 1, n = 2281 for Group 2) to diagnose caries in primary and permanent teeth. After the training phase, the validation process was initiated. The YOLOv5x-Seg object detection and instance segmentation architecture (Ultralytics, open-source implementation) was used in a PyTorch environment. Training was performed for 800 epochs with a batch size of 4 and an input resolution of 1280 × 1280 pixels. The model was initialised with pretrained YOLOv5x-Seg weights trained on the COCO dataset (yolov5x-seg.pt). Input bitewing radiographs were resized to 1280 × 1280 pixels and preprocessed according to the standard YOLOv5 pipeline, including scaling of pixel values to the [0, 1] range after resizing and letterboxing. During training, YOLOv5’s built-in data augmentation mechanisms were enabled. The augmentation pipeline included colour jitter in hue, saturation and value (hsv_h = 0.015, hsv_s = 0.70, hsv_v = 0.40), random translation of up to 10% of the image size, random scaling of up to 50%, horizontal flipping with a probability of 0.5, and mosaic augmentation (probability 1.0). Optimisation was performed using stochastic gradient descent (SGD) with an initial learning rate (lr0) of 0.01, momentum of 0.937 and weight decay of 0.0005. A short warm-up phase was applied at the start of training (3 warm-up epochs), with warm-up momentum set to 0.8 and a warm-up learning rate of 0.1 for bias parameters. Early stopping was employed with a patience of 100 epochs, according to the default YOLOv5 validation-based stopping criterion, to prevent overfitting and to select the best-performing model checkpoint on the validation set. Validation stage of AI model For the optimization of Yolov5x architecture and the adjustment of its hyperparameters, a dataset consisting of 100 bitewing radiographic images each (10% of the total dataset) was used for validation purposes (n = 100 for Group 1, n = 100 for Group 2). This dataset was used to monitor the learning performance during model training and to prevent overfitting. Accordingly, Yolov5x model was validated with a total of 534 labels (n = 271 for Group 1, n = 263 for Group 2). During the training of Yolov5x deep learning-based AI model, the loss and performance metrics on the validation set were tracked at the end of each epoch. At this point, an early stopping strategy was applied and training was terminated if a certain number of consecutive improvements in validation loss were not observed. The best model weights obtained during the validation phase were saved for final evaluation on the test set and performance measurements were performed using only the model showing this “best validation score.” This ensured that model selection was made objectively, adhering to the separation of training and test data. Hyperparameter optimization was performed using a grid initially determined through experimental trials, and the most suitable configuration was selected based on the training-validation balance. Following the validation phase, the model’s performance was measured in the testing process. Test stage of AI model The test dataset is an isolated data set used to evaluate the model performance in a neutral and objective manner on labels that AI architecture has not seen before. During the testing phase of Yolov5x architecture, a dataset consisting of 100 radiographic images each (10% of the total dataset) was used (n = 100 for Group 1, n = 100 for Group 2). Accordingly, a total of 200 bitewing radiographic images were used for Yolov5x testing process and Yolov5x model was tested with a total of 518 labels (n = 259 for Group 1, n = 259 for Group 2). PyTorch library was used for training, evaluating and performing analyses of Yolov5x model, as well as for visualizing and analyzing the results. Interpretation of results and performance evaluation metrics Model performance was evaluated at the lesion (caries) level. For each dentition group, predicted segmentation masks output by the YOLOv5x model were matched to ground-truth masks using the intersection-over-union (IoU) criterion. A predicted lesion was counted as a true positive (TP) if its IoU with a ground-truth lesion was ≥ 0.5. If multiple predictions overlapped the same ground-truth lesion, matching was resolved in a one-to-one manner by assigning the prediction with the highest IoU as TP, and any additional overlapping predictions were counted as false positives (FP). Predicted lesions with IoU < 0.5 to all ground-truth lesions were also counted as FP, whereas ground-truth lesions without any matching prediction at IoU ≥ 0.5 were counted as false negatives (FN). Lesion-level precision, recall, and F1 score were then computed in the standard way as precision = TP/(TP + FP), recall = TP/(TP + FN), and F1 = 2·(precision·recall)/(precision + recall). Receiver operating characteristic (ROC) curves and ROC–AUC values were also computed at the lesion level. For each predicted lesion, the model outputs a confidence score. By varying the confidence threshold applied to these scores, while keeping the IoU criterion fixed at 0.5 for lesion matching, this study obtained a sequence of lesion-level sensitivity (recall) and corresponding false-positive rate values and calculated the area under the ROC curve (ROC–AUC) for each dentition group. Because the dataset was enriched for images containing at least one interproximal caries lesion and non-carious structures were not exhaustively annotated, true negatives (TN) at the image, tooth, or surface level could not be reliably quantified. For this reason, the study procedure did not report specificity, negative predictive value (NPV), or accuracy based on TN counts, and the F1 score should not be interpreted as a global patient- or image-level diagnostic accuracy measure. The metric formulations included in the study were presented in Table 3 below (modified from Gumussoy et al. [17]). Table 3Performance metrics used in the studyMetricsMetric FormulaTrue Positive (TP)Number of correctly detected lesionsFalse Positive (FP)Number of non-lesion regions incorrectly detected as lesionsFalse Negative (FN)Number of lesions missed by the modelPrecisionTP / (TP + FP)Recall (Sensitivity)TP / (TP + FN)F1-Score2 × (Precision × Recall) / (Precision + Recall)ROC–AUCArea under the ROC curve derived from lesion-level TPR vs. FPR across confidence thresholds (IoU ≥ 0.5) Performance metrics used in the study Results In this study, TP, FP, FN, precision, sensitivity, F1 score and ROC-AUC metrics were analyzed in detail on bitewing radiographs belonging to both the primary (Group 1) and permanent tooth groups (Group 2). The findings obtained within the study were presented in detail below under two separate headings: Primary Tooth Radiographic Images and Permanent Tooth Radiographic Images. Group 1: results of primary tooth radiographic images During the test, interproximal caries segmented by Yolov5x model were marked with segmentation masks in the relevant areas. The images showed that the model can identify both single and multiple lesions with high precision (Fig. 3). Sample images of Yolov5x model in the testing phase for the diagnosis of interproximal caries in bitewing radiographs of primary teeth. The red areas indicate the caries marked by the model In segmentation-based analyses performed on radiographic images of primary teeth, Yolov5x model identified 202 TP, 52 FP and 57 FN segmentations for the diagnosis of caries lesions located on the interproximal surface during the test phase. Based on these results, the precision value of Yolov5x model for this group was determined to be 0.80, sensitivity (recall) value was 0.78 and the F1 score was 0.79. In addition, AUC calculated in ROC analysis was determined to be 0.87 (Table 4). Table 4Performance metrics of Yolov5x model on primary tooth radiographsMetricValueTP202FP52FN57Precision0.80Sensitivity0.78F1 Score0.79ROC-AUC0.87 Performance metrics of Yolov5x model on primary tooth radiographs The confusion matrix obtained for primary tooth radiographic images also visually supported the model’s classification ability for clinical use (Fig. 4). Confusion matrix obtained for radiographic images of primary teeth (TP = 202, FP = 52 and FN = 57) The model’s ROC analysis indicated strong discrimination performance with an AUC of 0.87. The steep initial rise of the curve suggests that high TP can be achieved at low FP levels and that the model can effectively capture positive cases at early thresholds. This result indicates that a randomly selected positive example is approximately 87% more likely to be assigned a higher probability/score than a randomly selected negative example (Fig. 5). ROC curve for the evaluated model for radiographic images of primary teeth Group 2: results of permanent tooth radiographic images During the testing phase of the model, interproximal caries segmented by Yolov5x model were marked along with segmentation masks in the relevant areas. The images show that the model can identify both single and multiple lesions with high precision (Fig. 6). Sample images of Yolov5x model in the testing phase for the diagnosis of interproximal caries in bitewing radiographs of permanent teeth. The red areas indicate the caries marked by the model In segmentation-based analyses performed on radiographic images of permanent teeth, Yolov5x model identified 193 TP, 54 FP and 66 FN segmentations for the diagnosis of caries lesions located on the interproximal area during the test phase. Based on these results, the precision value of Yolov5x model for this group was determined to be 0.78, sensitivity (recall) value was 0.75 and the F1 score was 0.76. In addition, AUC calculated in ROC analysis was determined to be 0.76 (Table 5). Table 5Performance metrics of Yolov5x model on permanent tooth radiographsMetricValueTP193FP54FN66Precision0.78Sensitivity0.75F1 Score0.76ROC-AUC0.76 Performance metrics of Yolov5x model on permanent tooth radiographs The confusion matrix obtained for permanent tooth radiographic images also visually supported the model’s classification ability for clinical use (Fig. 7). Confusion matrix obtained for radiographic images of permanent teeth (TP = 193, FP = 54 and FN = 66) ROC analysis showed that the model achieved an AUC of 0.76, indicating that the probability of assigning a higher score to a randomly selected positive example than to a randomly selected negative example is approximately 76%. The steep rise at the beginning of the curve indicates that high TP can be obtained at low FP levels, suggesting that the model captures positive cases relatively effectively at early thresholds (Fig. 8). ROC curve for the evaluated model for radiographic images of permanent teeth Discussion Dental caries, one of the most common diseases in the population, is a multifactorial disease that occurs as a result of complex interactions in the hard tissues of the teeth caused by acids produced by bacteria that can metabolize fermentable carbohydrates, and its severity varies depending on various factors [18–20]. If demineralized lesions without cavitation are left untreated, the increased acid production caused by cariogenic bacteria in the oral flora demineralizes the tooth surface, resulting in lesions called carious lesions due to the disruption of the enamel surface integrity. However, untreated carious dental lesions seen in advanced cases cause various physical symptoms such as severe pain and the development of dental abscesses, as well as psychological symptoms, resulting in varying degrees of decline in quality of life [18–22]. Early diagnosis of caries is extremely important for ensuring more effective treatment. Today, visual and tactile examination methods, which are among the traditional caries diagnosis methods, and radiological examination are frequently preferred as standard caries diagnosis methods in routine clinical practice [21, 23]. It has been reported that visual examination methods are effective in correctly identifying healthy surfaces, but their sensitivity in accurately detecting caries lesions is low [24]. In their study, Mialhe et al. [25] reported that the sensitivity rate of interproximal caries lesions detected by visual examination in permanent teeth was approximately 30%. Tactile assessments performed alongside visual examination are generally carried out with the aid of a dental probe and there are some limitations to the use of probes. The mechanical force applied during physical examination with a probe can cause structural damage to areas on tooth surfaces that have the potential for remineralization. In this regard, the early and effective detection of interproximal caries is crucial [23]. Radiographic examinations, in addition to visual and tactile examinations, increase the diagnostic reliability in the detection of dental caries [26, 27]. Panoramic radiographs are used as an important diagnostic tool in general intraoral evaluation and preliminary diagnosis. However, panoramic radiographs have disadvantages such as low resolution, image blurring, distortion, and overlapping of dental and anatomical structures [28]. Due to these disadvantages, the use of periapical and bitewing radiographs in addition to panoramic radiographs has become a common clinical practice to clarify the diagnosis and enable detailed evaluation [29]. Wenzel [30] demonstrated that the use of bitewing radiographs in addition to visual examination yielded more reliable and objective results during the detection of carious lesions compared to evaluations based solely on clinical examination. Bitewing radiographs, in particular, provide important clinical contributions to clinicians’ ability to make accurate treatment decisions on both occlusal and approximal surfaces [30, 31]. In recent years, technological developments have guided clinicians in the creation of diagnosis and treatment plans in dentistry. In this context, one of the phenomena gaining popularity in dentistry, as in many other fields, is AI and other methods related to AI [32–34]. It is stated that AI-supported medical imaging methods can produce faster, more reliable, and more efficient results. Thus, it has been stated that the use of AI in medical radiology will make significant contributions to healthcare by offering early intervention opportunities and personalized treatment approaches [35]. Numerous studies on the use of AI and related methods in dentistry from the past to the present constitute the scope of the current literature [32–34]. AI-supported image processing techniques have been successfully applied to panoramic, bitewing and periapical radiographs for the automatic detection/diagnosis of dental caries [8, 36, 37]. Although there is no standard method for the type of radiograph used in AI-based studies, bitewing radiographs taken for primary and permanent teeth were included in the current study as they are a more effective method for diagnosing caries lesions. A review of the dental literature reveals that there are various AI architectures with diverse features that can be used in the detection and diagnosis of dental caries. Lee et al. [36] reported that they achieved high accuracy/AUC values in detecting dental caries using a deep convolutional neural network (Inception v3) in periapical radiographs, suggesting that such deep learning algorithms can be used effectively and clinically useful in caries diagnosis. In another comprehensive study, Moran et al. [8] proposed a novel method combining image processing techniques and convolutional neural networks to identify interproximal caries lesions in bitewing radiographs and classify them according to lesion severity. The data obtained suggest that the proposed method can be used to assist dentists in evaluating bitewing images, identifying lesion severity and determining appropriate treatments [8]. Bayrakdar et al. [9] reported in their study that deep convolutional neural networks (VGG-16 and U-Net) effectively diagnosed dental caries. In this study, it was preferred to include Yolov5x, the fifth version of the AI-based family of algorithms called Yolo, in the working procedures. Yolov5x is an optimized deep learning model that offers high accuracy and speed performance in real-time object recognition tasks [38, 39]. The model’s ability to deliver effective results even at small sizes, its high suitability for use with dental radiographs of different resolutions, and the parametric flexibility it provides during the training process made it compatible with the purpose of the current study. Furthermore, Yolov5x AI-based object detection architecture’s fast decision-making capability and performance, which enhance its clinical applicability, make it a strong candidate for meeting the need for rapid and reliable diagnosis in pediatric patients. The main findings of the current study indicate that the precision, sensitivity and F1 score for primary teeth group were 0.80, 0.78, and 0.79, respectively, while for permanent teeth group, these values were 0.78, 0.75, and 0.76, respectively. Although there is a slight difference between these results, they indicate that the AI-based object detection architecture examined in this study diagnosed interproximal caries in bitewing radiographs of primary and permanent teeth with approximately similar performance. While similarities were found between the results of this study and those of previous studies in the dental literature in some areas, differences exist in other areas, both due to methodological differences, such as the use of different AI-based models, and due to the sample characteristics and features included in the study. Ying et al. [40] compared YOLOv5, Trans-UNet, DETR and U-Net in periapical X-rays; YOLOv5 F1 = 0.87 showed the highest performance. The ranking is Trans-UNet (0.86), DETR (0.82), and U-Net (0.80). For the Youden index, a combined indicator of sensitivity and specificity, the respective values reported were 0.76, 0.73, 0.69, and 0.64; thus, YOLOv5 stood out in caries detection [40]. Similarly, in a study conducted by Pérez de Frutos et al. [41], the deep learning system called AI-Dentify was developed to detect approximal caries in bitewing radiographs. The study utilized a total of 13,887 bitewing images annotated by six experienced dentists. The images were used to train the RetinaNet, Yolov5 and EfficientDet models. According to the results obtained, Yolov5 achieved the highest diagnostic accuracy with an F1 score of 54.8%. The RetinaNet and EfficientDet models showed relatively lower performance. Based on these findings, it was stated that Yolov5 algorithm can work with high accuracy even on large-scale, expert-labeled datasets and is a reliable choice for dental image analysis. Panyarak et al. [42], in their comparative study using 2575 bitewing images, evaluated the Yolov3 and Yolov7 models and reported F1 scores of 0.375 and 0.555, respectively. In this study, Yolov5x model found F1 scores of 76% for permanent teeth and 79% for primary teeth; differences from previous studies may be attributed to methodological design features, the characteristics of the radiographic images used, and the nature of the radiographs used in the training and validation stages. Although the current study did not include images with detectable distinct radiographic artifacts, even a loss of cooperation that may occur when taking radiographs from pediatric patients, especially compared to adult patients, could cause the results to be different. In a general assessment, the performance observed in the present study appears to lie within the upper range of values reported in current YOLO-based caries studies; this finding suggests that YOLOv5-based architectures may represent a promising alternative for assisting in the radiographic detection of interproximal caries, although such cross-study comparisons should be interpreted cautiously due to differences in datasets, annotation protocols, and evaluation metrics. Furthermore, this success rate demonstrates that the model exhibits balanced performance in terms of both precision and recall and that the training dataset contributes significantly to the AI system. When reviewing previous studies in the literature, it is observed that the performance of sub-algorithms belonging to the YOLO family is compared with each other and the results are reported in this context. Ayhan et al. [13] compared 11 models, including YOLOv5 (n/s/m/l), YOLOv7 and YOLOv7-tiny, YOLOv9 (t/s/m/c), and the improved YOLOv9c-Faster, to detect enamel and dentin caries using a total of 2150 bitewing radiographs. YOLOv9c-Faster achieved the highest performance on the test set with micro-average sensitivity = 0.727 and F1 = 0.687 [13]. In the present study, the Yolov5x model yielded sensitivity and F1 scores that were numerically higher than the values reported for YOLOv9c-Faster in the study by Ayhan et al. [13]; however, this difference should be interpreted with caution, as the models were trained and evaluated on different datasets under non-identical experimental conditions. This difference may be attributed to the optimization process not yet being complete due to the YOLOv9-c model being a newly developed architecture. Additionally, the use of different datasets, annotation quality and parameters used during model training can also be considered factors contributing to performance differences. Similarly, in another study including other members of the YOLO family, Yavsan et al. [43] compared a total of 13 models based on YOLOv4/5/6/7 and the latest YOLOv8, using 415 periapical radiographs collected from pediatric patients aged 5–12 years for the detection of interproximal caries, under the same 80% training-20% testing split. The evaluation was accuracy-based and performed using mAP; YOLOv5s model showed the highest performance with mAP = 90.8%, precision = 91.2%, recall = 89.3%, and F1 = 90.24%. The findings indicate that Yolov5s algorithm is an effective, reliable and clinically applicable AI solution for diagnosing interfacial caries in periapical radiographs [43]. In contrast, the current study preferred Yolov5x version and used bitewing radiographs, which are considered the gold standard for caries diagnosis, unlike the periapical radiographs used in the studies by Yavsan et al. [43]. This difference is particularly important in terms of the diagnostic clarity of approximal caries. Furthermore, the dataset used in our study is more comprehensive in terms of sample size and annotation count, which positively contributes to the model’s generalizability and diagnostic success. Thanks to Yolov5x architecture’s ability to process high-resolution details more effectively, its potential to produce consistent results, especially in clinical applications, has increased. Furthermore, in a study conducted by Kunt et al. [44], a comprehensive dataset consisting of a total of 3989 bitewing radiographs containing permanent teeth was created, and these images were used to train convolutional neural network architectures capable of detecting various objects [44]. The evaluated models included current and effective architectures such as Yolov5, Faster R-CNN, RetinaNet and EfficientDet. The success of the models was evaluated using various performance metrics such as F1 score. F1 scores of the 12 different architectures tested in this study were reported to be in the range of 0.72 to 0.76. These results are seen to be of a similar nature to our study. In summary, compared with previous YOLO-based caries studies [13, 40–44], the current work differs in three main respects. First of all, the dataset composition focuses on pediatric and young-adult patients and explicitly distinguishes primary and permanent dentitions on bitewing radiographs, whereas many earlier YOLO studies primarily used periapical images and/or adult populations. Second, present annotation protocol is lesion-centred and uses polygonal masks for interproximal caries segmentation, instead of tooth-level labels or bounding-box annotations. Third, the evaluation is performed at the lesion level with separate confusion matrices and ROC–AUC values for primary versus permanent teeth, providing clinically interpretable metrics that are directly aligned with decision-making about interproximal caries in pediatric dentistry. AI-supported caries detection studies targeting primary teeth are gaining increasing attention, albeit with a limited number of studies. Gonzalez et al. [45] developed a no-code AI model for detecting primary proximal caries from bitewing radiographs of pediatric patients. After inclusion, the model was tested on 66 bitewings (755 surfaces) and achieved a performance of F1 = 0.92, sensitivity = 88.8%, specificity = 98.8%, and ROC-AUC = 0.965. This high success can be attributed to the quality of the dataset and the controlled conditions in the limited sample. This study demonstrates the applicability of deep learning applications for segmentation approximal caries in primary teeth. However, it should not be overlooked that factors such as model architecture, labeling method, sample size, and data diversity have decisive effects on the results. In this context, F1 score of 0.79 obtained in our study by analyzing 1000 bitewing radiographs of primary teeth using the Yolov5x architecture is remarkable in terms of both sample size and model success when compared to existing studies in the literature. This result indicates that the Yolov5x architecture can achieve strong lesion-level diagnostic performance for primary tooth decay in a caries-enriched sample, while recognising that direct numerical comparisons with other studies are limited by differences in study design, patient populations, and evaluation protocols. In the current study, although there is a minor difference between permanent tooth radiographs, it is thought that there may be various physiological and radiographic reasons why the model performs better in primary teeth. Firstly, the enamel and dentin layers of primary teeth are thinner and this causes the decay to reach the dentin, resulting in higher contrast lesions on the radiograph [30]. This increase in contrast allows deep learning-based models to distinguish visual patterns more easily. Furthermore, caries progression is generally faster in children, which also leads to lesions being observed with clearer boundaries and spreading over larger areas [46]. This morphological distinctiveness provides an advantage for fast object recognition and segmentation algorithms such as Yolov5. Furthermore, the fact that tooth structures in images of pediatric patients are less worn may have facilitated the AI model’s decision-making process by increasing the lesion-to-contrast ratio. However, the involvement of a researcher working in pediatric dentistry in the annotation process may have contributed to the accuracy of the model training by improving the quality of labeling and annotation. On the other hand, the model’s relatively lower performance, albeit to a lesser extent, on permanent teeth may be related to the fact that caries in these teeth are relatively less visible on radiographs in their early stages. Furthermore, image artifacts such as tartar buildup or overlapping are more frequently observed in older patients, which may negatively affect the model’s segmentation success. However, distinguishing caries contours is challenging, especially for a model trained with limited annotations, due to the more crowded proximal areas in permanent teeth. These factors are thought to be possible contributors to the minor differences in Yolov5x model’s performance in diagnosing interproximal caries in both primary and permanent teeth. This study has several strengths and contributes novelty to the literature. From an algorithmic perspective, the YOLOv5x architecture and training procedure used in this study are conventional and build on existing YOLO-based work; the current study does not claim to introduce a new detection or segmentation algorithm. Instead, the main contributions of the present study are primarily application- and validation-oriented. First, the present procedure curated a bitewing radiograph dataset including both primary and permanent teeth, in which 5704 interproximal carious lesions were delineated using polygonal instance segmentation. Second, we trained and evaluated a YOLOv5x model separately in primary and permanent dentitions and reported lesion-level, clinically interpretable metrics (true positives, false positives and false negatives, F1 score, and ROC–AUC) for each group. These design choices enable a direct comparison of model performance across dentitions in a paediatric and young-adult population and provide a lesion-level reference that can be mapped to screening and treatment decision thresholds in pediatric dentistry. As with any AI-based study, certain limitations of the current study should be considered. First, the dataset used to train the model is single-center, which may limit the model’s generalizability to some extent. Related to this, while this is meaningful for methodological standardization, the diversity that may arise from different clinical settings and imaging devices may not be reflected in the results. On the other hand, from an evaluation-design perspective, this study has additional limitations. The train–validation–test split was performed at the radiograph level rather than the patient level. As a result, it is possible that multiple radiographs from the same patient contributed to different subsets, which may introduce some degree of statistical dependence between training, validation, and test images. Although this design reflects the way the dataset was originally curated and annotated, future studies should prioritise patient-level partitioning and, where feasible, cross-validated evaluation schemes. Another important limitation is the absence of a human-reader baseline. In this retrospective design, the study did not conduct a controlled observer in which pediatric dentists or radiologists re-evaluated the test set under standardised conditions and were compared directly with the model. As a result, the current findings should be interpreted as complementary to, rather than a replacement for, expert clinical judgment, and future work should include side-by-side evaluation of YOLO-based systems and experienced clinicians. Additionally, it should be emphasised that all reported metrics are instance-level segmentation measures rather than patient-level diagnostic outcomes. Lesion-level F1 and ROC–AUC quantify how accurately individual interproximal lesions are localised and classified within a caries-enriched dataset; they do not directly correspond to image-level screening performance or yield realistic positive and negative predictive values at the patient level. On the other hand, a further limitation relates to lesion selection. Only radiographically detectable interproximal carious lesions were included in the annotations, and early/incipient lesions that were clinically suspected but not visible on bitewing radiographs were not labelled. Consequently, the model was not trained to recognise the earliest stages of caries and the reported performance cannot be generalised to detection of radiographically occult or very early lesions. A further limitation relates to dataset composition and caries prevalence. By design, all bitewing radiographs included in this study contained at least one radiographically detectable interproximal carious lesion, resulting in a caries-enriched dataset. While this approach was useful for training and evaluating lesion-level segmentation, it did not mirror real-world screening settings, where a substantial proportion of bitewings are caries-free or contain only incipient lesions. As a consequence, the present work cannot directly estimate realistic image-level screening performance, such as positive or negative predictive values, or quantify the absolute false-positive burden per patient. Lesion-level F1 score and ROC–AUC are relatively robust to changes in prevalence, but in clinical deployment in lower-prevalence populations, positive predictive value at a fixed decision threshold would be expected to decrease. Future studies should therefore include caries-free bitewings and more representative prevalence distributions, and report both lesion-level and image-level performance under real-world conditions. A further limitation of the study is that the YOLOv5x model was evaluated in isolation, without the use of parallel baseline models or human-reader comparators that had been trained or tested on the same dataset. No simpler machine learning/artificial intelligence architectures (such as U-Net-style segmentation networks or smaller YOLO variants) and no independent paediatric dentists or radiologists were included as direct baselines under identical experimental conditions. Consequently, the reported results should be interpreted as a single-model evaluation rather than as evidence of superiority over alternative algorithms or human experts. As a means of providing context, qualitative comparisons with published F1 and ROC-AUC values from other studies are presented. It should be noted, however, that these comparisons do not constitute rigorous, controlled head-to-head comparisons. This is due to the substantial influence that cross-study differences in datasets, case mix, prevalence, annotation protocols, and performance metrics can have on the reported values. In addition, all performance metrics are based on a single 80/10/10 train–validation–test split without confidence intervals, repeated experiments, or formal statistical tests comparing primary and permanent dentitions; the reported differences between groups should therefore be interpreted as descriptive rather than statistically confirmed. Conclusions The present study posits that the Yolov5x architecture can attain commendable lesion-level performance for interproximal caries in both primary and permanent teeth, with results that are largely commensurate with those documented in analogous studies. These findings may be regarded as a preliminary step towards the integration of AI-based radiographic decision-support systems into clinical practice, rather than as evidence of a fully validated diagnostic tool. The utilisation of AI-based systems holds considerable promise in facilitating enhanced diagnostic outputs, particularly in paediatric cases, and their deployment as a second-opinion tool in oral health assessments is a promising avenue for future research. Nevertheless, further research in the form of multicentre, prospective, and externally validated studies is required in order to confirm the generalisability of the present results. In this context, future investigations should be supported by age-balanced and multicentre datasets; training strategies could be explored using separate models for primary and permanent teeth or multi-input architectures that directly incorporate tooth-type differences. In addition, comparative analyses with segmentation architectures that are specifically optimised for detecting small lesions may have the potential to further improve diagnostic performance. Acknowledgements This study was also supported by TÜBİTAK-BİDEB and the authors gratefully acknowledge this support. We also would like to express our gratitude to CranioCatch, Associate Professor Elif Bilgir, and Professor İbrahim Şevki Bayrakdar for their kind contributions. Abbreviations American Academy of Pediatric Dentistry Artificial Intelligence You Only Look Once Picture Archiving and Communication System Stochastic Gradient Descent Intersection over Union True Positive False Positive False Negative Receiver Operating Characteristic Area Under Curve True Negative Negative Predictive Value Authors’ contributions H.S.G., A.D. and K.O. conceived and designed the procedure; H.S.G. and A.D. performed the experimental procedure; H.S.G. and A.D. analyzed the data; H.S.G., A.D. and K.O. prepared figures and tables; H.S.G., A.D. and K.O. authored or reviewed drafts of the article; H.S.G., A.D. and K.O. approved the final draft. Funding This study was derived from a doctoral thesis and this work has been supported by Ankara University Scientific Research Projects Coordination Unit under grant number TDK-2024-3520. Data availability The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. Declarations Ethics approval and consent to participate Ethical approval for the current study was obtained from Ankara University Faculty of Dentistry Clinical Research Ethics Committee (Date: 06.11.2023, Decision: 12/11). The study protocol was also conducted in accordance with the World Medical Association’s Declaration of Helsinki ethical principles for medical research involving human participants, including research using identifiable human material or data (https://www.wma.net/policies-post/wma-declaration-of-helsinki/). The study was also conducted retrospectively using radiographs retrieved from our institution’s PACS system. In accordance with the routine institutional workflow, consent for radiographic imaging was obtained from patients and/or their parents at the time of image acquisition. Given the retrospective design and the use of fully anonymized images obtained under routine clinical consent, the ethics committee confirmed that no additional informed consent specific to research use was required. Furthermore, the use of included radiographs for research purposes of this study was approved by the institutional ethics committee and supported by the necessary institutional permissions. Consent for publication Not applicable. Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.