Comparison of the diagnostic accuracy of dentists and ChatGPT in jawbone lesions Authors: Nezahat Sena Satan, Umut Pamukçu, Barış Erkut Türk, Zühre Akarslan, Sibel Açık Kemaloğlu, Ilkay Peker Published: 2026 ABSTRACT Background/objectivesArtificial intelligence (AI) is leading to a significant paradigm shift in medical imaging and diagnostic sciences. In particular, Chat Generative Pre-trained Transformer (ChatGPT) is finding increasing application in diagnostic processes due to its ability to generate clinical outcomes. This study aims to evaluate the diagnostic accuracy of ChatGPT for jawbone lesions and also to compare it with that of Oral and Maxillofacial Radiologists (OMFR), Oral and Maxillofacial Surgeons (OMFS), and general dentists.Materials & methodsThirty cases with jawbone lesions, for which clinical information, panoramic radiographs, and histopathological diagnoses were available, were selected. A questionnaire was prepared, including participants’ (OMFR, OMFS, and general dentists) demographic information, the cases’ clinical findings and panoramic radiographs, and distributed via electronic communication channels. The same cases were loaded into ChatGPT-4 and asked to generate a preliminary diagnosis. The data were statistically analyzed using the Wilcoxon Signed Rank, Mann–Whitney U, and Kruskal–Wallis tests at a significance level of p < 0.05.ResultsOverall, ChatGPT’s diagnostic accuracy was limited to 46.67%, while the OMFR (67.71%) and OMFS (58.96%) groups had statistically significantly higher success rates (p < 0.05) than ChatGPT. General dentists (37.85%) had lower or similar diagnostic accuracy compared to ChatGPT in most subgroups (gender, age, workplace, professional experience).ConclusionsChatGPT demonstrated moderate diagnostic accuracy. While OMFR and OMFS participants had significantly higher accuracy rates than ChatGPT, ChatGPT generally outperformed general dentists. These results indicate that such AI systems cannot replace specialist clinicians but can provide valuable contributions as supportive tools that enhance diagnosis.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12903-026-08623-w. Introduction Rapid advances in Artificial Intelligence (AI) have significantly impacted medical imaging [1]. Large Language Models (LLMs), have attracted significant attention in the healthcare field in recent years due to their potential to analyze text-based data, synthesize clinical information, and support decision-making [1–6]. Among these models, Chat Generative Pre-trained Transformer (ChatGPT) is considered a helpful tool in diagnostic processes due to its ability to understand natural language and generate structured clinical outputs [1, 7–9]. Jawbone lesions require a complex interpretation of clinical, radiological, and histopathological findings. The correct interpretation of imaging findings, establishment of preliminary diagnosis and differential diagnosis, and ensuring histopathological correlation are critical for diagnosing lesions [10–12]. However, in clinical practice, case complexity, time constraints, and differences in experience can lead to diagnostic errors. At this point, it is thought that AI-based decision support systems can play an important role in increasing diagnostic accuracy. In situations requiring expert opinion or a multidisciplinary approach, existing conditions may not always allow for collaboration [5, 7, 13–15]. ChatGPT can be easily accessed from any internet-enabled environment without requiring additional software, hardware, or advanced technical knowledge. This makes it a practical, flexible, and user-friendly tool, especially in time-sensitive situations, compared to more complex AI systems that require specialized infrastructure and expertise. ChatGPT, the most popular example of an LLM, stands out for its ability to generate differential diagnoses and answer complex medical questions by analyzing text-based descriptions of clinical scenarios, symptoms, and radiological/histopathological reports [1, 2, 16, 17]. Studies show that current versions, particularly ChatGPT-4 and GPT-40, exhibit high accuracy rates in diagnosing oral and maxillofacial pathologies, sometimes approaching the performance of specialist physicians [7, 14, 16]. ChatGPT’s ability to provide instant access to vast literature and rapidly synthesized information makes it a valuable “digital assistant” and educational resource for clinicians [2, 18]. However, the model’s biggest limitation in clinical processes is its inability to directly analyze radiographs and histopathological sections and its reliance solely on text-based descriptions [8, 14, 19, 20]. Therefore, the literature emphasizes that ChatGPT, at its current stage, should be used not as an independent diagnostic authority, but rather as an auxiliary tool complementing clinical judgment under the supervision of a specialist physician [2, 16, 18]. Recent studies have demonstrated ChatGPT’s capacity to generate preliminary and differential diagnoses from clinical, radiological, and histopathological data; it has been emphasized that newer versions offer higher accuracy, but this accuracy depends on data quality and standardization [14, 19, 21]. Most existing research is text-based; however, there is a lack of multidisciplinary studies directly comparing ChatGPT’s performance with specialist dentists in evaluating panoramic radiographs [2, 5, 7, 22, 23]. This study aims to evaluate ChatGPT’s diagnostic accuracy for common jawbone lesions compared to Oral and Maxillofacial Radiologists (OMFR), Oral and Maxillofacial Surgeons (OMFS), and general dentists. Materials and methods Ethical approval and collection of data Ethical approval for this study was obtained from the Gazi University Ethics Committee in accordance with the guidelines of the Declaration of Helsinki (Research Code No: 2025 – 84, Date: 14.01.2025). The initial examination clinic and radiology archive of Gazi University of Dentistry were utilized. Clinical, radiological, and histopathological data from patients who visited this clinic for any dental reason and were registered in the Hospital Information Management System (HIMS) were used. Panoramic radiographs obtained with the same Sirona Orthophos XG device (70 kVp, 8 mA, 14 seconds; Sirona, Bensheim, Germany) in the radiology clinic, with positioning and irradiation settings in accordance with the manufacturer's instructions, were scanned using the archive records. The images found during the archive scan were from patients who had undergone surgery related to the lesion and had a histopathological report for the lesion recorded in HIMS. The preparation of the cases Cases were prepared with data on different jawbone lesions, including panoramic radiographs taken before surgery and brief patient clinical history, with localized jawbone lesions confirmed by histopathology. Case details include age, gender, presenting complaints, and clinical findings. The evaluation of panoramic radiographic images obtained from the radiology archive was performed on the same 24-inch medical monitor with an ideal screen display (resolution: 1920 × 1080 pixels) (Philips, Luchu Hsiang, Taiwan) with an NVIDIA QUADRO FX 380 graphics card, in a dimly lit and quiet room at a distance of approximately 50 cm. Preliminary diagnoses were evaluated based on complete clinical information, panoramic radiographs, and histopathological reports. A total of 96 images were selected from twelve different jawbone lesions. The selection of panoramic radiography images used was performed by a maxillofacial radiology specialist (N.S.S.). The selected panoramic radiographs were evaluated for compliance with the criteria by maxillofacial radiology specialists (U.P., İ.P., Z.A.) with at least 10 years of experience. A total of 50 images, five from each of ten different jawbone lesions, were selected by consensus to ensure diagnostic diversity and prevent overrepresentation. Before starting the study, a pilot study was conducted with 10 dentists to assess the clarity, coherence, and time required to answer the questions. As the participants in the pilot study indicated that the number of questions was excessive and time-consuming, the total number of images (three per lesion instead of five) was reduced to 30. The included lesions were selected from the most common jawbone lesions according to the 2022 (5th) edition of the World Health Organization (WHO) classification [10]. Accordingly, the included jawbone lesions were: radicular cyst, residual cyst, dentigerous cyst, odontogenic keratocyst, nasopalatine canal cyst, ameloblastoma, odontoma, simple bone cyst, fibro-osseous lesions, and staff bone cavity [10]. During the selection process, particular attention was paid to the typical findings most exhibited by the lesions. Brief clinical histories of the patients and panoramic radiographic images were provided. To facilitate participant selection and save time, 26 options were presented, and participants were asked to select one for each case. To ensure that attention was focused solely on the relevant pathology in the radiographs, each lesion was highlighted by outlining it with a red line (Fig. 1). Fig. 1Representative examples of jawbone lesions included in the dataset. A few example jawbone lesions from the dataset Representative examples of jawbone lesions included in the dataset. A few example jawbone lesions from the dataset Inquiry into the cases While the case questions were presented to participants in Turkish, the cases were also translated into English because ChatGPT performed better in English. Both the dentist participants and ChatGPT were shown the panoramic radiograph of the case, with the lesion area marked in red (Fig. 2). Fig. 2Case question example prepared for participants and ChatGPT (Ameloblastoma) Case question example prepared for participants and ChatGPT (Ameloblastoma) Care was taken to ensure that the content of the questions asked to the participants and the questions posed to ChatGPT were identical. On August 18, 2025, all cases were tested, and all responses from the model were recorded. All participants were presented with the cases in the same order. After each case was submitted to ChatGPT, the chat history was cleared to prevent previous responses from influencing new evaluations, and the process was restarted by starting a new session for each case [24]. The question was asked once by the ChatGPT interface, and no permission was given to regenerate the response. The model was not asked to generate different diagnoses or probability estimates, and no feedback or corrections were provided throughout the process. Additionally, the evaluation process has been standardized by issuing the “I will provide you with brief anamnesis information and panoramic radiographs of the patients; please state only the single most probable preliminary diagnosis” command to the chatbot before each case (Fig. 3). Apart from the standard instructions mentioned above, ChatGPT received no external guidance. Fig. 3Case data examples entered the ChatGPT interface and ChatGPT’s responses (Ameloblastoma and Stafne bone cavity) Case data examples entered the ChatGPT interface and ChatGPT’s responses (Ameloblastoma and Stafne bone cavity) The purpose of asking questions about jawbone lesions in panoramic radiographs without describing or interpreting them is to eliminate the differences in participants’ knowledge, experience, and education levels. This approach standardizes the assessment process for both participants and the chatbot, ensuring that responses are based solely on clinical examination and imaging. Participant selection and study design The study included dentists who had completed training and specialized in two different fields (Group 1: Oral and Maxillofacial Radiologists, Group 2: Oral and Maxillofacial Surgeons), general dentists (Group 3), and the subscription-based ChatGPT-4o (OpenAI, San Francisco, CA), a large language model released in 2024 (Group 4). Participation was entirely voluntary for the individuals involved. Patient confidentiality was ensured in this study through a meticulously applied anonymization process. No personal or identifying information about the participants was included in the data collection forms. A special questionnaire consisting of two sections was prepared for this study. In the first section, participants were asked four questions about their personal information (Age, Gender, Specialty, and Professional Experience). The case data prepared in the second section of the study was used. Participants were asked to indicate their most likely preliminary diagnosis for different jawbone lesions, accompanied by prepared panoramic images and brief clinical history information about the patient. The questionnaire, which included personal information and 30 lesion questions, was prepared using Google Forms (Alphabet, Mountain View, California) (Table 1). The questionnaire was sent to participants using various electronic communication channels (e-mail, WhatsApp, Telegram, etc.) (Fig. 4). Table 1Sample questionnaire form prepared for participants Table 1Sample questionnaire form prepared for participants Sample questionnaire form prepared for participants Fig. 4Study design and methodological flow (flow-chart ) Study design and methodological flow (flow-chart ) Statistical analysis of data IBM SPSS software (version 25) (SPSS Inc., Chicago, IL) was used for data summarization and analysis. First, the number and percentages of participants were tabulated by questionnaire response, and the necessary statistical analyses were performed by a statistician (S.A.K.) specializing in this field. The G-Power analysis 3.1.0 software package was used to determine the sample size for the study. In the analysis, the test power was set at 0.80, α at 0.05, and the effect size at 0.3. The sample size was planned to be 156 volunteers. The normality assumption was evaluated using the Shapiro–Wilk test, which indicated that the data did not follow a normal distribution. The nonparametric Wilcoxon Signed Rank test was used to compare participant and ChatGPT performance obtained from the same cases. The Mann–Whitney U test was applied for pairwise comparisons between independent groups, and the Kruskal–Wallis test was applied for comparisons involving three or more groups. Following Kruskal–Wallis tests that yielded significant results, the Dunn post hoc test and the Bonferroni correction were used. The statistical significance level was set at p < 0.05. Results A total of 161 participants were included in the study, with 32.3% (52) being OMFRs, 33.5% (54) being OMFSs, and 34.2% (55) being general dentists. The distribution of participants according to demographic characteristics is shown in Table 2. Table 2Demographic data distribution of participantsVariablesOMFR % (N)OMFS % (N)General dentists % (N)Total % (N)GenderFemale75 (39)61.1 (33)67.3 (37)67.7 (109)Male25 (13)38.9 (21)32.7 (18)32.3 (52)Age≤ 25 age-3.7 (2)10.9 (6)5 (8)25–45 age86.5 (45)83.3 (45)76.4 (42)82 (132)45 ≥ age13.5 (7)13 (7)12.7 (7)13 (21)Professional experience1–5 years30.8 (16)29.6 (16)56.4 (31)39.1 (63)6–10 years34.6 (18)27.8 (15)27.3 (15)29.8 (48)10 + years34.6 (18)42.6 (23)16.4 (9)31.1 (50)WorkplaceUniversity65.4 (34)59.3 (32)27.3 (15)50.3 (81)ODHC30.8 (16)16.7 (9)41.8 (23)29.8 (48)Private Clinic3.8 (2)24.1 (13)30.9 (17)19.9 (32)Total32.3 (52)33.5 (54)34.2 (55)100 (161)OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center Table 2Demographic data distribution of participantsVariablesOMFR % (N)OMFS % (N)General dentists % (N)Total % (N)GenderFemale75 (39)61.1 (33)67.3 (37)67.7 (109)Male25 (13)38.9 (21)32.7 (18)32.3 (52)Age≤ 25 age-3.7 (2)10.9 (6)5 (8)25–45 age86.5 (45)83.3 (45)76.4 (42)82 (132)45 ≥ age13.5 (7)13 (7)12.7 (7)13 (21)Professional experience1–5 years30.8 (16)29.6 (16)56.4 (31)39.1 (63)6–10 years34.6 (18)27.8 (15)27.3 (15)29.8 (48)10 + years34.6 (18)42.6 (23)16.4 (9)31.1 (50)WorkplaceUniversity65.4 (34)59.3 (32)27.3 (15)50.3 (81)ODHC30.8 (16)16.7 (9)41.8 (23)29.8 (48)Private Clinic3.8 (2)24.1 (13)30.9 (17)19.9 (32)Total32.3 (52)33.5 (54)34.2 (55)100 (161)OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center Demographic data distribution of participants OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center Table 3 compares the total number of correct diagnoses made by participants with ChatGPT’s performance based on the same cases. Comparisons were made using the Wilcoxon Signed Rank test for each demographic subgroup (gender, age, professional experience, and workplace). Table 3Comparison of participant performance by specialty and ChatGPT performance based on demographic variablesGroupsVariablesParticipants Average Correct Diagnosis Count (N)ChatGPT Correct Diagnosis Count (N)p-valueOMFRGenderFemale21 (39)140.0001*Male20.15 (13)140.0005*Age≤ 25 Age---25–45 Age20.8 (45)140.0001*45 ≥ Age20.71 (7)140.0739Professional experience1–5 years20.31 (16)140.0001*6–10 years20.83 (18)140.0006*10+21.18(18)140.0001*WorkplaceUniversity21.44 (34)140.0001*ODHC16 (2)141.0Private Clinic20 (16)140.0002*OMFSGenderFemale17.61 (33)140.0001*Male20.71 (21)140.0001*Age≤ 25 Age17 (2)140.5025–45 Age18.80 (45)140.0001*45 ≥ Age19.43 (7)140.0156*Professional experience1–5 years17.62 (16)140.0038*6–10 years19.07 (15)140.0009*10 + years19.48 (23)140.0001*WorkplaceUniversity18.66 (32)140.0001*ODHC18.33 (9)140.0273*Private Clinic19.54 (13)140.0002*General dentistsGenderFemale11.97 (37)140.0709Male9.17 (18)140.0040*Age≤ 25 Age9.83 (6)140.156225–45 Age11.00 (42)140.0056*45 ≥ Age12.43 (7)140.2969Professional experience1–5 years11.32 (31)140.0305*6–10 years10.53 (15)140.0408*10 + years11 (9)140.1289WorkplaceUniversity10.20 (15)140.0125*ODHC12.43 (23)140.3146Private Clinic9.94 (17)140.0297*OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center*p < 0,05 (Wilcoxon Signed Rank Test) Table 3Comparison of participant performance by specialty and ChatGPT performance based on demographic variablesGroupsVariablesParticipants Average Correct Diagnosis Count (N)ChatGPT Correct Diagnosis Count (N)p-valueOMFRGenderFemale21 (39)140.0001*Male20.15 (13)140.0005*Age≤ 25 Age---25–45 Age20.8 (45)140.0001*45 ≥ Age20.71 (7)140.0739Professional experience1–5 years20.31 (16)140.0001*6–10 years20.83 (18)140.0006*10+21.18(18)140.0001*WorkplaceUniversity21.44 (34)140.0001*ODHC16 (2)141.0Private Clinic20 (16)140.0002*OMFSGenderFemale17.61 (33)140.0001*Male20.71 (21)140.0001*Age≤ 25 Age17 (2)140.5025–45 Age18.80 (45)140.0001*45 ≥ Age19.43 (7)140.0156*Professional experience1–5 years17.62 (16)140.0038*6–10 years19.07 (15)140.0009*10 + years19.48 (23)140.0001*WorkplaceUniversity18.66 (32)140.0001*ODHC18.33 (9)140.0273*Private Clinic19.54 (13)140.0002*General dentistsGenderFemale11.97 (37)140.0709Male9.17 (18)140.0040*Age≤ 25 Age9.83 (6)140.156225–45 Age11.00 (42)140.0056*45 ≥ Age12.43 (7)140.2969Professional experience1–5 years11.32 (31)140.0305*6–10 years10.53 (15)140.0408*10 + years11 (9)140.1289WorkplaceUniversity10.20 (15)140.0125*ODHC12.43 (23)140.3146Private Clinic9.94 (17)140.0297*OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center*p < 0,05 (Wilcoxon Signed Rank Test) Comparison of participant performance by specialty and ChatGPT performance based on demographic variables OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center *p < 0,05 (Wilcoxon Signed Rank Test) Regardless of professional experience in the OMFR group, the average number of correct diagnoses for all dentists (both female and male) and participants aged 25–45 was statistically significantly higher than ChatGPT (p < 0.001). Analyses based on the workplace showed a statistically significant difference in favor of ChatGPT among participants working in university and private clinics (p < 0.001). In contrast, no statistically significant difference was observed in the ODHC group (p = 1.000). In the OMFS group, the average number of correct diagnoses across all participants (both female and male), regardless of workplace or professional experience, was statistically significantly higher than ChatGPT (p < 0.05 and p < 0.001, respectively). No statistically significant difference was found in the ≤ 25 age group (p = 0.5000), whereas participants’ performance in the 25–45 and ≥ 45 age groups was statistically significantly higher than ChatGPT’s (p < 0.001 and p = 0.0156, respectively). Results were more heterogeneous in the general dentists group. In analyses by gender, no statistically significant difference was found between female participants and ChatGPT (p = 0.0709), whereas male participants’ performance was statistically significantly lower than ChatGPT’s (p = 0.004). Participants aged 25–45 performed worse than ChatGPT, and this difference was statistically significant (p = 0.0056). No statistically significant difference in performance was observed in other age groups. In analyses based on years of professional experience, participants with 1–5 years and 6–10 years of experience performed worse than ChatGPT, and this difference was statistically significant (p < 0.05). In contrast, the difference observed among participants with 10 or more years of experience was not statistically significant (p = 0.1289). Comparisons based on the workplace showed that the performance of general dentists working in university and private clinics was statistically significantly lower than ChatGPT (p < 0.05). In contrast, no statistically significant difference was observed in the ODHM group (p = 0.3146). The results of multiple pairwise comparisons of diagnosis accuracy based on gender, age, professional experience, and the workplace are shown in Table 4. Table 4Results of multiple pairwise comparisons based on gender, age, professional experience, and the workplaceVariablesTest Statisticp-valueGenderFemale29340.717MaleAge≤ 25 Age5.7710.05625–45 Age45 ≥ AgeExperience year1–5 years-7.9580.019*6–10 years10 + yearsWorkplaceUniversity10.0590.007*ODHCPrivate ClinicSpecialtyOMFR72.868< 0.001*OMFSGeneral dentistsOMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center*p < 0,05 (Mann-Whitney U; Kruskal-Wallis) Table 4Results of multiple pairwise comparisons based on gender, age, professional experience, and the workplaceVariablesTest Statisticp-valueGenderFemale29340.717MaleAge≤ 25 Age5.7710.05625–45 Age45 ≥ AgeExperience year1–5 years-7.9580.019*6–10 years10 + yearsWorkplaceUniversity10.0590.007*ODHCPrivate ClinicSpecialtyOMFR72.868< 0.001*OMFSGeneral dentistsOMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center*p < 0,05 (Mann-Whitney U; Kruskal-Wallis) Results of multiple pairwise comparisons based on gender, age, professional experience, and the workplace OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center *p < 0,05 (Mann-Whitney U; Kruskal-Wallis) In comparisons made without regard to specialty, no statistically significant difference was found in diagnostic accuracy among participants based on gender and age groups (p > 0.05); Participants with 10 or more years of experience were found to have statistically significantly higher diagnostic accuracy rates compared to participants with 1–5 years of experience (p = 0.016). In workplace-based analyses, the diagnostic accuracy of participants working in university hospitals was found to be statistically significantly higher than that of participants working in private clinics (p = 0.011). In analyses stratified by specialty, the diagnostic accuracy of OMFR and OMFS specialists was statistically significantly higher than that of general dentists (p < 0.001). No statistically significant difference was found between OMFR and OMFS specialist participants (p = 0.113). The diagnostic accuracy of the participant groups (OMFR, OMFS, and general dentists) and ChatGPT was evaluated in Table 5 based on the total number of correct diagnoses across 30 cases. Table 5Comparison of diagnostic accuracy between participants and ChatGPTGroupsOMFROMFSGeneral dentistsChatGPTp-valueOMFR-0.113< 0.001*< 0.001*< 0.001*OMFS0.113-< 0.001*< 0.001*General dentists< 0.001*< 0.001*-0.001*ChatGPT< 0.001*< 0.001*0.001*--OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center*p < 0,05 (Wilcoxon testi; Kruskal-Wallis Table 5Comparison of diagnostic accuracy between participants and ChatGPTGroupsOMFROMFSGeneral dentistsChatGPTp-valueOMFR-0.113< 0.001*< 0.001*< 0.001*OMFS0.113-< 0.001*< 0.001*General dentists< 0.001*< 0.001*-0.001*ChatGPT< 0.001*< 0.001*0.001*--OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center*p < 0,05 (Wilcoxon testi; Kruskal-Wallis Comparison of diagnostic accuracy between participants and ChatGPT OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center *p < 0,05 (Wilcoxon testi; Kruskal-Wallis When comparing the diagnostic accuracy of the three groups of dentists, both the OMFR and OMFS groups showed statistically significantly higher diagnostic accuracy compared to the general dentist group (p < 0.001). While the OMFR and OMFS groups showed statistically significantly higher diagnostic accuracy compared to ChatGPT, ChatGPT was found to be more successful than general dentists (Wilcoxon test, p = 0.001). Table 6 presents the disease detection success (sensitivity), disease exclusion success (specificity), positive predictive value (PPV), negative predictive value (NPV), and overall accuracy values for OMFR, OMFS, general dentists, and ChatGPT for different jawbone lesions. Table 6Evaluation of clinicians and ChatGPT in terms of diagnostic accuracy metricsMetricsDiagnostic accuracy metricsOMFR (%)OMFS (%)General dentists (%)ChatGPT (%)Radicular cystSensitivity85.9084.5751.52100.00Specificity98.5096.8495.3596.30PPV86.4574.8655.1975.00NPV98.4398.2694.65100.00Accuracy97.2495.6290.9796.67Residual cystSensitivity91.0387.6546.670.00Specificity99.8699.5997.98100.00PPV98.6195.9571.960.00NPV99.0198.6494.3090.00Accuracy98.9798.4092.8590.00Dentigerous cystSensitivity60.2672.8450.3066.67Specificity96.5196.9194.5585.19PPV65.7372.3950.6133.33NPV95.6296.9894.4895.83Accuracy92.8894.5190.1283.33Odontogenic keratocystSensitivity54.4951.8517.580.00Specificity95.2396.9895.3596.30PPV55.9265.6229.590.00NPV94.9694.7791.2489.66Accuracy91.1592.4787.5886.67Nasopalatine duct cystSensitivity88.4684.5771.52100.00Specificity99.7299.7398.52100.00PPV97.1897.1684.29100.00NPV98.7398.3196.89100.00Accuracy98.5998.2195.82100.00OdontomaSensitivity76.9266.6744.8533.33Specificity99.7299.3897.85100.00PPV96.7792.3169.81100.00NPV97.4996.4194.1193.10Accuracy97.4496.1192.5593.33AmeloblastomaSensitivity41.6745.6818.7933.33Specificity96.4497.1996.1685.19PPV56.5264.3535.2320.00NPV93.7094.1591.4292.00Accuracy90.9692.0488.4280.00Fibro-osseous lesionsSensitivity57.0524.077.8833.33Specificity98.5098.0198.1896.30PPV80.9157.3532.5050.00NPV95.3892.0790.5692.86Accuracy94.3690.6289.1590.00Simple bone cystSensitivity41.6732.7216.970.00Specificity98.8697.5396.0988.89PPV80.2559.5532.560.00NPV93.8592.8891.2488.89Accuracy93.1491.0588.1880.00Stafne bone cavitySensitivity95.5176.5443.03100.00Specificity99.7299.1898.65100.00PPV97.3991.1878.02100.00NPV99.5097.4493.97100.00Accuracy99.2996.9193.09100.00OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center, PPV Positive Predictive Value, NPV Negative Predictive Value Table 6Evaluation of clinicians and ChatGPT in terms of diagnostic accuracy metricsMetricsDiagnostic accuracy metricsOMFR (%)OMFS (%)General dentists (%)ChatGPT (%)Radicular cystSensitivity85.9084.5751.52100.00Specificity98.5096.8495.3596.30PPV86.4574.8655.1975.00NPV98.4398.2694.65100.00Accuracy97.2495.6290.9796.67Residual cystSensitivity91.0387.6546.670.00Specificity99.8699.5997.98100.00PPV98.6195.9571.960.00NPV99.0198.6494.3090.00Accuracy98.9798.4092.8590.00Dentigerous cystSensitivity60.2672.8450.3066.67Specificity96.5196.9194.5585.19PPV65.7372.3950.6133.33NPV95.6296.9894.4895.83Accuracy92.8894.5190.1283.33Odontogenic keratocystSensitivity54.4951.8517.580.00Specificity95.2396.9895.3596.30PPV55.9265.6229.590.00NPV94.9694.7791.2489.66Accuracy91.1592.4787.5886.67Nasopalatine duct cystSensitivity88.4684.5771.52100.00Specificity99.7299.7398.52100.00PPV97.1897.1684.29100.00NPV98.7398.3196.89100.00Accuracy98.5998.2195.82100.00OdontomaSensitivity76.9266.6744.8533.33Specificity99.7299.3897.85100.00PPV96.7792.3169.81100.00NPV97.4996.4194.1193.10Accuracy97.4496.1192.5593.33AmeloblastomaSensitivity41.6745.6818.7933.33Specificity96.4497.1996.1685.19PPV56.5264.3535.2320.00NPV93.7094.1591.4292.00Accuracy90.9692.0488.4280.00Fibro-osseous lesionsSensitivity57.0524.077.8833.33Specificity98.5098.0198.1896.30PPV80.9157.3532.5050.00NPV95.3892.0790.5692.86Accuracy94.3690.6289.1590.00Simple bone cystSensitivity41.6732.7216.970.00Specificity98.8697.5396.0988.89PPV80.2559.5532.560.00NPV93.8592.8891.2488.89Accuracy93.1491.0588.1880.00Stafne bone cavitySensitivity95.5176.5443.03100.00Specificity99.7299.1898.65100.00PPV97.3991.1878.02100.00NPV99.5097.4493.97100.00Accuracy99.2996.9193.09100.00OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center, PPV Positive Predictive Value, NPV Negative Predictive Value Evaluation of clinicians and ChatGPT in terms of diagnostic accuracy metrics OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center, PPV Positive Predictive Value, NPV Negative Predictive Value When PPV is examined, higher values are generally observed in specialist participant groups (OMFR and OMFS). This situation shows that a specialist participants assessment of “this diagnosis is present” is often correct, meaning that the positive diagnosis is more reliable. The low PPV in some diagnoses among general dentists and ChatGPT, however, suggests that the reliability of positive diagnoses made by these groups may be limited. Regarding NPV, high values were observed for most diagnoses across all groups. This situation indicates that decisions of “this diagnosis is absent” are generally reliable and that false-negative decisions are less common. The diagnostic accuracy metrics for ChatGPT were calculated based on a small sample size (n = 3) in this study. Therefore, the sensitivity, specificity, and accuracy values obtained for ChatGPT are indicative rather than statistically generalized. Overall, it is observed that most expert participants (OMFR and OMFS) achieved significantly higher diagnostic accuracy rates compared to general dentists. This difference is particularly evident in jawbone lesions that are diagnostically more complex and difficult to differentiate, such as ameloblastoma, odontogenic keratocyst, and simple bone cyst, which are recognized in the literature. In jawbone lesions that are more radiologically and clinically characteristic, such as radicular cysts, nasopalatine canal cysts, and Stafne bone cavities, relatively high accuracy rates are observed across all groups. In these diagnoses, the OMFR and OMFS groups showed accuracy rates above 80%, while ChatGPT correctly diagnosed all cases (100%). In contrast, general dentists had lower accuracy rates for the same diagnoses. In jawbone lesions with more difficult differential diagnoses, such as residual cysts and odontogenic keratocysts, specialist participant groups maintain relatively high accuracy rates. In contrast, accuracy rates among general dentists are seen to drop significantly. ChatGPT, however, failed to make the correct diagnosis in both cases (0% accuracy), indicating that these jawbone lesions were missed entirely. In diagnosing dentigerous cysts and odontomas, the OMFR group achieved higher accuracy than the OMFS group, while general dentists lagged behind both specialist groups. ChatGPT’s performance in these diagnoses varied: it achieved limited success in dentigerous cysts but had low accuracy in odontoma cases. In jawbone lesions such as ameloblastoma, fibrous lesions, and simple bone cysts, which are more complex and may exhibit different radiological features, accuracy rates decrease across all groups. However, even in these diagnoses, specialist dentists achieve higher accuracy than general dentists. The fact that ChatGPT’s accuracy rates remain low or limited in this group indicates that its diagnostic reliability is limited in these jawbone lesions. While ChatGPT demonstrated high accuracy for distinctive, characteristic lesions, it did not perform consistently across complex jawbone lesions. The accuracy rates of general dentists, on the other hand, lagged behind those of specialist groups in most diagnoses. The heat map showing diagnosis-based accuracy rates is shown in Fig. 5, and the line graph is shown in Fig. 6 below. Fig. 5Heat map showing diagnostic-based accuracy rates (The color scale represents diagnostic accuracy rates (%). Dark red tones indicate higher accuracy rates. In comparison, light colors and blue tones indicate lower accuracy rates.) Heat map showing diagnostic-based accuracy rates (The color scale represents diagnostic accuracy rates (%). Dark red tones indicate higher accuracy rates. In comparison, light colors and blue tones indicate lower accuracy rates.) Fig. 6Line graph showing diagnosis-based accuracy rates Line graph showing diagnosis-based accuracy rates The heat map shows that OMFR and OMFS specialists achieve higher diagnostic accuracy than general dentists across most diagnostic groups. This difference is particularly evident in diagnostically more complex jawbone lesions such as odontogenic keratocyst, ameloblastoma, and fibro-osseous lesions, where general dentists show lower accuracy rates. In contrast, specialists achieve high accuracy rates for more characteristic jawbone lesions such as radicular cysts, nasopalatine canal cysts, and Stafne bone cavities. ChatGPT’s diagnostic accuracy varies by diagnosis; while it achieves high accuracy on distinctive lesions, it declines significantly on lesions with more difficult differential diagnoses. Table 7 compares the rates of correct and incorrect diagnoses among participant groups and demographic subgroups with ChatGPT performance obtained from the same case set. Table 7Comparison of participants and ChatGPT performance and interpretation of valuesVariablesGroupsCorrect (%)False (%)p-valueInterpretation of analysesLLMChatGPT46.6753.33SpecialtyOMFR67.7132.29< 0.001More successful than ChatGPTOMFS58.9641.040.001More successful than ChatGPTGeneral dentist37.8562.15< 0.001More unsuccessful than ChatGPTGenderFemaleOMFR7030< 0.001More successful than ChatGPTOMFS58.6941.31< 0.001More successful than ChatGPTGeneral dentist37.85600.071No statistically significant difference (similar to ChatGPT)MaleOMFR67.1832.82< 0.001More successful than ChatGPTOMFS69.2130.79< 0.001More successful than ChatGPTGeneral dentist37.8569.260.004More unsuccessful than ChatGPTAge≤ 25 AgeOMFR00-(No Data)OMFS56.6743.33-(Insufficient Data)General dentist37.8567.220.156No statistically significant difference (similar to ChatGPT)25–45 AgeOMFR69.3330.67< 0.001More successful than ChatGPTOMFS62.7437.26< 0.001More successful than ChatGPTGeneral dentist37.8563.250.006More unsuccessful than ChatGPT45 ≥ AgeOMFR69.0530.950.074No statistically significant difference (similar to ChatGPT)OMFS64.7635.240.016More successful than ChatGPTGeneral dentist37.8558.10.297No statistically significant difference (similar to ChatGPT)Professional experience1–5 yearsOMFR67.7132.29< 0.001More successful than ChatGPTOMFS58.9641.040.004More successful than ChatGPTGeneral dentist37.8562.150.029More unsuccessful than ChatGPT6–10 yearsOMFR69.4430.56< 0.001More successful than ChatGPTOMFS63.5636.44< 0.001More successful than ChatGPTGeneral dentists37.8564.890.041More unsuccessful than ChatGPT10 + yearsOMFR70.5629.44< 0.001More successful than ChatGPTOMFS64.9335.07< 0.001More successful than ChatGPTGeneral dentists37.8562.960.098No statistically significant difference (similar to ChatGPT)WorkplaceUniversityOMFR71.4728.53< 0.001More successful than ChatGPTOMFS62.2537.75< 0.001More successful than ChatGPTGeneral dentists37.85660.010More unsuccessful than ChatGPTODHCOMFR53.3346.67-(Insufficient Data)OMFS65.1334.87< 0.001More successful than ChatGPTGeneral dentists37.8566.80.030More unsuccessful than ChatGPTPrivate ClinicOMFR66.6733.33< 0.001More successful than ChatGPTOMFS61.1138.890.039More successful than ChatGPTGeneral dentists37.8558.490.300No statistically significant difference (similar to ChatGPT)OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center*p < 0,05, **p < 0,01, (Wilcoxon Signed Rank) Table 7Comparison of participants and ChatGPT performance and interpretation of valuesVariablesGroupsCorrect (%)False (%)p-valueInterpretation of analysesLLMChatGPT46.6753.33SpecialtyOMFR67.7132.29< 0.001More successful than ChatGPTOMFS58.9641.040.001More successful than ChatGPTGeneral dentist37.8562.15< 0.001More unsuccessful than ChatGPTGenderFemaleOMFR7030< 0.001More successful than ChatGPTOMFS58.6941.31< 0.001More successful than ChatGPTGeneral dentist37.85600.071No statistically significant difference (similar to ChatGPT)MaleOMFR67.1832.82< 0.001More successful than ChatGPTOMFS69.2130.79< 0.001More successful than ChatGPTGeneral dentist37.8569.260.004More unsuccessful than ChatGPTAge≤ 25 AgeOMFR00-(No Data)OMFS56.6743.33-(Insufficient Data)General dentist37.8567.220.156No statistically significant difference (similar to ChatGPT)25–45 AgeOMFR69.3330.67< 0.001More successful than ChatGPTOMFS62.7437.26< 0.001More successful than ChatGPTGeneral dentist37.8563.250.006More unsuccessful than ChatGPT45 ≥ AgeOMFR69.0530.950.074No statistically significant difference (similar to ChatGPT)OMFS64.7635.240.016More successful than ChatGPTGeneral dentist37.8558.10.297No statistically significant difference (similar to ChatGPT)Professional experience1–5 yearsOMFR67.7132.29< 0.001More successful than ChatGPTOMFS58.9641.040.004More successful than ChatGPTGeneral dentist37.8562.150.029More unsuccessful than ChatGPT6–10 yearsOMFR69.4430.56< 0.001More successful than ChatGPTOMFS63.5636.44< 0.001More successful than ChatGPTGeneral dentists37.8564.890.041More unsuccessful than ChatGPT10 + yearsOMFR70.5629.44< 0.001More successful than ChatGPTOMFS64.9335.07< 0.001More successful than ChatGPTGeneral dentists37.8562.960.098No statistically significant difference (similar to ChatGPT)WorkplaceUniversityOMFR71.4728.53< 0.001More successful than ChatGPTOMFS62.2537.75< 0.001More successful than ChatGPTGeneral dentists37.85660.010More unsuccessful than ChatGPTODHCOMFR53.3346.67-(Insufficient Data)OMFS65.1334.87< 0.001More successful than ChatGPTGeneral dentists37.8566.80.030More unsuccessful than ChatGPTPrivate ClinicOMFR66.6733.33< 0.001More successful than ChatGPTOMFS61.1138.890.039More successful than ChatGPTGeneral dentists37.8558.490.300No statistically significant difference (similar to ChatGPT)OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center*p < 0,05, **p < 0,01, (Wilcoxon Signed Rank) Comparison of participants and ChatGPT performance and interpretation of values OMFR Oral and Maxillofacial Radiology, OMFS Oral and Maxillofacial Surgery, ODHC Oral and Dental Health Center *p < 0,05, **p < 0,01, (Wilcoxon Signed Rank) In the overall evaluation, ChatGPT achieved an accuracy of 46.67% and an error rate of 53.33%. Compared by specialty, the OMFR and OMFS groups had statistically significantly higher accuracy rates than ChatGPT (p < 0.001). In general, dentists’ accuracy rates were lower than ChatGPT’s, and they were statistically significantly lower (p < 0.001). In the gender-based evaluation, both female and male participants demonstrated statistically significantly higher success rates compared to ChatGPT (p < 0.001). In the OMFS group, participant performance in both genders was also statistically significantly higher than ChatGPT’s (p < 0.001). In contrast, results differed by gender among general dentists; while no statistically significant difference was found between female participants and ChatGPT, male participants were more successful than ChatGPT (p = 0.004). In analyses by age group, OMFR and OMFS participants aged 25–45 were found to be statistically significantly more successful than ChatGPT (p < 0.001). In general dentists in the same age group, however, a statistically significant difference in favor of ChatGPT was found (p = 0.006). In the 45 years and older group, no statistically significant difference was observed between ChatGPT and OMFR, or between ChatGPT and general dentists, while participants in the OMFS group performed statistically significantly better than ChatGPT (p = 0.016). In the ≤ 25 age group, no statistical interpretation could be made due to insufficient data. In the evaluation based on years of professional experience, statistically significantly higher accuracy rates were obtained for all OMFR and OMFS participants with 1–5 years, 6–10 years, and over 10 years of experience compared to ChatGPT (all p < 0.01). In contrast, ChatGPT was more successful overall among general dentists with 1–5 and 6–10 years of experience (p < 0.05). In contrast, no statistically significant difference was observed between ChatGPT and practitioners with 10 years or more of experience. OMFR and OMFS participants working in university perform statistically significantly better than ChatGPT (p < 0.001). While statistical evaluation could not be performed for OMFR due to insufficient data in the ODHC group, OMFS were found to be statistically significantly more successful than ChatGPT (p < 0.001). Among participants working in private clinics, OMFR and OMFS groups were more successful than ChatGPT, while no significant difference was observed between ChatGPT and general dentists. Discussion The findings of this study indicate that ChatGPT have significant potential as supportive clinical tools in oral and maxillofacial radiology. However, their use as independent diagnostic tools or decision-makers is not appropriate currently. Consistent with recent studies in the literature, ChatGPT’s performance appears to vary significantly depending on case complexity, the level of detail in the clinical/radiological information provided, and the type of pathology. To the best of our knowledge, no study in the literature has directly compared the diagnostic performance of ChatGPT with assessments by specialist dentists, including OMFR, OMFS, and general practitioners [7, 21, 25]. This study employs standard clinical data for ChatGPT, which requires no specialized expertise and is universally accessible. Moreover, it evaluates both dentists and the model using only images accompanied by brief, standardized clinical information, without any prior guidance or commentary. Thus, the impact of personal opinions and expertise-related variables has been minimized in the evaluation process, and a standardized approach has been adopted. In this study, when comparing the diagnostic accuracy of three groups of dentists, both OMFR and OMFS participants demonstrated statistically significantly higher diagnostic accuracy compared to general dentists. While the OMFR and OMFS groups showed statistically significantly higher diagnostic accuracy than ChatGPT, ChatGPT outperformed general dentists. In our study, the overall accuracy rate of ChatGPT was determined to be 46.67%. This result shows both overlap and divergence with similar studies in literature. Tomo et al. reported that ChatGPT-4 achieved a high accuracy rate of 80.18% in the differential diagnosis of oral diseases and significantly outperformed general dentists (24.32%) [1]. On the other hand, Hu et al. found that the agreement of ChatGPT with pathological diagnosis in neoplastic/cystic case diagnoses based on CBCT findings was 38.9%; this value is closer to our findings [19]. In a study by Cuevas-Núñez et al., ChatGPT-4.0 achieved 59.8% accuracy in the histopathological diagnosis of jawbone lesions, demonstrating that this technology has a moderate diagnostic capability [2]. The represents a lower success rate when compared with the findings of the study by Temur et al. which examined complex oral pathology cases; Temur et al., reported that ChatGPT-4o provided 70% accuracy and ChatGPT-4 provided 66% accuracy [7]. This wide distribution in success rates may be due to the difficulty level of the questions and the complexity of the text-based radiological descriptions used, as stated by Hajibagheri et al. [16]. Our statistical analyses revealed that ChatGPT was significantly more successful than general dentists (p < 0.001). This finding suggests that the ability of artificial intelligence to provide instant access to a wide range of literature can compensate to some extent for the lack of experience of general dentists. ChatGPT demonstrated 100% accuracy in cases with characteristic features such as radicular cysts, nasopalatine canal cysts, and Stafne bone cavities. The fact that these lesions have very clearly defined cystic prototypes in the literature aligns with the model’s text-based pattern recognition ability. However, in lesions with more difficult differential diagnosis, such as odontogenic keratocyst and residual cysts, ChatGPT showed 0% success, completely missing these cases. In the study by Kim et al. (2024), the agreement rate of ChatGPT-4 in odontogenic keratocyst and odontogenic tumors remained at a limited level of 41.4% [8]. This indicates that artificial intelligence has not yet reached specialists clinical reasoning ability in “difficult and hard-to-diagnose” lesions [14, 19]. Studies in radiology show that ChatGPT has strengths, particularly in structured reporting, clinical summarization, information extraction, and educational use [9, 19]. However, as emphasized by Silva et al. and Oon et al., the model’s performance remains limited in complex cases that require contextual interpretation of radiographic findings [3, 26]. This situation demonstrates that using ChatGPT in diagnostic processes requiring clinical reasoning, patient-specific context, and multidimensional assessment without expert supervision is risky. When years of professional experience, age, and gender are evaluated together, specialty-based training emerges as the most decisive factor influencing diagnostic accuracy. In our study, both female and male OMFR and OMFS participants performed significantly better than ChatGPT, indicating that diagnostic accuracy is associated more with specialization and experience than with gender. Cuevas-Núñez et al. reported a moderate negative correlation between age and diagnostic scores (r = − 0.33) and identified age as a statistically significant predictor of diagnostic performance. Accordingly, diagnostic accuracy tended to decrease with increasing age. The same study also found that gender had no statistically significant effect on diagnostic scores [2]. Our findings are largely consistent with these results. Among specialists, participants of both genders demonstrated higher diagnostic accuracy than ChatGPT. In contrast, results among general dentists varied by age and gender. While no significant difference was observed between female general practitioners and ChatGPT, ChatGPT outperformed male general practitioners. Additionally, in the 25–45 age group, OMFR and OMFS participants showed significantly better performance than ChatGPT, whereas in the same age group, general practitioners performed significantly worse than ChatGPT. ChatGPT’s performance, when compared to other LLMs in the fields of dentistry and oral pathology, varies depending on the task and model tested. ChatGPT 4o demonstrated significantly higher accuracy than Gemini 2.5 Pro in tasks requiring multimodal data processing and radiological image analysis [22]. In the diagnosis of radiolucent jaw lesions in MCQ (multiple-choice) format, ChatGPT 4o achieved a success rate of 66–82%, while Gemini remained at 57–63%. This is thought to be due to ChatGPT’s training datasets being more exposed to medical and pathological content [22, 27]. However, a different picture emerges in oral pathology diagnoses based on clinical scenarios. DeepSeek-v3 was compared with ChatGPT 4o in 16 different clinical scenarios and was found to be statistically significantly more successful (p = 0.024). The average Likert score for the DeepSeek-v3 model was 4.02, while ChatGPT 40’s score remained at 3.15. Furthermore, DeepSeek-v3 demonstrated significantly superior reference reliability; 80.65% of the references provided by ChatGPT 40 turned out to be false, while this rate was only 16.67% in DeepSeek-v3 [5]. Future studies evaluating the performance of multimodal models with visual data processing capabilities (e.g., ChatGPT-4oV) and the clinical accuracy of alternative models such as DeepSeek in large case sets are of great importance [5, 19]. In summary, while ChatGPT is a strong contender in multimodal tasks and image synthesis, DeepSeek models may surpass ChatGPT in terms of clinical reasoning accuracy and academic integrity (providing true references) [5, 22, 28]. Limitations of the study This study has several limitations. First, the evaluation was conducted using only a single model (ChatGPT-4o). Since LLMs and multimodal models like Gemini or DeepSeek evolve rapidly, our findings are specific to the tested model and should not be generalized to all AI systems. Second, the sample size of 3 cases per lesion category is small for a robust subgroup analysis, and results should be interpreted as indicative. To address this issue, future studies could examine each lesion type separately and use a larger number of questions within each lesion group. Additionally, the exclusion of all jawbone lesions and the single-center design are important limitations. Future multicenter and more comprehensive studies will increase the generalizability of the findings. Furthermore, the study compared AI and clinicians independently; future research should evaluate ‘human-in-the-loop’ scenarios where AI acts as a direct decision support tool for practitioners. Conclusion In conclusion, ChatGPT-4o demonstrated moderate diagnostic accuracy (46.67%) for jawbone lesions, significantly lower than specialist dentists (OMFR and OMFS). While ChatGPT showed high performance in characteristic lesions like Stafne bone cavities, its reliability declined in complex cases requiring difficult differential diagnoses. Although ChatGPT outperformed general dentists in some metrics, it cannot replace specialist evaluation. It may serve as a preliminary supportive tool, but its clinical utility as a decision-support system requires further validation through human-AI collaborative studies. Supplementary Information Supplementary Material 1. Supplementary Material 1.