AI text detection in dentistry: a comparative analysis across generative models Authors: Jacopo Villa, Daniele Garcovich, Luca Lombardo, Giuseppe Siciliani, Milagros Adobes Martin Published: 2026 ABSTRACT AbstractBackgroundRobust screening for AI-generated scientific text is increasingly required by journals, yet detector performance on full-length manuscripts remains unclear. Six widely used detectors were benchmarked on dentistry manuscripts, and performance was compared across state-of-the-art generators.MethodsA total of 120 manuscripts was assembled in four groups (GPT-4.5, GPT-4o, DeepSeek-R2, and human-written; n = 30 each). Aidetector, GPTZero, Copyleaks, Originality.AI, Turnitin, and DetectingAI produced 0–100% scores. Discrimination was assessed with ROC/AUC and DeLong tests with Bonferroni correction. A pre-specified 60% threshold yielded document-level classifications. Inter-detector agreement was quantified with Cohen’s κ.ResultsFour detectors showed high discrimination; pairwise AUC differences among these were not significant after Bonferroni correction and each outperformed Turnitin and DetectingAI. Using the 60% cut-off, sensitivities and specificities were high, with no false positives observed among the 30 human texts included in this sample. Agreement was almost perfect among the high performers, slight for Turnitin, and none for DetectingAI. Across tools, DeepSeek-R2 texts were the easiest to detect.ConclusionsOn full dentistry manuscripts, four detectors showed high discrimination, whereas Turnitin showed moderate performance and DetectingAI was ineffective. A percentage-based 60% decision threshold provided reproducible, manuscript-level calls.Clinical relevanceThese results may help dental editors and reviewers compare detectors for preliminary screening of AI-generated text under controlled conditions.Supplementary InformationThe online version contains supplementary material available at 10.1186/s41073-026-00228-9.Keywords: AI-generated content, ChatGPT, AI content detection, Dentistry Background Artificial Intelligence (AI) is reshaping almost every aspect of contemporary life, including scientific research, education, and healthcare. With the advent of advanced large language models (LLMs) such as ChatGPT, AI is able to produce texts that are grammatically precise, structurally coherent, and often indistinguishable from those written by a human being [1]. This innovation represents a promising step forward by streamlining scientific writing, supporting non-native speakers, and enhancing access to academic communication [2]. However, this powerful technology also entails significant risks. AI systems can generate entirely fabricated content—data, references, and conclusions included—while maintaining the appearance of scientific rigor [3, 4]. Such outputs challenge the integrity of the scientific record and pose ethical concerns. As generative models become increasingly sophisticated (e.g., GPT-4o, DeepSeek-R2), the need for reliable mechanisms to distinguish between authentic and AI-generated research becomes more urgent [5, 6]. Recent studies have shown that AI can produce fraudulent but plausible scientific articles, capable of passing plagiarism checks and deceiving expert reviewers. For instance, AI-generated abstracts in dentistry have been rated as human-like, while genuine papers in surgical oncology have been mistakenly flagged as machine-written [4, 5]. Cheng et al. also reported that AI-generated abstracts may differ from human-written ones in quality and correctness [7]. These findings highlight the limitations of current detection tools, which frequently yield false positives and negatives depending on the context and discipline. In dentistry, recent bibliometric evidence suggests a growing uptake of ChatGPT-assisted writing in titles and abstracts, although this does not directly assess detector performance on complete manuscripts [8]. In addition to technical challenges, the use of AI raises ethical concerns related to authorship and citation. LLMs may fabricate sources—producing false references or misattributing citations—which can mislead researchers and undermine editorial trust [2, 3]. Consequently, peer review systems and academic publishing standards must adapt to prevent the dissemination of unreliable or deceptive content. Despite the growing integration of generative AI in dental education and publishing, systematic research on AI content detection in this field remains scarce. The few available studies have focused mainly on short texts or on a limited number of detectors and have not evaluated full-length manuscripts generated by state-of-the-art large language models in dentistry [4, 8]. Therefore, this computational diagnostic-accuracy study benchmarked six widely used AI content detectors on a corpus of full-length dental manuscripts produced by three contemporary LLMs and by human authors. The primary objective of this study was to quantify each detector’s sensitivity, specificity, predictive values, and area under the ROC curve (AUC) for the binary classification “AI-generated” versus “human-written” at the manuscript level. Secondary objectives were to compare detector performance across generative models and to examine inter-detector agreement. Material and methods This computational diagnostic-accuracy study benchmarked six AI content detectors on a series of dental manuscripts, evaluating their ability to distinguish human-written from AI-generated scientific texts. Reporting was aligned, where applicable, with items from the STROBE statement for observational studies [9]. Two categories of textual sources—AI-generated texts and human-written texts—were included in this analysis. In total, four groups of texts were analysed—GPT-4.5, GPT-4o, DeepSeek-R2, and human-written—30 per group (N = 120). The required sample size was estimated a priori with R (v4.3; R Foundation for Statistical Computing, Vienna, Austria) using the pROC package and Hanley–McNeil analytic formulas, targeting AUC precision (AUC = 0.90; 95% CI half-width ≤ 0.08) under balanced classes. The calculation yielded 28 AI and 28 human texts. This size also provides 80% power (α = 0.05) to detect AUC differences of 0.12–0.15 in paired comparisons. To allow for any potential exclusions, the target was rounded up to 30 texts per class (AI and human). AI-generated texts A set of scientific-style documents was created by one trained operator using three advanced generative language models—GPT-4.5 (OpenAI, San Francisco, CA, USA), GPT-4o (OpenAI, San Francisco, CA, USA), and DeepSeek-R2 (DeepSeek, Beijing, China)—under the supervision of a second experienced reviewer. Following a standardised methodology inspired by Flitcroft et al., who asked ChatGPT-3.5 to draft 3,000-word IMRaD-style manuscripts from a single title, the approach was adapted and extended to anchor the generated content more closely to real dental evidence [5]. For each AI-generated manuscript, the operator first identified at least four peer-reviewed scientific articles from PubMed in one of six dental specialties (orthodontics, periodontology, endodontics, prosthodontics, conservative dentistry, and paediatric dentistry) and uploaded the full-text PDFs to the LLM interface as a reference. For each topic, the same set of at least four full-text PDF articles was uploaded to all three LLMs (GPT-4.5, GPT-4o, and DeepSeek-R2), so that the source material remained constant across platforms for that manuscript. Different manuscript topics were based on different sets of reference PDFs. Using these attached PDF articles as the only source material, the models were instructed to simulate knowledge-based scientific writing: they were asked to read the uploaded articles and then produce a realistic full-length research article including the standard sections of a manuscript (Abstract, Introduction, Methods, Results, Discussion, and References), without tables or figures. The exact wording of the manuscript title was left to the LLM, within the boundaries of the topic defined by the attached PDFs. Because preliminary testing showed that long single-prompt generations often led to loss of coherence, each manuscript was generated using a standardised section-by-section prompting workflow, starting with the Title and Abstract and then proceeding sequentially through the Introduction, Methods, Results, Discussion, and References. For every section and model, the operator used identical instructions, changing only the set of attached PDFs. The final full-length manuscript (≥ 3,000 words) for each topic and model was assembled by concatenating the generated sections and performing only minimal formatting edits (e.g. removal of obvious repetitions and typographical errors), without substantive human rewriting. A third independent reviewer, blinded to the generation process, assessed each manuscript’s plausibility as human-written and checked for obvious internal inconsistencies; only texts judged to be realistic and credible were included in the final dataset. These AI-generated manuscripts were then saved as pdf and uploaded in full to each AI content detector using the standard upload function. The exact prompt templates used for each section are reported in Additional file 1. Human-written texts Authentic scientific articles authored by dental professionals and published before 2020 were used as the human-written comparators. The pre-2020 cut-off was chosen to minimise the risk that the manuscripts might themselves contain AI-generated content, as GPT-2 — the first widely accessible large language model able to produce coherent text — was released in 2019. A convenience sample of full-length research articles was drawn from peer-reviewed dental journals indexed in major bibliographic databases, ensuring coverage of the same six specialties represented in the AI-generated corpus (orthodontics, periodontology, endodontics, prosthodontics, conservative dentistry, and paediatric dentistry). Only complete research articles of typical length for a dental journal were selected; the scientific content of these articles was not truncated or edited. For the human-written condition, the original publisher PDF files of the selected articles were uploaded in their entirety to each AI detector, using the standard upload function provided by the vendor. Thus, the detectors received the full article as it appears in the journal (title page, abstract, main text, references, and any tables and figure legends contained in the PDF). AI detection tools Six publicly available AI content detection systems were selected based on relevance, accessibility, and usage in recent literature: a) Aidetector (https://aidetector.com/); b) GPTZero (https://gptzero.me/); c) Copyleaks (https://copyleaks.com/ai-content-detector); d) Originality.AI (https://originality.ai/); e) Turnitin (https://www.turnitin.com); f) DetectingAI (https://www.detecting-ai.com/). Each tool was used to assess all 120 texts. Detection outputs returned a document-level percentage score on a 0–100 scale. According to vendor documentation and prior benchmarking work, these scores are intended to quantify either the estimated probability that a document is AI-generated or the estimated proportion of AI-written content within the document, although the exact semantics and calibration differ across tools. For analytical purposes, we treated all scores as continuous “AI-likelihood” percentages and used the raw values without further transformation. For binary classification, we applied a pre-specified 60% cut-off (≥ 60% = AI-generated; < 60% = human), following Elkhatat et al. [10], who considered scores ≥ 60% as indicative of AI authorship. Tool-specific vendor thresholds were either not available or not directly comparable across all six systems; using a single percentage-based cut-off therefore facilitated cross-detector comparison. Statistical analysis Overall discrimination was summarized using receiver operating characteristic (ROC) curves and the area under the curve (AUC) with 95% confidence intervals, computed from the continuous detector scores across all observed thresholds. AUCs were compared between detectors using DeLong’s test with Bonferroni correction for the 15 pairwise contrasts (two-sided α ≈ 0.003). The binary classification “AI” vs “human” used a pre-specified 60% percentage threshold only for threshold-dependent metrics (Sensitivity, Specificity, PPV, NPV, Correct, and Cohen’s κ), to facilitate cross-detector comparison. Sensitivity, Specificity, PPV, NPV, and Correct (overall accuracy) were reported. Model-level categorical contrasts were tested with Fisher’s exact test (two-sided). Agreement between detectors for the binary classification was quantified using pairwise Cohen’s κ (60% threshold). Score distributions were visualised in Python 3.9 using Matplotlib and Seaborn. All statistical analyses were performed in R (v4.3; R Foundation for Statistical Computing, Vienna, Austria). Results AUCs indicated excellent performance for Aidetector (AUC = 1.00), GPTZero (0.99), Copyleaks (0.98), and Originality.AI (0.98), moderate performance for Turnitin (0.63), and null discrimination for DetectingAI (0.50), with corresponding 95% confidence intervals reported in Table 1. As reported in Fig. 1, the curves for Aidetector, GPTZero, Copyleaks, and Originality.AI cluster near the upper-left corner, consistent with near-perfect discrimination. Turnitin’s curve lies clearly below this group, in keeping with its moderate AUC. DetectingAI overlaps the no-discrimination diagonal, visually confirming its inability to separate AI from human texts. Taken together, Table 1 and Fig. 1 delineate two performance blocks (high-performing: Aidetector, GPTZero, Copyleaks, Originality.AI; low-performing: Turnitin, DetectingAI). Using the pre-specified 60% threshold, threshold-based indicators further corroborate these patterns (Table 1): Aidetector achieved perfect classification (Sensitivity, Specificity, PPV, NPV, and Correct all = 1.000), GPTZero was near-perfect with a slightly higher NPV than Copyleaks and Originality.AI, which showed identical summaries (Sensitivity 0.967, Specificity 1.000, PPV 1.000, NPV 0.909, Correct 0.98), whereas Turnitin combined high Specificity (1.000) with low Sensitivity (0.267) and low NPV (0.312), indicating limited discriminative ability and a tendency to classify many AI-generated texts as human, while DetectingAI failed to distinguish AI-generated from human-written texts at the prespecified threshold (Sensitivity 0.000, NPV 0.250, Correct 0.25). Table 1Diagnostic performance of six AI content detection tools in classifying 120 scientific texts (90 AI-generated and 30 human-written), using a 60% percentage threshold to classify a text as AI-generated. Sensitivity = True Positive Rate; Specificity = True Negative Rate; PPV = Positive Predictive Value; NPV = Negative Predictive Value; Correct = Total rate of correct predictions; AUC = area under the ROC curve; p-value for testing AUC = 0.50; N/A indicates that PPV is not computable because the detector produced no positive predictions (0/0). Sensitivity and specificity 95% confidence intervals were calculated using the exact Clopper–Pearson methodDetectorNSensitivity (95% CI)Specificity (95% CI)PPVNPVCorrectAUC95%CI AUCpAidetector1201.00(0.96–1.00)1.00(0.88–1.00)1.001.001.001.001.00–1.00< 0.001GPTZero1200.99(0.94–1.00)1.00(0.88–1.00)1.000.970.990.990.98–1.00< 0.001Copyleaks1200.97(0.91–0.99)1.00(0.88–1.00)1.000.910.980.980.96–1.00< 0.001Originality.AI1200.97(0.91–0.99)1.00(0.88–1.00)1.000.910.980.980.96–1.00< 0.001Turnitin1200.27(0.18–0.37)1.00(0.88–1.00)1.000.310.450.630.53–0.740.029DetectingAI1200.00(0.00–0.04)1.00(0.88–1.00)N/A0.250.250.500.38–0.621.00Bold p-values indicate statistical significance (p 0.05) Diagnostic performance of six AI content detection tools in classifying 120 scientific texts (90 AI-generated and 30 human-written), using a 60% percentage threshold to classify a text as AI-generated. Sensitivity = True Positive Rate; Specificity = True Negative Rate; PPV = Positive Predictive Value; NPV = Negative Predictive Value; Correct = Total rate of correct predictions; AUC = area under the ROC curve; p-value for testing AUC = 0.50; N/A indicates that PPV is not computable because the detector produced no positive predictions (0/0). Sensitivity and specificity 95% confidence intervals were calculated using the exact Clopper–Pearson method Bold p-values indicate statistical significance (p 0.05) Receiver operating characteristic (ROC) curves for six AI content detectors distinguishing AI-generated from human-written texts (N = 120; 90 AI, 30 human). Sensitivity is plotted against 1 − specificity. Curves were derived from the continuous detector scores across all observed thresholds and are therefore threshold-independent. The diagonal line denotes no discrimination After Bonferroni correction, no significant differences in AUC were found among Aidetector, Copyleaks, GPTZero, and Originality.AI, and no significant difference was found between Turnitin and DetectingAI. In contrast, each detector in the high-performing group showed significantly higher AUC than both Turnitin and DetectingAI. All pairwise DeLong comparisons are shown in Table 2. Figure 2 illustrates the score distributions assigned by each AI content detector to both AI-generated and human-written texts. Across most detectors, a clear bimodal distribution emerged, with AI texts receiving high scores and human texts clustering toward the lower end of the scale. Aidetector and Copyleaks both showed excellent separation: Aidetector assigned scores near 1.0 to AI-generated texts and below 0.4 to human texts, while Copyleaks assigned scores close to 1.0 to AI-generated texts and scores at or near 0.0 to human-written articles. Similarly, GPTZero effectively distinguished the two classes, assigning scores close to 1.0 for AI texts and low scores for human-written texts, indicating robust discriminative ability. Originality.AI followed the same pattern, producing a strict dichotomy between the two categories. Table 2Bonferroni-adjusted p-values from DeLong’s test comparing AUCs (α_adj = 0.003)AidetectorGPTZeroCopyleaksOriginality.AITurnitinDetectingAIAidetector—1.0001.0001.000< 0.001< 0.001GPTZero—1.0001.000< 0.001< 0.001Copyleaks—1.000< 0.001< 0.001Originality.AI—< 0.001< 0.001Turnitin—1.000DetectingAI—*Bonferroni-adjusted p ≤ 0.003; significance threshold α_adj = 0.003 (15 pairwise DeLong’s test) Bonferroni-adjusted p-values from DeLong’s test comparing AUCs (α_adj = 0.003) *Bonferroni-adjusted p ≤ 0.003; significance threshold α_adj = 0.003 (15 pairwise DeLong’s test) Score distributions of AI content detectors for AI-generated and human-written texts. Each subplot (a–f) illustrates the distribution of detection scores produced by a different AI content detector: a Aidetector, b GPTZero, c Copyleaks, d Originality.AI, e Turnitin, and f DetectingAI. Histograms represent the frequency of scores for AI-generated texts (blue) and human-written texts (red), with kernel density estimation (KDE) curves overlaid to visualize the score concentration and distribution trend In contrast, DetectingAI demonstrated only partial separation, with some overlap between human and AI distributions; AI texts tended to cluster around 0.4 and human ones near 0.1, increasing the risk of false negatives. Turnitin showed the weakest performance overall. While it consistently assigned a score of 0.0 to human texts, its scores for AI-generated texts were widely dispersed across the entire scale, resulting in considerable overlap between groups. These patterns visually reinforce the sensitivity, specificity, and predictive values previously reported, highlighting substantial variability in the detection reliability of these tools. Figure 3 highlights that all detectors classify human texts correctly, assigning low scores that remain well below the 60% threshold with narrow dispersion, yet large performance gaps emerge for AI-generated texts across generator models: the high-performing detectors maintain median scores well above 60% with tight IQRs for all three models, signalling consistent, confident detection irrespective of the generator, whereas Turnitin shows model-dependent failures (wider IQRs and medians near or below 60%, particularly for DeepSeek-R2), and DetectingAI remains near baseline across models with medians below threshold. Finally, Agreement analyses (Table 3) revealed almost-perfect κ among Aidetector, Copyleaks, and GPTZero (≈0.91–0.98). Turnitin showed only slight agreement with the high-performing detectors (≈0.13–0.17), whereas DetectingAI showed no agreement (κ = 0.000). These findings coherently mirror the weaker discrimination seen in their AUCs, thresholds, and score distributions across Figs. 1, 2 and 3. As highlighted in Table 4, Aidetector achieved 100% sensitivity across GPT-4.5, GPT-4o, and DeepSeek-R2; GPTZero was 96.7% for GPT-4.5 and 100% for the others; Copyleaks and Originality.AI dipped to 90% for GPT-4o but were otherwise 100%, whereas Turnitin varied widely (66.7% for GPT-4.5; 10% for GPT-4o; 3.3% for DeepSeek-R2). Fisher’s exact tests were non-significant for the high-performing detectors and significant only for Turnitin (DeepSeek-R2 vs GPT-4.5 p < 0.001; GPT-4.5 vs GPT-4o p < 0.001). Box plots of detector scores (%) by text group (Human, DeepSeek-R2, GPT-4o, GPT-4.5; N = 120). Each colored box corresponds to one detector (Aidetector, GPTZero, Copyleaks, Originality.AI, Turnitin, DetectingAI). The dashed horizontal line marks the 60% percentage threshold used for binary classification. Boxes show the interquartile range with median line; whiskers extend to 1.5 × IQR; points are outliers Table 3Pairwise agreement (Cohen’s κ) between the six AI detectors for 120 articles, using a 60% percentage thresholdAidetectorGPTZeroCopyleaksOriginality.AITurnitinDetectingAIAidetector0.9780.9350.9350.1540.000GPTZero0.9780.9150.9150.1340.000Copyleaks0.9350.9150.9160.1730.000Originality.AI0.9350.9150.9160.1730.000Turnitin0.1540.1340.1730.1730.000DetectingAI0.0000.0000.0000.0000.000 Pairwise agreement (Cohen’s κ) between the six AI detectors for 120 articles, using a 60% percentage threshold Table 4Sensitivity rates across detectors and generative models (GPT-4.5, GPT-4o, DeepSeek-R2). Fisher’s exact test is reported for DeepSeek-R2 vs GPT-4.5 (main column). Additional pairwise tests (DeepSeek-R2 vs GPT-4o and GPT-4.5 vs GPT-4o) revealed no further significant differences for any detector except Turnitin, which also differed between GPT-4.5 and GPT-4o (p < 0.001). Two-tailed p-values < 0.05 are considered statistically significantDetectorModelNTrue PositiveSensitivity (%)Fisher p-value (DeepSeek-R2 vs GPT-4.5)AidetectorGPT-4.530301001GPT-4o3030100DeepSeek-R23030100GPTZeroGPT-4.5302996.71GPT-4o3030100DeepSeek-R23030100CopyleaksGPT-4.530301001GPT-4o302790DeepSeek-R23030100Originality.AIGPT-4.530301001GPT-4o302790DeepSeek-R23030100TurnitinGPT-4.5302066.7< 0.001*GPT-4o30310DeepSeek-R23013.3DetectingAIGPT-4.530001GPT-4o3000DeepSeek-R23000An asterisk (*) indicates statistical significance (p 0.05) Sensitivity rates across detectors and generative models (GPT-4.5, GPT-4o, DeepSeek-R2). Fisher’s exact test is reported for DeepSeek-R2 vs GPT-4.5 (main column). Additional pairwise tests (DeepSeek-R2 vs GPT-4o and GPT-4.5 vs GPT-4o) revealed no further significant differences for any detector except Turnitin, which also differed between GPT-4.5 and GPT-4o (p < 0.001). Two-tailed p-values < 0.05 are considered statistically significant An asterisk (*) indicates statistical significance (p 0.05) Discussion This study provides additional evidence under controlled conditions on the performance of publicly available AI content detectors when applied to full-length scientific manuscripts in dentistry. Under the controlled conditions of this study, four detectors—Aidetector, GPTZero, Copyleaks, and Originality.AI—demonstrated high discrimination, with near-perfect AUCs and high sensitivity and specificity in distinguishing human- from AI-generated texts, whereas Turnitin displayed only moderate discrimination and DetectingAI failed to separate AI-generated from human texts. These findings align with previous work in surgical oncology and orthopaedic subspecialties, where similar tools—particularly GPTZero and Originality.AI—demonstrated robust performance in identifying synthetic text with minimal false positives [5, 11]. However, discrepancies across tools were also apparent: Turnitin and DetectingAI performed poorly in detecting AI-generated content in this dataset, replicating findings from other studies in oncology and general medicine where these tools failed to identify synthetic abstracts with sufficient reliability [12]. This variability highlights a broader concern: detector performance is not uniform and appears to depend on both the source material and the generative model used. A distinctive feature of this study was the inclusion of the DeepSeek-R2 model, which—to our knowledge—had not been examined in prior dental literature. Across four of six detectors (Aidetector, GPTZero, Copyleaks, Originality.AI) DeepSeek-R2 texts were at least as likely to be flagged as AI-generated as those generated by GPT-4.5 or GPT-4o, corresponding to lower false-negative rates, whereas Turnitin showed the opposite pattern and DetectingAI remained non-discriminative. These findings illustrate that detector behaviour can vary substantially across contemporary LLMs. Importantly, the current analysis also differs from earlier dental work by applying multiple AI content detectors to full scientific manuscripts rather than isolated abstracts or introductions. By examining entire papers, the study provides a manuscript-level testing environment that more closely approximates editorial screening than abstract-only evaluation. Dentistry was selected as a coherent biomedical corpus and a relevant editorial context, not because dental writing is assumed to constitute a distinct linguistic system. Accordingly, the present findings should be interpreted as a domain-specific benchmark on full-length manuscripts from one biomedical specialty, rather than as evidence that dentistry requires a unique detection framework or that detector performance is identical across all scientific fields. Nevertheless, the findings should be interpreted as applying to full-length, minimally edited, purely AI-generated manuscripts under controlled conditions, rather than to hybrid or heavily revised human–AI texts. Recent dentistry studies limited to abstracts report that GPTZero is the only detector with consistently strong performance, ~ 93% accuracy on 160 ChatGPT-3.5 vs. human abstracts, while other tools and human reviewers show variable accuracy [13]. In the same vein, Alencar-Palha et al. found frequent human misclassification and a mean “Human Score” of 16.9% for AI-generated abstracts from an online detector [4]. Outside dentistry, Barrot and Aranda evaluated ZeroGPT and SciSpace on 450 student-produced, AI-edited, and AI-generated essays and reported that ZeroGPT achieved high accuracy and reliability for purely human and purely AI texts, whereas both tools performed markedly worse on AI-edited essays [14]. This pattern highlights the difficulty of detecting hybrid authorship, where human writing is subtly refined by generative AI, and reinforces the concern that current detectors may be less dependable in mixed human–AI scenarios than in purely synthetic manuscripts. More broadly, Gao et al. reported that AI-generated scientific abstracts could often be detected by an AI output detector (AUROC 0.94), while blinded human reviewers identified 68% of generated abstracts but also misclassified 14% of original abstracts as AI-generated [15]. Extending beyond abstracts, the current study evaluates full-length manuscripts with six detectors across three LLMs (including DeepSeek-R2) and reports ROC/AUC with DeLong comparisons, thereby identifying stable top performers (Aidetector, GPTZero, Copyleaks, Originality.AI) under manuscript-level conditions. Other contributions, such as D’Aniello [16], have raised alarms about the overuse of AI in publishing and the risks of information overload, but without examining how well AI-generated texts can be identified. Thus, the present study adds empirical, domain-specific evidence to support the evaluation of AI-generated academic content. Although the overall aim was not to criticize or dismiss the use of AI in scientific writing, the findings raise important questions about the current capabilities and limitations of detection tools. We acknowledge that AI can be a powerful resource for researchers. Tools such as ChatGPT and DeepSeek-R2 can support the drafting process, especially for early-career academics and non-native English speakers, by improving grammar, structure, and clarity [4, 17]. These benefits should not be ignored. However, the risks associated with unregulated or undisclosed use of generative AI are equally important. Several authors have warned about the possibility of hallucinated references, fabricated data, and misleading arguments in AI-generated texts [16, 18]. When such texts are introduced into scientific communication without proper disclosure, they undermine the transparency and trust that underpin scholarly discourse. The notion of “model collapse,” described by Gray [19], further adds to the urgency. As more AI-generated text enters the training datasets of future models, the risk increases that those models will become less accurate, less original, and more error-prone. At the same time, AI detection tools themselves face limitations. The study confirms that even high-performing detectors are not infallible. Although we observed no false positives among the 30 human texts in the current dataset, this finding should be interpreted cautiously given the modest size of the human comparator sample. Ethically, such false-positive classifications may lead to unwarranted suspicion or accusations if detector outputs are treated as definitive proof rather than as preliminary screening signals. Prior studies have shown that off-the-shelf AI detectors may misclassify human-written academic prose as AI-generated and are vulnerable to simple paraphrasing techniques that can substantially reduce detection accuracy [5, 11, 12, 20]. Recent review-level evidence outside dentistry has likewise concluded that currently available AI-text detectors show inconsistent performance, limited reliability, and important fairness concerns, including false positives, disproportionate risks for non-native English writers, and vulnerability to paraphrasing [21–23]. These limitations suggest that current tools cannot yet be relied upon as definitive arbiters of authorship. Accordingly, current editorial guidance increasingly favors transparency- and disclosure-based policies over reliance on technical detection alone. WAME (World Association of Medical Editors) recommends that authors be transparent when chatbots are used and specify the extent and type of use, while the ICMJE (International Committee of Medical Journal Editors) states that authors, reviewers, and editors should disclose which AI tool was used and for what purpose, and that journals should maintain explicit policies governing AI use [24, 25]. In this context, detector outputs are better viewed as preliminary screening signals than as definitive evidence of authorship. Accordingly, the main intended users of this information are editors, reviewers, and journal research-integrity teams seeking comparative evidence to guide preliminary manuscript screening. These considerations support the need for clear institutional and journal policies on the transparent use of AI in scientific writing [26, 27]. Several limitations of the present study should also be acknowledged. Although the sample size was carefully balanced across models and authorship types, it remains relatively modest. The analysis was restricted to English-language manuscripts in the dental field, and the conclusions may not fully generalize to other specialties. In addition, vendors differ in how their percentage scores are defined and calibrated, and a single 60% decision threshold was imposed across tools based on previous work [8]. While this facilitates cross-detector comparison, data-driven, tool-specific cut-offs (e.g., using Youden’s index) might yield slightly different sensitivities and specificities. A further limitation is that the AI-generated corpus in the current study consisted of full-length manuscripts that were only minimally edited and did not include hybrid human–AI writing, paraphrased AI outputs, or heavily revised AI-assisted drafts. A further practical limitation is that the study does not reproduce the most common real-world editorial use-case, namely mixed human–AI, translated, or substantially post-edited text, in which detector performance is likely to be reduced. This is also consistent with broader evidence showing that currently available AI-text detectors may perform variably and that their performance worsens substantially after manual editing, paraphrasing, or machine translation [20]. In addition, the reference-generation prompt explicitly allowed plausible but not necessarily fully verifiable citations; although this reflects a realistic feature of minimally edited LLM outputs, it may have contributed to detector discrimination and should therefore be considered when interpreting the results. Finally, because AI detectors and LLMs evolve rapidly, the findings should be interpreted as a time-specific performance snapshot based on the versions available during data collection. Future studies should explore whether similar detection patterns are observed in other medical disciplines, across languages, and in hybrid texts combining human and AI input, ideally using longitudinal designs that track detector performance over time. Further exploration of prompt design, editorial revision, and human–AI collaboration is also warranted. In conclusion, this study adds domain-specific evidence on the diagnostic performance of AI content detectors applied to scientific writing in dentistry. To our knowledge, it is the first to evaluate full-length manuscripts generated by multiple LLMs, including DeepSeek-R2, using a diverse set of detectors. The results show substantial variability in detection outcomes, influenced not only by the tool used but also by the language model and possibly the subject matter. These findings underscore the importance of continued evaluation, the need for transparency, and the development of more robust and context-sensitive detection systems to safeguard the integrity of scientific publishing in the age of AI. Conclusions Under the controlled conditions of this study, detector performance on full-length dentistry manuscripts showed marked differences across tools. A group comprising Aidetector, GPTZero, Copyleaks, and Originality.AI achieved high and statistically indistinguishable discrimination on full-length, minimally edited AI-generated dentistry manuscripts, whereas Turnitin showed only moderate discrimination and DetectingAI failed to separate AI-generated from human text. Using the prespecified decision threshold, no false positives were observed among the human manuscripts included in this sample. Agreement was very high among the high-performing detectors, limited for Turnitin, and absent for DetectingAI. Among the generators assessed, DeepSeek-R2 texts were the easiest for detectors to identify. These findings should be interpreted within the controlled design of the study and should not be considered definitive evidence of detector performance in hybrid human–AI or heavily edited real-world submissions. Supplementary Information Additional file 1. Authors’ contributions D.G. and M.A.M. conceived the study and defined the research question. D.G., J.V., L.L., and M.A.M. designed the study protocol, detector evaluation workflow, and analytic plan. J.V. and D.G. curated the corpus, ran the detector comparisons, and extracted the outputs. D.G. performed the statistical analyses and prepared figures and tables; M.A.M. verified the analyses. D.G. drafted the initial manuscript. J.V., L.L., G.S., and M.A.M. contributed to the interpretation of results and critically revised the manuscript for important intellectual content. G.S. and L.L. provided senior oversight and supervision. All authors (J.V., D.G., L.L., G.S., M.A.M.) approved the final version of the manuscript. Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. Data availability All data, prompts, detector outputs, analysis code, and figure files are openly available on Mendeley Data at DOI: https://doi.org/10.17632/yx6pvbdpn6.1, under a CC BY 4.0 license. Declarations Ethics approval and consent to participate This study analyzed AI-generated and publicly available published texts only; no human participants or animals were involved. Ethics approval and informed consent were not required. Consent for publication Not applicable. Competing interests The authors declare no competing interests. Footnotes Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Supplementary Materials Additional file 1. Data Availability Statement All data, prompts, detector outputs, analysis code, and figure files are openly available on Mendeley Data at DOI: https://doi.org/10.17632/yx6pvbdpn6.1, under a CC BY 4.0 license.