Role-based evaluation of artificial intelligence chatbot responses in orthodontic emergency scenarios: accuracy, readability, understandability, and patient-oriented communication

Mustafa Ozdemir, Aysegul Gulec · 2026

Read Role-based evaluation of artificial intelligence chatbot responses in orthodontic emergency scenarios: accuracy, readability, understandability, and patient-oriented communication by Mustafa Ozdemir, Aysegul Gulec (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13466132.txt.

Description

Ozdemir and Gulec tested in 2026 how four leading chatbots answer orthodontic emergencies when addressed as a patient versus as a specialist. Twenty-three standard emergency scenarios went to Gemini 2.5, Copilot, Claude 3 Opus, and ChatGPT-4o in both roles. Expert orthodontists graded accuracy on a three-point scale. Readability and clarity were scored with Turkish formulas, plus a consistency check.

Role changed the answers. Claude and Gemini scored significantly more accurate when addressed as orthodontists. ChatGPT-4o and Claude gave the most correct information when the asker played the patient. Patient-facing answers were significantly easier to understand than specialist-facing ones. ChatGPT-4o and Gemini were the most internally consistent across repeats. Patient answers often took a reassuring tone and stressed home self-care, which could soften a patient's sense of urgency. The authors conclude chatbots may brief patients in emergencies but must never replace professional judgment.

Catalog record

CREATORMustafa Ozdemir, Aysegul Gulec
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC13466132

Browse the full library · About the Tower of Babel