Role-based evaluation of artificial intelligence chatbot responses in orthodontic emergency scenarios: accuracy, readability, understandability, and patient-oriented communication
Mustafa Ozdemir, Aysegul Gulec · 2026
Read Role-based evaluation of artificial intelligence chatbot responses in orthodontic emergency scenarios: accuracy, readability, understandability, and patient-oriented communication by Mustafa Ozdemir, Aysegul Gulec (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13466132.txt.
Description
Ozdemir and Gulec tested in 2026 how four leading chatbots answer orthodontic emergencies when addressed as a patient versus as a specialist. Twenty-three standard emergency scenarios went to Gemini 2.5, Copilot, Claude 3 Opus, and ChatGPT-4o in both roles. Expert orthodontists graded accuracy on a three-point scale. Readability and clarity were scored with Turkish formulas, plus a consistency check.
Role changed the answers. Claude and Gemini scored significantly more accurate when addressed as orthodontists. ChatGPT-4o and Claude gave the most correct information when the asker played the patient. Patient-facing answers were significantly easier to understand than specialist-facing ones. ChatGPT-4o and Gemini were the most internally consistent across repeats. Patient answers often took a reassuring tone and stressed home self-care, which could soften a patient's sense of urgency. The authors conclude chatbots may brief patients in emergencies but must never replace professional judgment.
Catalog record
| CREATOR | Mustafa Ozdemir, Aysegul Gulec |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |