Evaluating search-enabled large language model interfaces for mpox public health consultation: a guideline-based comparative study
Qiqi Zheng, Ru Chen, Mingming Cai, Yanyan Chen, Xiuli Lin, Tingting Wang · 2026
Read Evaluating search-enabled large language model interfaces for mpox public health consultation: a guideline-based comparative study by Qiqi Zheng, Ru Chen, Mingming Cai, Yanyan Chen, Xiuli Lin, Tingting Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13553459.txt.
Description
Qiqi Zheng and co-authors put six search-enabled chatbots through a public health exam on mpox. Fifty-two consultation questions, drawn from what the public actually asks, went once to each interface. Five trained raters scored the three hundred twelve first answers against guideline-based references for safety, accuracy, empathy, quality, and readability.
Unsafe answers were uncommon but appeared in every interface, with safe rates between about eighty-seven and ninety-two percent. Beyond safety, the interfaces differed significantly on nearly everything, and which one won depended on the measure. The authors argue that guideline-based testing, source transparency, plainer reading levels, and stronger safety guardrails are prerequisites before the public leans on these tools for outbreak advice.
Catalog record
| CREATOR | Qiqi Zheng, Ru Chen, Mingming Cai, Yanyan Chen, Xiuli Lin, Tingting Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |