Assessing the accuracy and consistency of answers by ChatGPT to questions regarding carbon monoxide poisoning
Jun Qiu, Youlian Zhou · 2024
Read Assessing the accuracy and consistency of answers by ChatGPT to questions regarding carbon monoxide poisoning by Jun Qiu, Youlian Zhou (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11578497.txt.
Description
Jun Qiu and Youlian Zhou grade ChatGPT's answers on carbon monoxide poisoning, in a 2024 study. Patients and students may ask chatbots medical questions, so the answers must be right and steady. The team drew 44 single-choice exam questions on the topic from a study app.
Each question went to the chatbot ten times in Chinese and ten times in English. About half the questions drew highly steady answers across repeats. About a third drew answers so varied they counted as unacceptably blurred. Accuracy was measured against the app's key as well. The mixed report warns that a fluent chatbot can still wobble on core medical facts, in either language.
Catalog record
| CREATOR | Jun Qiu, Youlian Zhou |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |