Performance of ChatGPT and Bard on the medical licensing examinations varies across different cultures: a comparison study
Yikai Chen, Xiujie Huang, Fangjie Yang, Haiming Lin, Haoyu Lin, Zhuoqun Zheng, Qifeng Liang, Jinhai Zhang, Xinxin Li · 2024
Read Performance of ChatGPT and Bard on the medical licensing examinations varies across different cultures: a comparison study by Yikai Chen, Xiujie Huang, Fangjie Yang, Haiming Lin, Haoyu Lin, Zhuoqun Zheng, Qifeng Liang, Jinhai Zhang, Xinxin Li (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11590336.txt.
Description
Chen and co-authors sit four large language models down to four national doctor licensing exams and watch scores shift with culture and language. The June 2023 test covers the United States, British, Hong Kong, and mainland Chinese exams, all multiple choice. Two GPT versions, a newer GPT-4o, and Google's Bard take each paper.
GPT-4o leads overall, topping ninety percent on the American and British papers and the Hong Kong paper, and near eighty-five percent on the Chinese exam. The older GPT-3.5 and Bard trail far behind, sometimes barely passing sixty percent. Every model sagged on the Chinese exam relative to the others. The 2024 authors say performance varies across cultures and that the Chinese exam needs work.
Catalog record
| CREATOR | Yikai Chen, Xiujie Huang, Fangjie Yang, Haiming Lin, Haoyu Lin, Zhuoqun Zheng, Qifeng Liang, Jinhai Zhang, Xinxin Li |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |