Performance of large language model in cross-specialty medical scenarios
Zhen Cui, Wuzheng Liu, Xuan Tian, Conglei You, Xiangyu Meng, Huijuan Zhang, Kangzi Gong, Xu Wang, Jun Wu · 2025
Read Performance of large language model in cross-specialty medical scenarios by Zhen Cui, Wuzheng Liu, Xuan Tian, Conglei You, Xiangyu Meng, Huijuan Zhang, Kangzi Gong, Xu Wang, Jun Wu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-12903388.txt.
Description
Zhen Cui and co-authors compare three large language models on medical cases across many specialties, in a study from 2025. Chatbots are being eyed as aids for doctors, but their skill may vary by field. The team built fifty case stories drawn from published reports. Two doctors graded the models' answers in a blind setup, with a senior doctor settling splits.
One model, GPT-4o, led on diagnosis by a clear margin. Its scores were also the most steady across fields. On treatment advice, it beat the older, smaller model but not the third rival. Treatment scores swung far more than diagnosis scores for all three. The authors see real promise for diagnosis support. They warn that shaky treatment advice blocks clinic use for now.
Catalog record
| CREATOR | Zhen Cui, Wuzheng Liu, Xuan Tian, Conglei You, Xiangyu Meng, Huijuan Zhang, Kangzi Gong, Xu Wang, Jun Wu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |