Performance of DeepSeek-R1, ChatGPT (GPT-o3-mini), and Gemini 2.0 Flash on German Medical Multiple-Choice Questions: Comparative Evaluation
Annika Meyer, Yassin Karay, Andrea U Steinbicker, Thomas Streichert, Remco Overbeek, Alicia Stone, Lucia Mosca, Priyanshu Sharma · 2025
Read Performance of DeepSeek-R1, ChatGPT (GPT-o3-mini), and Gemini 2.0 Flash on German Medical Multiple-Choice Questions: Comparative Evaluation by Annika Meyer, Yassin Karay, Andrea U Steinbicker, Thomas Streichert, Remco Overbeek, Alicia Stone, Lucia Mosca, Priyanshu Sharma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-12757712.txt.
Description
The authors test three chatbots on 200 German medical exam questions, asking whether the one that can run offline keeps up with cloud rivals. DeepSeek-R1 matters here because local running protects patient data, a standing worry in medicine. The questions came from the German Progress Test Medicine, and the rivals were ChatGPT on a small recent model and Gemini 2.0 Flash. Accuracy meant the share of questions answered right.
All three cleared the usual 60 percent pass bar with room to spare. DeepSeek scored 96 percent, Gemini 94 percent, and ChatGPT 92.5 percent, and the gaps were too small to call real. One pattern did stand out: longer questions tripped up DeepSeek and ChatGPT, but not Gemini. The authors present the offline model as a credible privacy-friendly option.
Catalog record
| CREATOR | Annika Meyer, Yassin Karay, Andrea U Steinbicker, Thomas Streichert, Remco Overbeek, Alicia Stone, Lucia Mosca, Priyanshu Sharma |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |