Evaluating foundation models on official German medical licensing examinations: Implications for high-stakes assessment and AI-assisted medical education

Lasse Cirkel, Johannes Knitza, Volker Schillings, Alexander Oksche, Jan Carl Becker, Sebastian Kuhn · 2026

Read Evaluating foundation models on official German medical licensing examinations: Implications for high-stakes assessment and AI-assisted medical education by Lasse Cirkel, Johannes Knitza, Volker Schillings, Alexander Oksche, Jan Carl Becker, Sebastian Kuhn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13529676.txt.

Description

How well top AI models do on German medical license exams is gauged by Lasse Cirkel and co-authors in work dated 2026. The team fed twenty-four real exams from six years to a set of models. Near seventy-five hundred items came with scores from vast student sittings. Eight models took the full set with images, and thirteen took a shared text-only set.

The best model scored near perfect on both exam halves. Open models ran close, and even small ones beat the mean student. Image items hurt models far more than students. Hard sets for men and models barely overlapped, a gap worth heed.

Catalog record

CREATORLasse Cirkel, Johannes Knitza, Volker Schillings, Alexander Oksche, Jan Carl Becker, Sebastian Kuhn
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70 KB
VERSION1.0
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC13529676

Browse the full library · About the Tower of Babel