Performance of large language models in delivering accurate and comprehensible patient information on heart failure and cardiomyopathy

Christoph Reich, Jule Leverenz, Charlotte Brand, Lasse Niemeier, Isabel Branzei, Mustafa Yildirim, Farbod Sedaghat-Hamedani, Ali Amr, Norbert Frey, Benjamin Meder · 2026

Read Performance of large language models in delivering accurate and comprehensible patient information on heart failure and cardiomyopathy by Christoph Reich, Jule Leverenz, Charlotte Brand, Lasse Niemeier, Isabel Branzei, Mustafa Yildirim, Farbod Sedaghat-Hamedani, Ali Amr, Norbert Frey, Benjamin Meder (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13286976.txt.

Description

Christoph Reich and a team of evaluators grade six leading chatbots on answers to patient questions about heart failure and cardiomyopathy. The 2026 study poses fifty expert-written questions, hides which model answered, and collects ratings from cardiologists, medical students, and automated graders across nine quality domains. Patients increasingly ask chatbots about heart disease, so accuracy and readability both matter.

Gemini earned the highest overall marks and was chosen most often by reviewers, with Grok second. Avoiding invented facts scored highest for every model, while brevity scored lowest. Gemini also wrote the longest answers, though at the most readable level. Human experts graded more strictly than the automated judges. Serious errors were rare but not absent.

Catalog record

CREATORChristoph Reich, Jule Leverenz, Charlotte Brand, Lasse Niemeier, Isabel Branzei, Mustafa Yildirim, Farbod Sedaghat-Hamedani, Ali Amr, Norbert Frey, Benjamin Meder
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC13286976

Browse the full library · About the Tower of Babel