Performance of large language models in delivering accurate and comprehensible patient information on heart failure and cardiomyopathy
Christoph Reich, Jule Leverenz, Charlotte Brand, Lasse Niemeier, Isabel Branzei, Mustafa Yildirim, Farbod Sedaghat-Hamedani, Ali Amr, Norbert Frey, Benjamin Meder · 2026
Read Performance of large language models in delivering accurate and comprehensible patient information on heart failure and cardiomyopathy by Christoph Reich, Jule Leverenz, Charlotte Brand, Lasse Niemeier, Isabel Branzei, Mustafa Yildirim, Farbod Sedaghat-Hamedani, Ali Amr, Norbert Frey, Benjamin Meder (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13286976.txt.
Description
Christoph Reich and a team of evaluators grade six leading chatbots on answers to patient questions about heart failure and cardiomyopathy. The 2026 study poses fifty expert-written questions, hides which model answered, and collects ratings from cardiologists, medical students, and automated graders across nine quality domains. Patients increasingly ask chatbots about heart disease, so accuracy and readability both matter.
Gemini earned the highest overall marks and was chosen most often by reviewers, with Grok second. Avoiding invented facts scored highest for every model, while brevity scored lowest. Gemini also wrote the longest answers, though at the most readable level. Human experts graded more strictly than the automated judges. Serious errors were rare but not absent.
Catalog record
| CREATOR | Christoph Reich, Jule Leverenz, Charlotte Brand, Lasse Niemeier, Isabel Branzei, Mustafa Yildirim, Farbod Sedaghat-Hamedani, Ali Amr, Norbert Frey, Benjamin Meder |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |