Benchmarking reliability and calibration of LLMs for multi-cancer early detection test communication
Koki Takabatake, Maria Sol Rosito, Danielle Braun, Catherine Marinac, Elizabeth O’Donnell, Giovanni Parmigiani · 2026
Read Benchmarking reliability and calibration of LLMs for multi-cancer early detection test communication by Koki Takabatake, Maria Sol Rosito, Danielle Braun, Catherine Marinac, Elizabeth O’Donnell, Giovanni Parmigiani (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/PMC13264670.txt.
Description
Koki Takabatake and colleagues published a 2026 benchmarking study on how well large language models talk about multi-cancer early detection tests, blood tests that screen for many cancers at once. Clear talk about such tests matters, and models that sound sure while wrong could mislead. The team set expert-checked questions and graded five models.
The set held 94 multiple choice and 13 free response questions drawn from 19 papers. Each model answered three times and rated its own confidence. All scored high overall, with Gemini on top at 96.3 percent, though its test terms differed. Most errors were wrong recalls of exact numbers. Several models still made errors while claiming high confidence. The authors urge human checks before clinical use.
Catalog record
| CREATOR | Koki Takabatake, Maria Sol Rosito, Danielle Braun, Catherine Marinac, Elizabeth O’Donnell, Giovanni Parmigiani |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39173 |
| LICENSE | https://creativecommons.org/licenses/by/4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |