Large language models encode clinical knowledge

Karan Singhal, Shekoofeh Azizi, Tao Tu et al. · 2023

Read Large language models encode clinical knowledge by Karan Singhal, Shekoofeh Azizi, Tao Tu et al. (2023) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-10396962.txt.

Description

Singhal, Azizi, Tu, and co-authors test how much medical knowledge large language models hold. These computer systems can write and answer in striking ways, yet medicine sets a very high bar for safe use. Past tests leaned on small, fixed sets of exam-style questions.

The team set out to judge clinical knowledge in a broader and tougher way. Their work asks whether a model truly grasps diagnosis and care, or merely recalls text patterns. Clear answers matter for doctors, patients, and builders alike, since weak tests could let unsafe tools reach the clinic too soon.

Catalog record

CREATORKaran Singhal, Shekoofeh Azizi, Tao Tu et al.
YEAR2023
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91 KB
LICENSECreative Commons Attribution License
RIGHTSCC BY
SOURCEPubMed Central

Source: https://pmc.ncbi.nlm.nih.gov/articles/PMC10396962/

Browse the full library · About the Tower of Babel