Biomedical Large Language Models and Prompt Engineering for Causality Assessment of Individual Case Safety Reports in Pharmacovigilance
Nicole Sonne Heckmann, Despoina Georgia Papoutsi, Maria Antonietta Barbieri, Vera Battini, Søren Norlin Mølgaard, Simon Ørum Schmidt, Lars Melskens, Maurizio Sessa · 2026
Read Biomedical Large Language Models and Prompt Engineering for Causality Assessment of Individual Case Safety Reports in Pharmacovigilance by Nicole Sonne Heckmann, Despoina Georgia Papoutsi, Maria Antonietta Barbieri, Vera Battini, Søren Norlin Mølgaard, Simon Ørum Schmidt, Lars Melskens, Maurizio Sessa (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13593631.txt.
Description
Nicole Heckmann and seven co-authors tested medical language models at judging whether a drug caused a reported harm, in a 2026 study. Such judgments, called causality assessments, guide drug safety work. One hundred fifty case reports were drawn from United States databases. Two standard scoring algorithms structured the task. Five model and prompting combinations were compared with human experts.
The best setup reached sixty-four percent agreement with experts on final calls. It paired a mid-size medical model with stepwise prompting and one scoring algorithm. Agreement collapsed on key sub-questions, though. Models struggled with timing, other causes, and listed side effects. Irrelevant answers were a frequent failure. The models beat earlier general-purpose systems but remain unreliable. The authors advise against trusting them alone for safety judgments.
Catalog record
| CREATOR | Nicole Sonne Heckmann, Despoina Georgia Papoutsi, Maria Antonietta Barbieri, Vera Battini, Søren Norlin Mølgaard, Simon Ørum Schmidt, Lars Melskens, Maurizio Sessa |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |