Consistency in causal reasoning for large language models in scenarios of HIV antiretroviral treatment, drug interactions, and side effects
Mattia Prosperi, Simone Rancati, Yi Guo, Jingchuan Guo, Rotana M Radwan, Marco Salemi, Jiang Bian, Balu Bhasuran, Eric F Egelund, Jennifer Janelle, Preeti Manavalan, Nicole Maranchick, Simone Marini, Zhe He · 2026
Read Consistency in causal reasoning for large language models in scenarios of HIV antiretroviral treatment, drug interactions, and side effects by Mattia Prosperi, Simone Rancati, Yi Guo, Jingchuan Guo, Rotana M Radwan, Marco Salemi, Jiang Bian, Balu Bhasuran, Eric F Egelund, Jennifer Janelle, Preeti Manavalan, Nicole Maranchick, Simone Marini, Zhe He (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13562555.txt.
Description
Mattia Prosperi and colleagues tested whether large language models reason consistently about cause in HIV treatment scenarios, publishing the evaluation in 2026. Causal reasoning underpins diagnosis and drug choices, yet most benchmarks test pattern matching rather than the three rungs of Pearl's ladder: association, intervention, and counterfactuals. The team prompted several open and closed models, including GPT-4o and LLaMA-3, with rich HIV antiretroviral vignettes about interactions and side effects.
Models agreed with themselves moderately well, but agreement faded from association to counterfactual questions, and agreement between models was notably lower. Some models cited inaccessible online sources, and clinical experts only modestly aligned with model outputs, rating counterfactual scenarios weakest in realism and depth. The authors conclude the models are not yet reliable enough for causal reasoning in HIV pharmacoepidemiology.
Catalog record
| CREATOR | Mattia Prosperi, Simone Rancati, Yi Guo, Jingchuan Guo, Rotana M Radwan, Marco Salemi, Jiang Bian, Balu Bhasuran, Eric F Egelund, Jennifer Janelle, Preeti Manavalan, Nicole Maranchick, Simone Marini, Zhe He |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |