Agreement between ChatGPT and human-derived multilevel meta-analyses: a reproducibility study across clinical evidence syntheses
Nikolai Ramadanov, Hannah Jaeger, Mikhail Salzmann, Robert Hable · 2026
Read Agreement between ChatGPT and human-derived multilevel meta-analyses: a reproducibility study across clinical evidence syntheses by Nikolai Ramadanov, Hannah Jaeger, Mikhail Salzmann, Robert Hable (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13621747.txt.
Description
Nikolai Ramadanov and colleagues tested whether ChatGPT can redo complex layered meta-reviews from hip studies. Language models now aid medical research, yet their grasp of stats-heavy layered models was unknown. Their rerun study collected seven earlier layered meta-reviews with data tables allowing solo reworking, spanning 40-plus pooled results.
ChatGPT's pooled results matched human-derived results closely, with small median gaps for number-based and binary results and minimal overall bias. Spread results also agreed, with larger gaps limited to extreme-spread or messy-input results. Ramadanov and colleagues back language models as adjunct rerun aids, with expert watch still needed.
Catalog record
| CREATOR | Nikolai Ramadanov, Hannah Jaeger, Mikhail Salzmann, Robert Hable |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |