Using full agreement across multiple large language models for title-and-abstract screening in systematic reviews: a proof-of-concept
Frederic Hilkenmeier, Merle Stoltenberg, Christian Stierle · 2026
Read Using full agreement across multiple large language models for title-and-abstract screening in systematic reviews: a proof-of-concept by Frederic Hilkenmeier, Merle Stoltenberg, Christian Stierle (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/PMC13263951.txt.
Description
Frederic Hilkenmeier and two colleagues present a 2026 proof-of-concept study on using large language models to screen papers for systematic reviews. Screening titles and short summaries by hand is slow as the number of papers grows. Their idea is to trust a machine label only when three models agree.
The team tested ChatGPT, Gemini, and Claude on six data sets. About four in five summaries got the same label from all three models and could be sorted without a person. On that agreed group, scores beat earlier single-model methods on all set measures. Cases where models disagreed still went to human review. The authors say wider testing is still needed.
Catalog record
| CREATOR | Frederic Hilkenmeier, Merle Stoltenberg, Christian Stierle |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55502 |
| LICENSE | https://creativecommons.org/licenses/by/4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |