Performance of Zero-Shot Classifiers for Categorizing RCT Abstracts by Intervention Type: Validation Study
Diana Buitrago-Garcia, Delphine S Courvoisier, Sami Capderou, Michele Iudici, Denis Mongin, Andrew Coristine · 2026
Read Performance of Zero-Shot Classifiers for Categorizing RCT Abstracts by Intervention Type: Validation Study by Diana Buitrago-Garcia, Delphine S Courvoisier, Sami Capderou, Michele Iudici, Denis Mongin, Andrew Coristine (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/PMC13278612.txt.
Description
Diana Buitrago-Garcia leads a 2026 validation study of AI classifiers that sort clinical trial abstracts by intervention type, drug versus non-drug. Sorting abstracts is a slow step in evidence synthesis. The team tested two zero-shot models, which classify text without task-specific training, plus a prompted large language model, on 1,054 rheumatology trial abstracts against a human standard.
The best zero-shot model reached about eighty-eight percent accuracy with simple labels. The prompted model scored slightly higher. Both matched the accuracy of a human reviewer who lacked evidence synthesis training. Errors clustered on hard cases such as injections, supplements, and biological treatments. Such models could support one reviewer in screening work, the authors conclude.
Catalog record
| CREATOR | Diana Buitrago-Garcia, Delphine S Courvoisier, Sami Capderou, Michele Iudici, Denis Mongin, Andrew Coristine |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23419 |
| LICENSE | https://creativecommons.org/licenses/by/4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |