Closing the screening gap but not the writing gap: a two-topic evaluation of LLMs for systematic reviews and meta-analyses in hepatology

Yuntao Zou, Iris Kim, Nan Gao, Michelle Li, Mi-Ok Kim, Jin Ge · 2026

Read Closing the screening gap but not the writing gap: a two-topic evaluation of LLMs for systematic reviews and meta-analyses in hepatology by Yuntao Zou, Iris Kim, Nan Gao, Michelle Li, Mi-Ok Kim, Jin Ge (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/PMC13287001.txt.

Description

Yuntao Zou and colleagues split-test language models on the two halves of evidence synthesis in a 2026 evaluation study. Two hepatology questions, a heart drug in cirrhosis and blood thinners for a portal vein clot, supplied the test bed. Models screened titles and abstracts against set criteria, then a retrieval-fed model drafted ten full reviews per topic for scoring against the PRISMA checklist by a separate judge model.

Screening shone: agreement with human reviewers ran high, catching most eligible studies while wrongly passing few. Hours of manual screening collapsed from 62 and 30 down to 3 and 2. Drafting lagged badly, acing titles and introductions but sagging below two-thirds compliance in abstracts and discussions. Judge scores tracked human scoring closely, and the authors conclude screening is ready for work while drafting still needs a human hand.

Catalog record

CREATORYuntao Zou, Iris Kim, Nan Gao, Michelle Li, Mi-Ok Kim, Jin Ge
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39850
LICENSEhttps://creativecommons.org/licenses/by/4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/article/MED/PMC13287001

Browse the full library · About the Tower of Babel