Evaluating Large Language Models for Automated Evidence Synthesis in Neuroimaging AI: A Multi-Model Benchmark

Umid Sulaimanov, Nafiye Sanlier, Ariorad Moniri, Behman Demir, Yerkebulan Serikkanov, Ahmed Rasim Bayramoglu, Maryam Sabah Al-Jebur, Melih Yucel Sanlier, Ugur Erginoglu, Erkin Otles, Simon Gashaw Ammanuel, Abdullah Keles, Ufuk Erginoglu, Mustafa Kemal Baskaya · 2026

Read Evaluating Large Language Models for Automated Evidence Synthesis in Neuroimaging AI: A Multi-Model Benchmark by Umid Sulaimanov, Nafiye Sanlier, Ariorad Moniri, Behman Demir, Yerkebulan Serikkanov, Ahmed Rasim Bayramoglu, Maryam Sabah Al-Jebur, Melih Yucel Sanlier, Ugur Erginoglu, Erkin Otles, Simon Gashaw Ammanuel, Abdullah Keles, Ufuk Erginoglu, Mustafa Kemal Baskaya (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13257454.txt.

Description

Umid Sulaimanov and co-authors benchmark four leading language models on evidence extraction from brain imaging papers in a 2026 study. Systematic reviews need structured data pulled from many articles, a slow manual chore. Ninety-one neuroimaging artificial intelligence papers each yielded twenty-two variables of rising difficulty.

Exact-match accuracy landed near half overall for every model, with the top system reaching about fifty-six percent. Simple categorical fields scored near ninety percent for all models. Accuracy collapsed on variables needing clinical judgment or synthesis across sections, falling as low as a few percent. Misclassification was the commonest error, and one metric scored zero through a format mismatch rather than true failure. Review teams get a sober guide: automate the easy fields, keep experts on the hard ones.

Catalog record

CREATORUmid Sulaimanov, Nafiye Sanlier, Ariorad Moniri, Behman Demir, Yerkebulan Serikkanov, Ahmed Rasim Bayramoglu, Maryam Sabah Al-Jebur, Melih Yucel Sanlier, Ugur Erginoglu, Erkin Otles, Simon Gashaw Ammanuel, Abdullah Keles, Ufuk Erginoglu, Mustafa Kemal Baskaya
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC13257454

Browse the full library · About the Tower of Babel