Evaluating Large Language Models for Automated Evidence Synthesis in Neuroimaging AI: A Multi-Model Benchmark
Umid Sulaimanov, Nafiye Sanlier, Ariorad Moniri, Behman Demir, Yerkebulan Serikkanov, Ahmed Rasim Bayramoglu, Maryam Sabah Al-Jebur, Melih Yucel Sanlier, Ugur Erginoglu, Erkin Otles, Simon Gashaw Ammanuel, Abdullah Keles, Ufuk Erginoglu, Mustafa Kemal Baskaya · 2026
Read Evaluating Large Language Models for Automated Evidence Synthesis in Neuroimaging AI: A Multi-Model Benchmark by Umid Sulaimanov, Nafiye Sanlier, Ariorad Moniri, Behman Demir, Yerkebulan Serikkanov, Ahmed Rasim Bayramoglu, Maryam Sabah Al-Jebur, Melih Yucel Sanlier, Ugur Erginoglu, Erkin Otles, Simon Gashaw Ammanuel, Abdullah Keles, Ufuk Erginoglu, Mustafa Kemal Baskaya (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13257454.txt.
Description
Umid Sulaimanov and co-authors benchmark four leading language models on evidence extraction from brain imaging papers in a 2026 study. Systematic reviews need structured data pulled from many articles, a slow manual chore. Ninety-one neuroimaging artificial intelligence papers each yielded twenty-two variables of rising difficulty.
Exact-match accuracy landed near half overall for every model, with the top system reaching about fifty-six percent. Simple categorical fields scored near ninety percent for all models. Accuracy collapsed on variables needing clinical judgment or synthesis across sections, falling as low as a few percent. Misclassification was the commonest error, and one metric scored zero through a format mismatch rather than true failure. Review teams get a sober guide: automate the easy fields, keep experts on the hard ones.
Catalog record
| CREATOR | Umid Sulaimanov, Nafiye Sanlier, Ariorad Moniri, Behman Demir, Yerkebulan Serikkanov, Ahmed Rasim Bayramoglu, Maryam Sabah Al-Jebur, Melih Yucel Sanlier, Ugur Erginoglu, Erkin Otles, Simon Gashaw Ammanuel, Abdullah Keles, Ufuk Erginoglu, Mustafa Kemal Baskaya |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |