Automating data extraction in meta-research: A multi-model benchmark in network psychometrics papers
Benjamin Simsa, Artem Buts, Ivan Ropovik, Matúš Adamkovič · 2026
Read Automating data extraction in meta-research: A multi-model benchmark in network psychometrics papers by Benjamin Simsa, Artem Buts, Ivan Ropovik, Matúš Adamkovič (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13197235.txt.
Description
Benjamin Simsa and co-authors test whether today's large language models can pull data out of research papers accurately enough for meta-research. Hand extraction is slow and error-prone, and the tasks range from copying plain facts, like sample sizes, to judging softer points. The team ran five leading models through an automated pipeline over 43 papers in network psychometrics, a field that maps links between symptoms.
Accuracy ranged from about 80 to 91 percent across the models. Scores were higher for facts stated plainly in the text and lower where judgment was needed, and most models could signal doubt in the contentious cases. The authors compare cost against accuracy and list the kinds of variables that suit automatic coding, plus common pitfalls to avoid.
Catalog record
| CREATOR | Benjamin Simsa, Artem Buts, Ivan Ropovik, Matúš Adamkovič |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |