Accuracy Assessment of OpenAlex and Clarivate Scholar ID with an LLM-Assisted Benchmark

Renyu Zhao, Yunxin Chen · 2025

Read Accuracy Assessment of OpenAlex and Clarivate Scholar ID with an LLM-Assisted Benchmark by Renyu Zhao, Yunxin Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11610.txt.

Description

Renyu Zhao and Yunxin Chen grade two big scholar databases on a basic task: telling researchers with similar names apart. Studies of science itself depend on knowing who wrote what, yet name variants, initials, and spellings tangle the record. OpenAlex and Clarivate each mint their own scholar IDs as a fix. The team sampled 100 scholars per group from top-quartile journals, split by country, field, and output, and hand-built each scholar's true paper list with help from a search-augmented language model.

They then matched those lists against what each database's IDs claim, and scored precision and recall on the same journal set. The design lets readers see where each system shines and where it merges or splits people wrongly. The authors frame the benchmark as a guide for choosing an ID system by use case.

Catalog record

CREATORRenyu Zhao, Yunxin Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-11610

Browse the full library · About the Tower of Babel