SciNetBench: A Relation-Aware Benchmark for Scientific Literature Retrieval Agents

Chenyang Shao, Yong Li, Fengli Xu · 2026

Read SciNetBench: A Relation-Aware Benchmark for Scientific Literature Retrieval Agents by Chenyang Shao, Yong Li, Fengli Xu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03260.txt.

Description

Chenyang Shao, Fengli Xu, and Yong Li build SciNet, a dataset for testing whether AI research agents understand the relationships between scientific papers, not just their contents. Keyword and embedding search can find similar papers but miss corroborating or conflicting studies and the lineages along which techniques develop. SciNet is built on a meta-database of two hundred sixty-nine million papers across seven disciplines.

Its nearly nine thousand tasks probe three levels of relational understanding. Some tasks ask for papers around a focal work with novel knowledge structures. Others ask for the relationship between pairs of papers, or for paths through the citation network. The authors present SciNet as the first relation-aware benchmark of its kind for scientific retrieval agents. It aims to expose the fragmented knowledge and misread sentiment that purely content-based retrieval produces.

Catalog record

CREATORChenyang Shao, Yong Li, Fengli Xu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-03260

Browse the full library · About the Tower of Babel