Who Gets Cited Most? Benchmarking Long-Context Language Models on Scientific Articles
Miao Li, Alexander Gurung, Irina Saparina & Mirella Lapata · 2025
Read Who Gets Cited Most? Benchmarking Long-Context Language Models on Scientific Articles by Miao Li, Alexander Gurung, Irina Saparina & Mirella Lapata (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21028.txt.
Description
Answering a question that needs arithmetic over dozens of full scientific papers is a stern test for language models. Miao Li and three co-authors build SciTrek, a 2025 dataset for long-context numerical reasoning. Existing long-context tests mostly check simple retrieval or use artificial contexts.
SciTrek questions require counting, sorting, aggregation, and comparison over collections of real articles. The questions are generated automatically by turning them into database queries over article metadata, such as titles, authors, and references. Ground-truth answers come from executing those queries, so they are exact. Inputs exceed sixty-four thousand tokens. The dataset both evaluates current models and supplies training material to improve their numerical reasoning over long documents.
Catalog record
| CREATOR | Miao Li, Alexander Gurung, Irina Saparina & Mirella Lapata |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-21028