ArxivBench: Can LLMs Assist Researchers in Conducting Research?

Ning Li, Jingran Zhang, Justin Cui · 2025

Read ArxivBench: Can LLMs Assist Researchers in Conducting Research? by Ning Li, Jingran Zhang, Justin Cui (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-10496.txt.

Introduction

Ning Li, Jingran Zhang, and Justin Cui benchmark how reliably large language models cite real research papers. Their paper came out in 2025. Models often produce confident but wrong references, including links to papers that do not exist, which can mislead students and researchers.

Their benchmark asks models to supply relevant papers with correct archive links across eight broad subjects and five computing subfields. Accuracy varied widely by subject and by model. One leading commercial model held a clear advantage in giving both relevant and accurate answers. Most models did best in artificial intelligence topics and worse elsewhere. The authors release their code and dataset openly. They intend the benchmark to push developers toward more dependable academic assistance.

Catalog record

CREATORNing Li, Jingran Zhang, Justin Cui
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-10496

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ArxivBench: Can LLMs Assist Researchers in Conducting Research? Ning Li, Jingran Zhang, Justin Cui Abstract Large language models (LLMs) have demonstrated remarkable effectiveness in completing various tasks such as reasoning, translation, and question answering. However the issue of factual incorrect content in LLM-generated responses remains a persistent challenge. In this study, we evaluate both proprietary and open-source LLMs on their ability to respond with relevant research papers and accurate links to articles hosted on the arXiv platform, based on high level prompts. To facilitate this evaluation, we introduce arXivBench, a benchmark specifically designed to assess LLM performance across eight major subject categories on arXiv and five subfields within computer science, one of the most popular categories among them. Our findings reveal a concerning accuracy of LLM-generated responses depending on the subject, with some subjects experiencing significantly lower accuracy than others. Notably, Claude-3.5-Sonnet exhibits a substantial advantage in generating both relevant and accurate responses. And interestingly, most LLMs achieve a much higher accuracy in the Artificial Intelligence sub-field than other sub-fields. This benchmark provides a standardized tool for evaluating the reliability of LLM-generated scientific responses, promoting more dependable use of LLMs in academic and research environments. Our code is open-sourced at https://github.com/arxivBenchLLM/arXivBench and our dataset is available on huggingface at https://huggingface.co/datasets/arXivBenchLLM/arXivBench . 1 Introduction As large language models (LLMs) are increasingly employed in academic studies and research ( Liang et al. 2024 ; Si et al. 2024 ) , the accuracy of the content they generate need to be inspected carefully, particularly in rigorous academic settings. One of the major challenges with LLMs is their tendency to generate fabricated content without valid references ( Ji et al. 2023 ; Huang et al. 2023 ) , which can undermine the integrity of the research process. Although LLMs have revolutionized many aspects of daily life, the lack of a standardized benchmark for evaluating their effectiveness in finding research papers, a crucial factor for advancing LLM techniques, is particularly concerning despite that they have been trained on related corpses such as arXiv ( Tilwani et al. 2024 ; Naveed et al. 2023 ) , a major platform for sharing research papers. In this paper, we introduce arXivBench, the first benchmark specifically designed to assess the accuracy of model-generated content in response to research-related prompts on arXiv. ArXivBench comprises a carefully curated dataset of 6,500 prompts from 13 fields, providing a robust platform for evaluating their performance in academic contexts, along with an automatic prompt generation pipeline allowing for the easy scalability of the dataset and an evaluation suite for final performance evaluation. Our benchmark offers the flexibility to examine a diverse range of LLMs, both proprietary and open-source, across different academic fields. We evaluated a total of 15 widely-used LLMs, comparing their ability to generate accurate and relevant content under academic rigor. In summary, our contributions are: • We contribute a comprehensive prompt dataset designed to assess the accuracy of large language models in generating accurate responses based on arXiv. • We benchmark the performance of 15 different models across 8 major subjects and 5 specific sub-fields within computer science. • Our findings reveal several insights such as a high correlation between model size and accuracy and interestingly, most LLMs achieve a much higher accuracy in the Artificial Intelligence (AI) sub-field than other subfields in computer science . 2 Related work Large language models (LLMs) have demonstrated remarkable proficiency across a wide array of tasks, showcasing their versatility and effectiveness in different domains ( Abdin et al. 2024 ; Kenton and Toutanova 2019 ; Brown 2020 ; Basyal and Sanghvi 2023 ; Vaswani 2017 ) . In mathematical reasoning, LLMs can solve complex problems, provide step-by-step explanations, and assist in advanced problem-solving ( Kaddour et al. 2023 ; Ahn et al. 2024 ) . In the realm of literature creation, these models are capable of generating coherent and creative content, from poetry to narrative storytelling, often emulating diverse writing styles ( Gómez-Rodríguez and Williams 2023 ; Wang et al. 2024 ) . One similar work to ours is the REASONS benchmark ( Tilwani et al. 2024 ) , which assesses the ability of LLMs to generate accurate citations, but it operates within a relatively constrained framework. Specifically, it prompts LLMs to retrieve the authors of a specific paper title. In contrast, our arXivBench takes a more open-ended approach, evaluating the models’ ability to generate relevant and accurate information across a wider range of topics, such as providing research papers related to a particular field. Instead of limiting the responses to predefined papers, we aim to assess the models’ capacity to generate response that are applicable to broader subject areas. This approach allows for a comprehensive evaluation of the models’ effectiveness. Retrieval-Augmented Generation (RAG) technology has garnered significant attention and been extensively studied in recent years due to its exceptional ability to mitigate the generation of factually incorrect content by large language models (LLMs) ( Gao et al. 2023b ; Ram et al. 2023 ) . By integrating a retrieval mechanism, RAG enhances the accuracy of responses by sourcing relevant information from external knowledge, which the model may not have directly learned during training ( Lewis et al. 2020 ) . Since RAG is a post-training augmentation technology that’s not applied to most of the LLM APIs, we specifically evaluate LLM performances without RAG. Figure 1: Overall process of arXivBench. Responses shown above are real responses from benchmarked LLMs. 3 Dataset Construction 3.1 Prompt generation Our benchmark consists of two main components. The first part includes 4,000 prompts across eight major subject categories on arXiv: Math, Computer Science (CS), Quantitative Biology (QB), Physics, Quantitative Finance (QF), Statistics, Electrical Engineering and Systems Science (EESS), and Economics. For each subject, we use a LLM to generate 500 prompts, each containing specific requests for arXiv links. The second part of arXivBench includes 2,500 prompts from five subfields within computer science, one of the most popular fields among all the categories: Artificial Intelligence (AI), Cryptography and Security (C&S), Data Structures and Algorithms (DS&AL), Operating Systems (OS), and Programming Languages (PL). The motivation behind this is to assess LLMs’ abilities to generate accurate and relevant information within more specialized domains. Math Computer Science Quantitative Biology Physics Quantitative Finance Statistics EESS Economics GPT-3.5-turbo 7.63 26.55 3.85 15.78 8.60 8.92 2.57 0.94 GPT-4o-2024-08-06 9.92 26.84 3.19 21.07 8.06 9.09 3.91 1.43 Llama-3.1-8B-Instruct-Turbo 2.38 9.65 0.22 7.92 0.28 1.06 0.28 - Llama-3.1-70B-Instruct-Turbo 8.40 22.87 2.54 23.31 4.00 7.88 2.94 0.73 Llama-3.1-405B-Instruct-Turbo 14.87 28.21 2.28 24.43 11.49 15.70 6.17 2.62 Qwen2-72B-Instruct 4.56 13.81 1.13 6.47 0.13 9.87 1.00 - Gemini-1.5-flash 2.88 5.73 1.20 9.21 0.13 1.87 0.27 - Gemini-1.5-pro 3.64 17.56 0.94 11.99 2.48 5.27 0.40 0.27 Gemma-2 Instruct (9B) 3.40 12.65 0.49 6.92 0.14 2.00 0.23 0.34 Gemma-2 Instruct (27B) 2.27 10.92 0.20 5.70 0.07 2.00 0.47 0.07 Mixtral-8x22B-Instruct-v0.1-2 7.55 16.99 0.82 15.06 1.72 4.66 1.01 0.13 Mistral-small-latest 4.01 16.73 1.63 17.12 0.14 2.83 0.80 0.14 Mistral-large-latest 9.33 21.20 2.16 21.99 3.47 7.21 1.20 0.33 Claude-3-haiku 3.90 18.98 1.39 17.17 2.80 4.44 1.74 0.47 Claude-3.5-sonnet 12.53 39.93 6.07 21.80 20.00 16.67 7.67 1.00 Average 6.48 19.24 1.87 15.06 4.23 6.63 2.04 0.56 Table 1: Accuracy rates on arXivBench of eight major subjects, where

Read the complete text