CS-PaperSum : A Large-Scale Dataset of AI-Generated Summaries for Scientific Papers
Javin Liu, Aryan Vats, Zihao He · 2025
Read CS-PaperSum : A Large-Scale Dataset of AI-Generated Summaries for Scientific Papers by Javin Liu, Aryan Vats, Zihao He (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20582.txt.
Introduction
Javin Liu and co-authors release, in a 2025 paper, a large data set of AI-written summaries of computer science papers. The field grows too fast for anyone to read it all, and old data sets give facts about papers but not tight summaries of what each one did. Their set covers more than ninety thousand papers from thirty-one top conferences. Each entry adds a structured summary made by ChatGPT.
Quality checks compare the summaries' inner representations with the papers' own, and count shared key terms; both show the core ideas survive the squeeze. A case study then reads the whole set for trends, catching the rise of self-supervised learning, retrieval-augmented generation, and multimodal AI. The authors offer the set as fuel for trend forecasting and machine-aided discovery.
Catalog record
| CREATOR | Javin Liu, Aryan Vats, Zihao He |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-20582
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CS-PaperSum : A Large-Scale Dataset of AI-Generated Summaries for Scientific Papers Javin Liu, Aryan Vats, Zihao He Abstract . The rapid expansion of scientific literature in computer science presents challenges in tracking research trends and extracting key insights. Existing datasets provide metadata but lack structured summaries that capture core contributions and methodologies. We introduce CS-PaperSum , a large-scale dataset of 91,919 papers from 31 top-tier computer science conferences, enriched with AI-generated structured summaries using ChatGPT. To assess summary quality, we conduct embedding alignment analysis and keyword overlap analysis, demonstrating strong preservation of key concepts. We further present a case study on AI research trends, highlighting shifts in methodologies and interdisciplinary crossovers, including the rise of self-supervised learning, retrieval-augmented generation, and multimodal AI. Our dataset enables automated literature analysis, research trend forecasting, and AI-driven scientific discovery, providing a valuable resource for researchers, policymakers, and scientific information retrieval systems. 1 1 1 Our dataset is available at https://github.com/zihaohe123/CS-PaperSum . DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper 1. Introduction The rapid expansion of scientific literature in computer science has created significant challenges for researchers attempting to stay informed about new developments, track emerging research trends, and synthesize insights from an overwhelming number of publications. Each year, thousands of papers are published in top-tier venues, spanning a wide range of subfields including artificial intelligence, natural language processing, computer vision, and cybersecurity. As the pace of research accelerates, the ability to efficiently extract key insights from the vast body of scientific literature has become increasingly important. Existing open-access repositories and metadata-rich datasets such as the Semantic Scholar Open Research Corpus (S2ORC) ( Lo et al., 2020 ) and Microsoft Academic Graph (MAG) ( Wang et al., 2020 ) provide bibliographic information, abstracts, and citation networks, but they lack structured summaries that capture the core contributions, methodologies, and future research directions of individual papers. This limitation makes large-scale automated analysis of research progress difficult, hindering the development of AI-driven tools for literature synthesis and trend detection. To address these challenges, we introduce CS-PaperSum , a large-scale dataset of 91,919 computer science papers drawn from 31 major conferences. Unlike previous datasets that primarily provide raw text and metadata, CS-PaperSum incorporates structured AI-generated summaries produced using GPT-3.5 ( Ouyang et al., 2022 ) . Each summary distills the paper’s key contributions, novel methodologies, evaluation metrics, and future research directions, offering a concise yet comprehensive view of its content. By standardizing this information across a broad collection of papers, the dataset enables large-scale scientometric studies, automated literature reviews, and AI-assisted research synthesis. The motivation for constructing this dataset arises from several critical challenges in scientific paper analysis. The sheer volume of research output makes it impractical for scholars to manually read and summarize papers at scale, even with the aid of search engines and citation networks ( Fortunato et al., 2018 ) . While platforms such as Google Scholar and Semantic Scholar index vast numbers of publications, they do not provide structured insights that help researchers quickly understand the significance of a paper beyond its title and abstract. Moreover, existing approaches to research trend analysis, such as topic modeling ( Blei et al., 2003 ) , offer only high-level insights without capturing the fine-grained technical contributions of individual studies. Leveraging large language models to generate structured paper summaries, CS-PaperSum provides a scalable solution for extracting and organizing scientific knowledge. To ensure the reliability of the AI-generated summaries, we conduct a thorough quality assessment, evaluating the semantic alignment between ChatGPT summaries and original papers. We employ SciBERT ( Beltagy et al., 2019 ) , a domain-specific transformer model trained on scientific text, to generate embeddings for both the original paper content and the AI-generated summaries. Visualization using t-SNE dimensionality reduction demonstrates that the embeddings of ChatGPT-generated summaries align well with those of the original papers while maintaining clear separation between different research domains. Additionally, we perform keyword overlap analysis using KeyBERT 2 2 2 https://maartengr.github.io/KeyBERT/ to compare the extracted keywords from the summaries and the original text. High overlap percentages indicate that the AI-generated summaries effectively preserve key concepts, making them a reliable tool for automated literature synthesis. Beyond evaluating summary quality, we showcase a practical application of CS-PaperSum by analyzing research trends across major AI conferences. By extracting key topics from the structured summaries, we identify the evolution of research focus areas in NeurIPS, AAAI, EMNLP, and ICCV, revealing shifts in methodologies and emerging interdisciplinary intersections. For instance, we observe a transition in NeurIPS from traditional reinforcement learning methods ( Mnih et al., 2015 ) toward graph neural networks ( Bronstein et al., 2021 ) and large-scale self-supervised learning models ( Liu et al., 2021 ) . In NLP research, the dominance of statistical machine translation in earlier years ( Koehn et al., 2003 ) has been replaced by pretrained transformer-based architectures ( Devlin et al., 2019 ) , retrieval-augmented generation ( Lewis et al., 2020 ) , and efficient fine-tuning techniques ( Houlsby et al., 2019 ) . Computer vision research has similarly evolved, with ICCV showing increasing interest in vision transformers ( Dosovitskiy et al., 2020 ) , generative adversarial networks ( Goodfellow et al., 2014 ) , and neural radiance fields for 3D reconstruction ( Mildenhall et al., 2021 ) . The dataset also allows for the detection of cross-disciplinary research trends, such as the adoption of self-supervised learning techniques originally developed for vision ( Chen et al., 2020 ) into NLP applications ( Gao et al., 2021 ) . CS-PaperSum represents a significant step forward in the automated structuring and synthesis of scientific knowledge. By providing a large-scale dataset enriched with ChatGPT-generated structured summaries, we aim to bridge the gap between raw bibliographic data and actionable scientific insights. The dataset enables automated literature analysis, facilitates research trend detection, and provides a foundation for developing AI systems that assist in scientific discovery. As the landscape of AI and computer science continues to evolve, datasets like CS-PaperSum will play a crucial role in making scientific literature more accessible, navigable, and analyzable at scale. 2. Related Work Our work builds upon prior efforts in scientific literature analysis, automatic summarization, and AI-driven research trend detection. Several large-scale datasets provide access to scientific papers and metadata, facilitating bibliometric studies and information retrieval. The Semantic Scholar Open Research Corpus (S2ORC) ( Lo et al., 2020 ) contains full-text scientific papers and metadata, supporting NLP tasks such as citation analysis and topic modeling. Similarly, Microsoft Academic Graph (MAG) ( Wang et al., 2020 ) provides a large knowledge graph of scholarly publications, including citation networks and author affiliations. Other datasets, such as CORE ( Knoth and Zdrahal, 2012 ) , focus on aggregating open-access research papers. However, these resources primarily contain raw text, abstracts, and citation metadata, without structured summaries of research contributions, making