Citegeist: Automated Generation of Related Work Analysis on the arXiv Corpus
Claas BegerCornell University [email protected], Carl-Leander HennekingCornell University [email protected] · 2025
Read Citegeist: Automated Generation of Related Work Analysis on the arXiv Corpus by Claas BegerCornell University [email protected], Carl-Leander HennekingCornell University [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23229.txt.
Introduction
Claas Beger and Carl-Leander Henneking built Citegeist, a tool that drafts related-work sections from real papers, in a 2025 release. Large language models write fluent research prose but invent sources and lack a live link to the literature. For survey sections, fake citations are a fatal flaw.
Citegeist grounds the model in the arXiv corpus through retrieval augmented generation. Embedding matches find candidate papers, summaries condense them, and a multi-stage filter weeds out weak fits before drafting. The system also folds new and changed papers into its base without a full rebuild. Outputs carry real citations a reader can check. The team released a public website plus a harness that runs with several model back ends, aiming at everyday use by working researchers.
Catalog record
| CREATOR | Claas BegerCornell University [email protected], Carl-Leander HennekingCornell University [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-23229
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Citegeist: Automated Generation of Related Work Analysis on the arXiv Corpus Claas BegerCornell University [email protected], Carl-Leander HennekingCornell University [email protected] Abstract Large Language Models provide significant new opportunities for the generation of high-quality written works. However, their employment in the research community is inhibited by their tendency to hallucinate invalid sources and lack of direct access to a knowledge base of relevant scientific articles. In this work, we present Citegeist: An application pipeline using dynamic Retrieval Augmented Generation (RAG) on the arXiv Corpus to generate a related work section and other citation-backed outputs. For this purpose, we employ a mixture of embedding-based similarity matching, summarization, and multi-stage filtering. To adapt to the continuous growth of the document base, we also present an optimized way of incorporating new and modified papers. To enable easy utilization in the scientific community, we release both, a website , as well as an implementation harness that works with several different LLM implementations. 1 Introduction Significant advances in language generation quality in recent years have enabled the employment of Large Language Models (LLMs) as writing assistants for various fields and purposes. However, while the quality of produced texts is often high, LLMs face critical challenges in domains requiring factual precision and verifiable citations, such as academic writing. Their tendency to hallucinate invalid or non-existent sources Chelli et al. 2024 ; Magesh et al. 2024 ; Buchanan et al. 2024 , combined with the lack of direct integration with up-to-date scientific knowledge bases, limits their applicability in research contexts. To address the former challenge, Lewis et al. 2021 introduced Retrieval-Augmented-Generation (RAG) by directly employing a Wikipedia-based vector index to insert external knowledge into the model context. This work aims to transition this approach to the arXiv corpus, a popular and ever-growing dataset consisting of approximately 2.6 million academic papers. The vast size, frequent updates, and the availability of a dedicated Python library make the arXiv an intuitive choice for research-centered retrieval-augmented generation (RAG). However, this scale imposes challenges in data interaction, as the full-text dataset exceeds 1 TB. To mitigate this, we perform multiple filtering steps based on abstracts and retrieve full documents only for identified relevant sources. We then construct a vector database of abstract embeddings to enable efficient similarity search. Additionally, recognizing that the related work requirements vary across research fields and paper contents, we introduce three key hyperparameters— breadth , depth , and diversity —which are employed in our retrieval algorithm. We also place a strong emphasis on proper citation techniques and rigorously evaluate our approach in terms of source relevancy and the writing quality of the generated results. Beyond the generation of related work sections, we adapt our method to answer scientific questions with citation-based responses, yielding promising results. Key Contributions: • Development of a dynamic retrieval and synthesis application for related work generation. • Introduction of three key hyperparameters— breadth , depth , and diversity —to fine-tune the content and style of the result. • Support for uploading full PDFs to enhance content-based retrieval. • Employment of full paper texts through alternating between importance weighting and summarization techniques. Figure 1: Our pipeline takes either an abstract or a full paper as input and produces a related works analysis, incl. all citations. We initially retrieve a set of candidate works that we filter in multiple steps, and then we summarize and synthesize relevant content to produce the related works analysis. 2 Related Work Recent advancements using RAG have been shown to enhance performance on knowledge-intensive tasks by integrating external sources Lewis et al. 2021 . Building on this, Jiang et al. 2023 and Trivedi et al. 2023 demonstrate how retrieval datasets can be dynamically updated to incorporate the latest knowledge. Alternatively, Yao et al. 2023 demonstrates how data may be accessed directly through Agentic Workflow. Our project aims to use the arXiv corpus arXiv.org submitters 2024 for a dynamic RAG system applied to the task of related work analysis. These works are highly relevant to our approach since arXiv experiences close to 20.000 article submissions a month (see Figure 4 ) and correct data migration is non-trivial. This issue is especially prominent for applications that explicitly train a model for information or document retrieval Kishore et al. 2023 . Furthermore, there has been research on the efficiency of such retrieval models for scientific articles Ajith et al. 2024 with a focus on extractive question-answering but without utilization of summarization or RAG. In recent literature, related work generation has emerged as a prominent task within the domain of scientific multi-document summarization Li and Ouyang 2022 . However, many prior approaches rely heavily on a predefined citation list Hu and Wan 2014 ; Chen and Zhuge 2019 ; Wang et al. 2020 ; Deng et al. 2021 . In contrast, we propose an independent extraction pipeline that does not require citation data, addressing a significant gap in the existing body of work. Our method leverages multi-stage retrieval, filtering, and summarization techniques to enhance related work generation. To the best of our knowledge, few works have explored this direction. For instance, git focuses on synthesizing large bodies of academic literature, but their approach is designed for general literature reviews rather than for extracting related work tailored to a specific input abstract. Similarly, Asai et al. 2024 aims at unifying relevant scientific papers for a given query, with a primary focus on scientific question answering. However, their method does not incorporate hyperparameters to fine-tune results, nor does it support processing full PDFs. Furthermore, our approach distinguishes itself by employing a dynamic combination of summarization and relevance estimation during full-text processing. 3 Method We leverage an existing arXiv metadata corpus, which contains abstracts and paper IDs. To build up our vector database, we extract abstracts and perform an embedding operation using the all-mpnet-base-v2 Henderson et al. 2019 Sentence Transformer. We chose this model because of its applicability to paragraphs of up to about 384-word pieces, which is suitable for academic abstracts and filtered page contents. Using this model, we embed all abstracts contained in the corpus and generate a corresponding hash, which we utilize later to refresh the existing database efficiently. Further, we compute a topic assignment using a fine-tuned BERTopic version Grootendorst 2022 , which can be used to produce content-based subsets. Based on this data, we build up our vector database using Milvus Wang et al. 2021 , which is optimized for similarity-based lookups. 3.1 Retrieval and Generation Our proposed architecture works on an excerpt of the target paper, most of the time this will be the abstract (alternatives are discussed later in this section). We embed this abstract and perform a similarity-based search using a cosine-similarity metric on our embedding metadata. Based on the results, we extract candidate abstracts. We parameterize the selection criteria to account for different user requirements. The selection starts with the abstract that has the highest similarity score. Thereafter, the next paper i^{*} is chosen iteratively according to i^{*}=\arg\max_{i\notin S}\left[(1-w)\cdot s_{i}+w\cdot\left(1-\min_{p_{j}\in S}\operatorname{sim}(e_{i},e_{j})\right)\right] (1) Where P=\{p_{1},p_{2},\dots,p_{n}\} is the set of all papers in the query result, S is the subset of chosen papers, and each paper p_{i} has an abstract embedding e_{i}\in\mathbb{R}^{d} . The parameter w\in[0,1] determines the trade-off between prioritizing similarity and