AgentRxiv: Towards Collaborative Autonomous Research
Samuel Schmidgall, Michael Moor · 2025
Read AgentRxiv: Towards Collaborative Autonomous Research by Samuel Schmidgall, Michael Moor (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18102.txt.
Introduction
Samuel Schmidgall and Michael Moor let AI research labs share work through AgentRxiv in this 2025 paper. Science advances by many groups building on each other, not by lone breakthroughs. AI agents usually research in isolation, though. Their shared preprint server lets agent labs post and fetch reports.
Labs tasked with inventing reasoning tricks improved more with archive access. Gains reached 11.4 percent relative on a hard math benchmark over baseline. The best strategy also transferred to other domains. Several labs sharing one archive beat isolated labs by 13.7 percent relative. Collaboration, the old human trick, works for agents too. Shared memory may matter as much as raw model power.
Catalog record
| CREATOR | Samuel Schmidgall, Michael Moor |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18102
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AgentRxiv: Towards Collaborative Autonomous Research Samuel Schmidgall, Michael Moor Abstract Progress in scientific discovery is rarely the result of a single "Eureka" moment, but is rather the product of hundreds of scientists incrementally working together toward a common goal. While existing agent workflows are capable of producing research autonomously, they do so in isolation, without the ability to continuously improve upon prior research results. To address these challenges, we introduce AgentRxiv —a framework that lets LLM agent laboratories upload and retrieve reports from a shared preprint server in order to collaborate, share insights, and iteratively build on each other’s research. We task agent laboratories to develop new reasoning and prompting techniques and find that agents with access to their prior research achieve higher performance improvements compared to agents operating in isolation (11.4% relative improvement over baseline on MATH-500). We find that the best performing strategy generalizes to benchmarks in other domains (improving on average by 3.3%). Multiple agent laboratories sharing research through AgentRxiv are able to work together towards a common goal, progressing more rapidly than isolated laboratories, achieving higher overall accuracy (13.7% relative improvement over baseline on MATH-500). These findings suggest that autonomous agents may play a role in designing future AI systems alongside humans. We hope that AgentRxiv allows agents to collaborate toward research goals and enables researchers to accelerate discovery. AgentRxiv.github.io Figure 1 : Collaborative Autonomous Research via AgentRxiv. Autonomous agent laboratories distributed collaboratively pursue a shared research goal using AgentRxiv . Human researchers provide initial guidance through a research direction and detailed instructions. Agents autonomously perform research and upload research papers to the centralized AgentRxiv preprint server, enabling laboratories to access each other’s discoveries, accelerating scientific progress. 1 Introduction Scientific discovery has historically been an iterative process, characterized by the systematic formulation of hypotheses, the execution of controlled experiments, and the evaluation of results ( Chalmers 2013 ). Over time, these methods lead to the steady accumulation of knowledge, forming the basis upon which further inquiries are built ( Shapere 1964 ). In this manner, scientific progress does not typically arise from isolated breakthroughs but rather from incremental improvements that collectively advance our understanding of complex phenomena. In an effort to accelerate the process of scientific discovery, recent work has explored the ability of LLM agents to perform autonomous research ( Schmidgall et al. 2025 ; Swanson et al. 2024 ; Lu et al. 2024b ). The AI Scientist framework ( Lu et al. 2024b ) is a large language model (LLM)-based system that generates research ideas in machine learning, writes research code, run experiments, and produces a scientific paper with using an automated peer-review to evaluate the work. Virtual Lab ( Swanson et al. 2024 ) uses a multi-agent system of LLM-based experts from different backgrounds (e.g. chemist or biologist) working together with human scientists to produce novel nanobody binders for SARS-CoV-2, where discovered nanobodies demonstrate promising efficacy in wet-lab validations. Finally, Agent Laboratory ( Schmidgall et al. 2025 ) is a multi-agent autonomous research system that is able to incorporate human feedback, with greatly reduced cost compared to Lu et al. 2024b . While these works demonstrate progress toward accelerated scientific discovery, they often operate in isolation and do not support the continuous, cumulative development of research across time that reflects the nature of science. Therefore, we aim to provide a unified platform that enables agents to build upon the research of other agents. In this study, we introduce AgentRxiv , an autonomous research collaboration framework that supports LLM agents in generating, sharing, and building upon scientific research. By implementing a centralized, open-source preprint server for autonomous agents, AgentRxiv enables systematic sharing of research findings, allowing agents to cumulatively build on previous work. AgentRxiv also supports parallel research across multiple agentic systems, enabling scalability with available computational resources. When incorporating AgentRxiv , each generation of papers shows measurable improvements. For example, accuracy on the MATH-500 benchmark increased from 70.2% to 78.2% with the best discovered reasoning technique, using gpt-4o mini as the base model. Moreover, reasoning strategies developed on MATH-500 are shown to generalize to other tasks and language models, improving performance on benchmarks such as GPQA, MMLU-Pro, and MedQA across models ranging from DeepSeek-v3 to Gemini-2.0 pro. Although the parallelized mode accelerates improvements in wall-clock time, it introduces trade-offs between speed and computational efficiency. A detailed list of our contributions are provided below: • We introduce AgentRxiv —a novel, open-source framework designed to archive and disseminate research outputs from autonomous agents. This platform enables agents to build upon the discoveries of other agents, driving iterative improvements over time. • We show that each generation of papers produces measurable improvements when agents are given access to AgentRxiv . For instance, accuracy on the MATH-500 benchmark steadily increased from a 70.2% baseline to 78.2% (11.4% relative improvement) using newly discovered techniques like Simultaneous Divergence Averaging (SDA). • We demonstrate that reasoning strategies discovered through AgentRxiv on MATH-500 generalize to other benchmarks and language models. Using the highest performing discovered reasoning method by AgentRxiv , our experiments show performance increase across a range of tasks—from GPQA and MMLU-Pro to MedQA—while also revealing consistent gains across five different language models, from DeepSeek-v3 to Gemini-2.0 pro (improving on average by 3.3%). • We introduce a parallelized mode for AgentRxiv , allowing multiple agentic systems to run simultaneously and share findings. We show that this setup accelerates improvements on MATH-500 by +6.0% with 3 parallel labs. We also find that there are a trade-offs between speed and computational efficiency, with discoveries occurring faster but at a higher computational cost. 2 Background and Related Work Large language models. Large language models (LLMs) typically employ transformer architectures ( Vaswani 2017 ), which rely on self-attention to capture long-range dependencies in text without using recurrent ( Rumelhart et al. 1985 ; Jordan 1997 ) or convolutional layers ( Fukushima 1980 ; LeCun et al. 1998 ). These models can contain hundreds of millions or even billions of parameters, often trained on large-scale corpora drawn from diverse sources such as web pages, books, and online forums ( Radford et al. 2018 ; Brown 2020 ; Radford et al. 2019 ; Devlin et al. 2019 ). During training, LLMs are often trained to model statistical regularities in the data by predicting the next token in a sequence, which enables them to generate coherent text, answer questions, and support a wide variety of language understanding tasks. Recent reasoning models—such as OpenAI’s o1 ( Jaech et al. 2024 ), o3-mini ( OpenAI 2025 ), and DeepSeek’s R1 ( Guo et al. 2025 )—extend the paradigm of next-token prediction by incorporating chain-of-thought prompting ( Wei et al. 2022 ) and reinforcement learning techniques ( Shao et al. 2024 ) to support multi-step problem solving. LLM Agents. LLM Agents expand the capabilities of LLMs by integrating structured workflows in order to autonomously perform task execution ( Zhang et al. 2024a ; Wu et al. 2023 ; Yao et al. 2023b ; Yang et al. 2024a ; Lu et al. 2023 ; Zhuge et al. 2024 ). Rather than only generating text outputs, these agents interact with the external environment, using techniques such as reasoning ( Kojima et al.