Agents of Discovery
2025
Read Agents of Discovery (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-08535.txt.
Introduction
Teams of language-model agents tackle particle physics data analysis in a 2025 paper by Sascha Diefenbacher and 5 co-authors. Modern fundamental physics produces data volumes that demand complex analysis workflows. Machine learning tools in the field are usually special-purpose algorithms with physics knowledge baked in, powerful but narrow and hard to retool.
The authors explore an orthogonal direction: several language model instances, each assigned specific subtasks, jointly solve analysis problems as a team. Agentic division of labour mirrors how human collaborations split a search into selection, fitting, and validation. If general agents can reach specialised tools' performance, the cost of trying a new analysis idea in physics could fall dramatically.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 124 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-08535
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Agents of Discovery Abstract The substantial data volumes encountered in modern particle physics and other domains of fundamental physics research allow (and require) the use of increasingly complex data analysis tools and workflows. While the use of machine learning (ML) tools for data analysis has recently proliferated, these tools are typically special-purpose algorithms that rely, for example, on encoded physics knowledge to reach optimal performance. In this work, we investigate a new and orthogonal direction: Using recent progress in large language models (LLMs) to create a team of agents — instances of LLMs with specific subtasks — that jointly solve data analysis-based research problems in a way similar to how a human researcher might: by creating code to operate standard tools and libraries (including ML systems) and by building on results of previous iterations. If successful, such agent-based systems could be deployed to automate routine analysis components to counteract the increasing complexity of modern tool chains. To investigate the capabilities of current-generation commercial LLMs, we consider the task of anomaly detection via the publicly available and highly-studied LHC Olympics dataset. Several current models by OpenAI (GPT-4o, o4-mini, GPT-4.1, and GPT-5) are investigated and their stability tested. Overall, we observe the capacity of the agent-based system to solve this data analysis problem. The best agent-created solutions mirror the performance of human state-of-the-art results. Sascha Diefenbacher 1 , Anna Hallin 2 , Gregor Kasieczka 2 , Michael Krämer 3 , Anne Lauscher 4 , Tim Lukas 2 , 1 Physics Division, Lawrence Berkeley National Laboratory, Berkeley, USA 2 Institut für Experimentalphysik, Universität Hamburg, Germany 3 Institute for Theoretical Particle Physics and Cosmology, RWTH Aachen University, Germany 4 Data Science Group, Universität Hamburg, Germany August 10, 2026 1 Introduction Large-scale experiments in high-energy physics such as those at the Large Hadron Collider (LHC) produce increasingly complex datasets that require sophisticated, multi-stage analysis workflows to extract results – including subtle potential signals of new physics – from high-dimensional data representations. Although mature tools and shared computing resources are available, the analysis process itself is becoming more demanding. Workflows must be repeatedly re-optimized for changing detector conditions, calibration procedures must be adjusted and background estimates must be carefully validated. These developments mean that an increasing proportion of scientific output relies on detailed coordination and significant human effort. Without improvements in efficiency, this growing complexity could limit the potential of current and future high-energy physics experiments. Current automation efforts in high-energy physics tend to focus on individual tasks such as reconstruction, calibration, and statistical analysis. However, they provide limited support for coordinating entire analysis workflows. As a result, a significant amount of time is spent on integration and bookkeeping tasks, such as connecting tools, synchronizing configurations, converting formats and tracking parameters. This slows down the iterative process and makes it hard to ensure reproducibility. Pipelines are often closely linked to specific tools or experiments, making reuse across domains difficult and costly. These bottlenecks are not unique to high-energy physics, but also occur in other data-intensive sciences, such as astrophysics and cosmology [ 1 , 2 ] . Recent advances in large language models (LLMs) present a new opportunity. Beyond generating code and assisting with individual tasks, like devising selection cuts for a particular particle physics analysis [ 3 ] , LLMs can be embedded in agent-based systems that plan multi-stage workflows, call standard analysis tools, inspect their output and refine subsequent steps [ 4 , 5 , 6 , 7 , 8 , 9 ] . Importantly, these agents do not replace existing analysis codes, but rather operate them in a transparent and reproducible manner. This design preserves a clear record of what has been run, enabling human researchers to inspect, rerun and validate results. Iterative refinement loops, in which the analysis strategy is adapted based on intermediate results, promise to accelerate discovery while maintaining scientific standards. Agent-based approaches have recently attracted attention across several scientific domains. In chemistry and materials science, for example, frameworks such as ChemCrow [ 10 ] and AutoGPT-Chem [ 11 ] use LLM agents to plan and execute experiments, interface with external tools and suggest new chemical compounds. Across the sciences, systems such as SciAgent [ 12 ] and AI Scientist [ 13 , 14 ] can generate hypotheses, design experiments and analyse results iteratively. In software engineering and applied machine learning, agentic systems have been used to automate code generation [ 15 , 16 ] , while evaluation suites benchmark agent capabilities [ 17 , 18 ] , and AutoML frameworks streamline hyperparameter searches [ 19 ] . These studies demonstrate the potential of agent-based science, but also highlight open questions about robustness, reproducibility, and domain adaptation [ 4 , 5 ] . While [ 20 ] outlines the overall picture of large physics models, to our knowledge, no systematic effort has yet been made to concretely explore such approaches for high-energy physics data analysis (in contrast to, e.g., recent work in accelerator operations [ 21 ] or astronomy [ 22 ] ). In this work, we take a first step towards closing this gap by designing a framework for agent-based analysis in physics. The long-term idea is that role-based agents will coordinate standard analysis steps, connect to existing tools through simple interfaces, and automatically keep track of provenance. The focus here is not on replacing single tasks, but on organizing complete workflows for faster iteration, better reuse and more robust reproducibility. As a starting point, we let one agent carry out a full anomaly-detection analysis end to end, which provides a clear test of current capabilities and at the same time prepares the ground for more complex multi-agent workflows. We evaluate our framework using anomaly detection tasks inspired by the LHC Olympics challenge [ 23 ] . Our study benchmarks different prompting strategies and LLM models, comparing their behaviour across multiple runs and assessing stability, cost and reproducibility. These results are a first step towards agent-based discovery machines in high-energy physics and demonstrate how such systems could facilitate more efficient analysis in current and future experiments. The remainder of the paper is organized as follows. Section 2 introduces the physics problem the agent is tasked with solving and the associated dataset. Section 3 describes the proposed framework: the agents and their roles, their connections to one another and the tools they have at their disposal. This section also presents the chosen LLMs and describes the prompting strategies used. Results are presented in Section 4 , starting with general functionality and performance and continuing with the LHC Olympics studies. Section 5 discusses these results and concludes the paper. The appendices provide the complete lists of metrics and prompts used in this study. 2 Dataset and Task 2.1 Problem In order to compare the output of the agent with that of human researchers, we test the agent’s performance on a dataset from the LHC Olympics (LHCO) anomaly detection challenge [ 23 ] . In this challenge, participants were asked to develop anomaly detection methods using a labeled “R&D” dataset [ 24 ] . This simulated dataset contains background events (labeled 0) and signal events (labeled 1), where the signal represents new particles. The participants were then to apply these methods to