Toward Evaluation Frameworks for Multi-Agent Scientific AI Systems
Marcin Abram · 2026
Read Toward Evaluation Frameworks for Multi-Agent Scientific AI Systems by Marcin Abram (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26718.txt.
Introduction
Abram analyses how to evaluate multi-agent scientific AI systems fairly. Such systems blur the line between reasoning and retrieval, and face contamination risks in their test data. Novel research questions often lack a trusted ground truth, while tool use and shifting knowledge bases complicate repeats. He discusses building tasks that resist contamination and scale in families. Multi-turn tests that mirror real research practice get special weight. A pilot dataset of fresh research ideas, plus interviews with quantum scientists, grounds the proposals.
Catalog record
| CREATOR | Marcin Abram |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41052 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.26718
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Toward Evaluation Frameworks for Multi-Agent Scientific AI Systems Marcin Abram Abstract We analyze the challenges of benchmarking scientific (multi)-agentic systems, including the difficulty of distinguishing reasoning from retrieval, the risks of data/model contamination, the lack of reliable ground truth for novel research problems, the complications introduced by tool use, and the replication challenges due to the continuously changing/updating knowledge base. We discuss strategies for constructing contamination-resistant problems, generating scalable families of tasks, and the need for evaluating systems through multi-turn interactions that better reflect real scientific practice. As an early feasibility test, we demonstrate how to construct a dataset of novel research ideas to test the out-of-sample performance of our system. We also discuss the results of interviews with several researchers and engineers working in quantum science. Through those interviews, we examine how scientists expect to interact with AI systems and how these expectations should shape evaluation methods. I Introduction We consider a class of scientific AI systems that integrate large language models with external tools, simulation capabilities, and iterative reasoning. Such systems have been proposed in various forms in recent literature and industry prototypes 1 ; 12 ; 18 ; 11 . Here, we will focus on workflows supporting (or augmenting) the work of physicists in such areas as condensed matter physics and quantum information. The central question we explore in this work is how such a system should be tested 4 . Existing benchmarks for language models primarily evaluate narrow capabilities in isolation, such as retrieval accuracy, mathematical problem solving, or code generation. These abilities are certainly important. However, they do not capture the core activities that define scientific work. A physicist (or a scientist in general) is not simply someone who searches for papers, performs mathematical reasoning, or writes code. The essence of scientific practice lies in formulating hypotheses, critically evaluating assumptions, identifying mistakes, synthesizing ideas across sources, and developing intuition about which questions are meaningful and which approaches are promising. Designing systems that can assist in these activities raises a deeper question. Concepts such as physical intuition, critical reasoning, and hypothesis generation are difficult to formalize and, therefore, difficult to benchmark. Yet without some operational definition of these abilities, it is impossible to evaluate whether an AI system is genuinely contributing to scientific discovery or merely reproducing existing knowledge. In this report, we attempt to systematize this problem. Our goal is not to claim that there exists a perfect way to measure scientific capabilities ; not even that there exists a single metric that can capture those; but rather to explore how practical tests can be designed to measure some of the non-trivial aspects of a scientific AI system. These include, among others, the ability to generate novel hypotheses, detect mistakes in existing arguments, synthesize information from multiple sources, and reason about partially specified or unfamiliar physical scenarios. In the following sections, we discuss the conceptual challenges of benchmarking scientific (multi-)agentic systems and propose a set of evaluation strategies to probe selected aspects of those systems. The emphasis throughout the work is on designing tests that go beyond simple retrieval or computation and instead assess aspects of reasoning central to the practice of science. II The Benchmarking Problem Benchmarking (multi-)agentic systems is fundamentally harder than benchmarking a single language model. The first challenge comes from having many interacting elements. We can test the tool selection capability, the performance of specific tools, the LLM’s ability to digest and interpret questions, its capacity to synthesize answers from tool outputs, the orchestration layer that ties everything together, and many other. The second challenge is that the system has access to a constantly updated knowledge base and can search the internet. If we propose any problem that has already been solved (say, proving the no-cloning theorem), we will likely only measure the system’s retrieval and synthesis ability, not its ability to actually solve problems. If we pose a problem that has not yet been solved, we may not have a ground truth to verify whether the answer is correct. Note a well-designed agentic system should use retrieval when appropriate (the ability to recognize that a solution to a given problem exists and to choose to retrieve it rather than attempt to solve it from scratch is a feature, not a bug). Testing retrieval seems easier (we can simply compare the output to the source). Testing reasoning seems more challenging. One of the major concerns is data contamination. The LLM cutoff date is not enough. We need to control both channels: training data contamination and retrieval contamination (we might need to mask identifying information or restrict web access during testing). Even then, information about a specific paper can be distributed across Reddit posts, lecture slides, and transcripts of academic talks. To ensure that we limit data contamination, we may need to restrict ourselves to newly published data (e.g., new arXiv papers). There is also a temporal confound to this problem. Papers published in different months may differ in difficulty, topic distribution, or quality. Differences between months 6 and 12 might reflect changes in arXiv content rather than improvements in the system. Another approach is to use (or create) novel problems. However, if we pose a problem that has not yet been solved, we may not have ground truth to verify whether the answer is correct. We are faced with two uncomfortable options: known problems (where retrieval contaminates the benchmark) and unknown problems (where we cannot verify the results). Nevertheless, there might be a path forward. We can restrict ourselves to classes of problems that can be programmatically-generated, that are hard to solve but easy to verify, and whose difficulty (or complexity) can be scaled (controlled by some parameters). Later in this work, we discuss some ideas that have those properties. An additional challenge is how to score open-ended problems. It is hard (if not impossible) to assign a single score to “how good is this theoretical explanation.” We need to evaluate multiple dimensions, including mathematical correctness, physical plausibility, logical clarity, novelty, specificity, and acknowledgment of limitations. This makes the final metric multi-dimensional. III Benchmark Design Principles Below, we discuss some of the desirable properties of a good benchmark. For example, results should be automatically checkable, at least in principle. For numerical results, this means comparing against known values. For analytical derivations, it means checking step validity (ideally through proof tooling). For constructed objects (a stabilizer code, a counterexample, a quantum state), this means verifying that they have the claimed properties. Problems should avoid retrieval shortcuts. The strongest approaches are planted-structure problems (in which we engineer the answer), constraint-modified problems (in which non-standard modifications of an original problem render straightforward lookup futile), and problems constructed from very recent arXiv papers (novel work not yet discussed elsewhere). We discuss those ideas in much more details in the following sections of this work. Benchmarks should enable difficulty scaling via a continuous parameter, yielding a scaling curve rather than a single accuracy value. Such a scaling curve is far more informative, as it might reveal some interesting breaking points of the systems, like e.g., in 16 . Tasks should resemble real scientific workflows. Researchers do not solve problems in a single shot; they iterate, consult