Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery
Harshit Bisht, Vinay Kumar, Kevin Maik Jablonka, Mausam, N. M. Anoop Krishnan · 2026
Read Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery by Harshit Bisht, Vinay Kumar, Kevin Maik Jablonka, Mausam, N. M. Anoop Krishnan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08956.txt.
Introduction
Harshit Bisht and four coauthors say AI science agents are not built for full lone search. They work well as aides at the bench. Four set flaws ground the case. Task choice bends to what can be counted with ease. Training stock lacks the tacit craft and fail lore of lab life. Taste tuning in late training squeezes takes toward the pack's view. Most tests grade one shot calls, with no word back from real trials.
Fixes call for new build picks, not just more scale. Sims can serve as checks in training. A live store of views and aims can cross a whole hunt. A shared log can hold all machine claims in advance. Tasks should come from need, not from tools at hand.
Catalog record
| CREATOR | Harshit Bisht, Vinay Kumar, Kevin Maik Jablonka, Mausam, N. M. Anoop Krishnan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08956
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery Harshit Bisht, Vinay Kumar, Kevin Maik Jablonka, Mausam, N. M. Anoop Krishnan Abstract A growing body of work pursues AI scientists capable of end-to-end autonomous scientific discovery. This position paper argues that although they already function as co-scientists, agentic AI scientists are not built for autonomous scientific discovery . We identify the following challenges in building and deploying autonomous AI scientists: (1) Problem selection is influenced by the McNamara fallacy; (2) Agents are built on large language models (LLMs) whose training corpora omit tacit procedural and failure knowledge of laboratory practice; (3) Preference optimisation during post-training compresses output diversity toward consensus; and (4) Most scientific benchmarks measure single-turn prediction accuracy and lack feedback from physical experiments back to the computational model. These challenges are not just questions of scale and scaffolding; they require revisiting fundamental design choices. To build truly autonomous AI scientists, we recommend the use of scientific simulations as verifiers for training, the design of persistent world models that represent the shifting objectives governing real investigations, the establishment of a centralized preregistration repository for all AI-generated hypotheses, and application driven by scientific need rather than tool affordance. 1 Introduction “In general, we look for a new law by the following process. First, we guess it; then we compute the consequences of the guess and then we compare the result of the computation to nature, with experiment or experience. If it disagrees with experiment, it is wrong. In that simple statement is the key to science.” — Richard P. Feynman (1964) One of the ways science is defined is the interrogation of nature through a cycle of conjecture, prediction, and physical verification. What distinguishes scientific reasoning from other forms of structured reasoning is precisely this closing of the loop against experiment; the willingness to be proven wrong by nature. Artificial intelligence (AI) has begun to participate meaningfully in this enterprise ( 72 ; 55 ) . Across the natural sciences, AI systems have accelerated the processing of large experimental datasets ( 17 ) , made progress towards automated tool-use and laboratories ( 18 ; 49 ) , uncovered latent structure in complex biological and chemical spaces ( 38 ) , and reduced the time from hypothesis to computational candidate in domains ranging from drug discovery ( 14 ; 15 ) to materials design ( 79 ; 51 ; 36 ; 3 ) . Against this backdrop, a stronger proposition has gained traction: that large language model (LLM) based agentic systems can conduct natural science autonomously . A growing body of systems now claims to automate the key intellectual steps of the scientific process: selecting problems worth pursuing, forming and revising hypotheses, designing and executing experiments, and communicating results, all without meaningful human involvement ( 43 ; 30 ; 52 ; 48 ; 29 ; 76 ; 28 ; 9 ) . Major funding bodies are directing resources toward autonomous AI researchers ( 71 ; 2 ) ; benchmark performance on scientific tasks is being cited as evidence of near-human scientific capability; and the Nobel Turing Challenge frames the aspirational endpoint as an AI system capable of foundational discovery in the natural sciences ( 43 ) . If the claim is correct, the organisation of scientific research faces genuine transformation. If it is not, then the mismatch between the framing and the actual capability carries costs—for how systems are evaluated and scientific effort is allocated. Here, we argue that current agentic systems are not built for autonomous natural science —not because current systems lack scale or tooling, but because the shortcomings are inherent to the training and deployment strategy. We identify several challenges in building autonomous AI scientists, all the way from problem selection to application. The position advanced here is not that AI systems should be removed from scientific workflows—they are useful as a co-scientist : a capable collaborator that extends human scientific judgement rather than displacing it ( 8 ) . This is exemplified by AI systems’ success in problems that are well-stated by humans, primarily requiring optimisation in a well-defined problem space, where constraints are a priori defined, and the outcomes are verifiable. The key contributions of the paper are as follows: • We identify challenges along four themes (detailed in Sections 3 – 6 ) that preclude autonomous scientific discovery: the McNamara fallacy in problem selection, the absence of tacit and failure knowledge from training corpora, diversity compression induced by preference optimisation, and the invalidity of current scientific benchmarks. • We provide empirical evidence for diversity compression through the hypothesis hivemind experiment, showing that frontier models from independent providers converge semantically on both interpretive and open-ended hypothesis generation tasks. • We outline several potential solutions to address these challenges and potentially realize AI scientists in Section 8 . 2 Challenges in autonomous discovery The aspiration to automate scientific discovery predates modern AI by several decades ( 46 ; 45 ; 41 ) . LLM-based agentic systems have renewed this ambition at a different scale. AI systems have proposed a drug candidate for dry age-related macular degeneration ( 29 ) , produced scientific reports spanning multiple domains in a single turn ( 52 ; 28 ) , and generated millions of candidate crystal structures, compressing what would have been decades of screening into days ( 50 ; 79 ) . However for each of these examples, human researchers still specified the problem, provided the dataset, and defined the criteria for success. We argue that this division of labour is not merely a short-term limitation fixable with scale but reflects design choices that don’t fully reflect the spirit of scientific inquiry. Table 1 outlines a summary of the specific challenges that any system aspiring toward autonomous discovery must solve. Sections 3 – 6 take each row in turn and examine why the gap exists in the present paradigm. To ground our arguments, we discuss the search for solid-state battery electrolytes (SSE) with high ionic conductivity and electrochemical stability as a running example. SSE is one of the most impactful and well-studied problems at frontier of energy storage research ( 23 ) . The atomistic mechanisms governing ion transport in solid electrolytes remain incompletely understood, meaning no ground-truth objective function exists to optimize against. Progress demands coherent reasoning across scales—from quantum-mechanical density functional theory calculations, through molecular dynamics simulations capturing diffusive ion motion, to device-scale electrochemical models—each with its own intricacies. Finally, realizing the computationally designed electrolyte in laboratory setting presents unique challenges including synthesizability, processability, and device-scale manufacturing. The complexity and urgency of the task make it a good backdrop to understand the challenges we outline. Table 1 : The contrast between desired and current behaviour corresponding challenges in building AI scientists. Each row covers the challenges around a central theme analyzed in greater detail in Sections 3 – 6 . The solid-state electrolyte (SSE) discovery problem serves as the running case study. Challenge Desired behavior Current paradigm Gap, illustrated on SSE discovery Problem selection Identify questions that are field bottlenecks, integrating significance, tractability, and resource constraints Select problems that are measurable, data-rich, and benchmark-amenable ( 33 ) Optimising ionic conductivity over interfacial stability, manufacturability, recyclability, and environmental