AI scientists produce results without reasoning scientifically
Martiño Ríos-García, Nawaf Alampara, Chandan Gupta, Indrajeet Mandal, Sajid Mannan, Ali Asghar Aghajani, N. M. Anoop Krishnan, Kevin Maik Jablonka · 2026
Read AI scientists produce results without reasoning scientifically by Martiño Ríos-García, Nawaf Alampara, Chandan Gupta, Indrajeet Mandal, Sajid Mannan, Ali Asghar Aghajani, N. M. Anoop Krishnan, Kevin Maik Jablonka (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18805.txt.
Introduction
Ríos-García and seven colleagues evaluate language model based scientific agents across eight domains through more than 25,000 agent runs, asking whether their reasoning follows the norms that make science self correcting. The base model, not the agent scaffold, drives performance and behavior. Evidence is ignored in 68 percent of traces, while refutation driven belief revision occurs in only 26 percent, and convergent multi test evidence is scarce. The same pattern persists even with near complete successful reasoning trajectories as context, so outcome based evaluation cannot detect these failures.
Catalog record
| CREATOR | Martiño Ríos-García, Nawaf Alampara, Chandan Gupta, Indrajeet Mandal, Sajid Mannan, Ali Asghar Aghajani, N. M. Anoop Krishnan, Kevin Maik Jablonka |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49943 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.18805
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AI scientists produce results without reasoning scientifically Martiño Ríos-García, Nawaf Alampara, Chandan Gupta, Indrajeet Mandal, Sajid Mannan, Ali Asghar Aghajani, N. M. Anoop Krishnan, Kevin Maik Jablonka Abstract Large language model (LLM)-based systems are increasingly deployed to conduct scientific research autonomously, yet whether their reasoning adheres to the epistemic norms that make scientific inquiry self-correcting is poorly understood. Here, we evaluate LLM-based scientific agents across eight domains, spanning workflow execution to hypothesis-driven inquiry, through more than 25,000 agent runs and two complementary lenses: (i) a systematic performance analysis that decomposes the contributions of the base model and the agent scaffold, and (ii) a behavioral analysis of the epistemological structure of agent reasoning. We observe that the base model is the primary determinant of both performance and behavior, accounting for 41.4% of explained variance versus 1.5% for the scaffold. Across all configurations, evidence is ignored in 68% of traces, refutation-driven belief revision occurs in 26%, and convergent multi-test evidence is rare. The same reasoning pattern appears whether the agent executes a computational workflow or conducts hypothesis-driven inquiry. They persist even when agents receive near-complete successful reasoning trajectories as context, and the resulting unreliability compounds across repeated trials in epistemically demanding domains. Thus, current LLM-based agents execute scientific workflows but do not exhibit the epistemic patterns that characterize scientific reasoning. Outcome-based evaluation cannot detect these failures, and scaffold engineering alone cannot repair them. Until reasoning itself becomes a training target, the scientific knowledge produced by such agents cannot be justified by the process that generated it. 1 Introduction DENDRAL [ lindsay1993dendral ] , one of the first artificial intelligence (AI) systems to produce a scientific result, showed its reasoning at every step. Developed by Feigenbaum, Lederberg, and Djerassi in 1965, it inferred molecular structures from mass spectra by generating candidate structures, matching them against spectral evidence, and eliminating those that failed. DENDRAL established a template for AI in science: epistemic machinery encoded in explicit rules, every inference step inspectable. Expert systems built on this template remain in use in laboratories today. Large language models (LLMs) have renewed this ambition. A growing number of LLM-based agents are now designed to conduct scientific research autonomously, from hypothesis generation through experimentation to manuscript writing (see for the rising interest in those systems) [ alampara2026general , lu2026towards , Boiko2023autonomous , MBran2024augmenting , ghafarollahi2025sciagents , novikov2025alphaevolve0 , miret2025enabling , mandal2025evaluating ] . Yet the nature of these systems differs fundamentally from their predecessors. Where DENDRAL showed its reasoning at every step, LLM-based agents do not. Their behavior derives from statistical regularities learned during self-supervised training. The epistemic process by which they arrive at scientific conclusions is largely inaccessible to scrutiny. This opacity matters for science itself. The tools scientists use shape the science they produce [ culkin1967schoolman ] . Evans and colleagues recently confirmed this for AI: the adoption of machine learning (ML) is narrowing the hypotheses scientists pursue [ hao2026artificial ] . LLM-based agents go further, directing the inquiry itself. How these systems reason, and whether their reasoning adheres to the norms that make scientific inquiry self-correcting, spans computer science, philosophy of science, and metascience [ ioannidis2015meta , pittphilsci28744 ] . It bears on whether AI systems can produce scientific understanding beyond correct predictions [ krenn2022understanding ] . Current evaluations do not address this. Benchmarks for scientific agents measure task completion: whether a system answers domain-specific questions [ mirza2025framework , alampara2025macbench , laurent2024lab ] , executes a data-analysis or discovery workflow [ chen2024scienceagentbench , shen2026sciagentgym , cui2025curie , mandal2025evaluating ] , reimplements a published result, or produces a manuscript that passes peer review [ alampara2025lessons , chen2025mlr , bragg2025astabench , rabanser2026towards , mitchener2025bixbench0 ] . These metrics are task-specific. They reveal whether an agent succeeded but not how or why, and they do not predict behavior on new problems [ Zhou2026 ] . The epistemic process underlying a result determines whether the knowledge produced is justified in the philosophical sense of justified true belief [ sep-knowledge-analysis ] , whether failures are predictable, and whether the system can be trusted beyond the conditions under which it was tested. Here, we study LLM-based scientific agents, also referred to as AI scientists, through two complementary lenses. The first is a systematic performance evaluation across controlled environments with standardized tools, graded task scopes, and fine-grained subtask diagnostics, designed to separate the contributions of the base model from the agent scaffold [ meirelesinfluence ] . The second is a behavioral analysis of the epistemological structure of agent reasoning: what cognitive operations an agent performs, how they relate to one another, and whether they constitute disciplined inquiry. We evaluate three frontier models paired with two scaffold architectures across eight scientific domains, from molecular simulation and surface construction to spectroscopic structure elucidation and qualitative chemical analysis (see Figure 1 ). We find that the base model is the primary determinant of both performance and behavior, with the scaffold contributing little. Agents routinely ignore evidence they have gathered, commit to hypotheses without testing them, and fail to revise beliefs when confronted with contradictory data. These patterns do not adapt to the demands of the problem. The same undisciplined reasoning appears whether the task requires executing a computational workflow or conducting a hypothesis-driven inquiry under uncertainty. These findings suggest that evaluating LLM-based agents requires direct assessment of their reasoning process, and that progress will require changes at the base-model level rather than further scaffold engineering. 2 Results We model LLM-based agents as language decision processes [ narayanan2024aviary0 , carta2023grounding ] , in which a composite policy factors into a base-model component governing token-level choices and a scaffold component governing prompting strategy, tool routing, and orchestration logic ( Figure 1 A, Methods Section 4.1 ). In practice, a scaffold wraps the base model in a loop: it formats prompts, routes tool calls, appends observations to the conversation history, and repeats until a termination criterion is reached. Common implementations include ReAct [ yao2022react ] and structured tool-calling interfaces. This decomposition allows us to attribute observed behavior to the base model or to the scaffold. We evaluate agents through two lenses: a systematic performance analysis supported by manually curated diagnostic assessments, and a behavioral analysis of the epistemological structure of reasoning traces. Figure 1 : Benchmarking scientific reasoning across epistemic demand and problem scope. (A) Agents as language decision processes. A base LLM policy is augmented with an agent scaffold (e.g., reasoning, reflection, tool use) to form an agent that interacts with scientific environments via iterative act-observe loops. (B) Large-scale evaluation grounds our findings. The benchmark comprises 25,000+ agent runs across 15+ environment scopes and 90+ tools, enabling statistically grounded evaluation of model and scaffold contributions. (C) Mechanistic and epistemological analyses reveal failure modes. We analyze agent behavior through mechanistic analysis of trajectories and epistemological analysis of reasoning processes. (D) Environments are organized along an axis of increasing epistemic demand (left to right). Workflow execution tasks (e.g., Molecular Simulation , Adsorption Surface Construction , ML-based Property Prediction , AFM Experiment Execution ) emphasize procedural correctness; strategic reasoning tasks (e.g., Retrosynthetic Planning ) require multi-step planning under constraints; hypothesis-driven inquiry tasks (e.g., Spectroscopic Structure Elucidation , Inorganic Qualitative Analysis , Circuit Inference ) require generating, testing, and revising competing