LLM Agents for Knowledge Discovery in Atomic Layer Processing
Andreas Werbrouck, Marshall B. Lindsay, Matthew Maschmann, Matthias J. Young · 2025
Read LLM Agents for Knowledge Discovery in Atomic Layer Processing by Andreas Werbrouck, Marshall B. Lindsay, Matthew Maschmann, Matthias J. Young (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-26201.txt.
Introduction
Andreas Werbrouck and co-authors send language model agents to explore a machine on their own in a 2025 paper. Process tuning follows orders, but knowledge discovery means roaming freely, posing claims about a system, and checking them.
The agents were given tools to probe a black box function and nothing else. A children's parlor game served as the proving ground, showing how trial, error, and persistence shape what gets found, and how strongly the path taken steers the result. The same setup was then pointed at a simulation of an atomic layer processing reactor, a tool that builds films one atom layer at a time. With deliberately weak probes, the agents found and used varied chemical behaviors.
Catalog record
| CREATOR | Andreas Werbrouck, Marshall B. Lindsay, Matthew Maschmann, Matthias J. Young |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-26201
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLM Agents for Knowledge Discovery in Atomic Layer Processing Andreas Werbrouck, Marshall B. Lindsay, Matthew Maschmann, Matthias J. Young Abstract Large Language Models (LLMs) have garnered significant attention for several years now. Recently, their use as independently reasoning agents has been proposed. In this work, we test the potential of such agents for knowledge discovery in materials science. We repurpose LangGraph’s tool functionality to supply agents with a black box function to interrogate. In contrast to process optimization or performing specific, user-defined tasks, knowledge discovery consists of freely exploring the system, posing and verifying statements about the behavior of this black box, with the sole objective of generating and verifying generalizable statements. We provide proof of concept for this approach through a children’s parlor game, demonstrating the role of trial-and-error and persistence in knowledge discovery, and the strong path-dependence of results. We then apply the same strategy to show that LLM agents can explore, discover, and exploit diverse chemical interactions in an advanced Atomic Layer Processing reactor simulation using intentionally limited probe capabilities without explicit instructions. 1 Introduction Large Language Models (LLMs) and agentic systems have become hard to escape in many professional fields. In materials science, LLM or agent use focuses either on synthesizing, integrating, and validating domain knowledge [ 1 , 2 , 3 , 4 ] , or leverages them as components in self-driving labs for materials design, synthesis, or characterization (either computational or real). [ 5 , 6 , 7 , 8 ] . In most cases, such agents leverage knowledge absorbed during the training phase, are given access to tools and databases, and generally are part of complex, multi-agent workflows designed to reach well-defined objectives (e.g., property optimization). This makes it challenging to separate their ability for knowledge synthesis, discovery of latent knowledge, and discovery of new knowledge [ 9 ] . Figure 1: The goal of probing an experimental system is to discover the rules mapping the input space to the output space. Asking specific questions limits discovery to very specific rules. In materials science, one interpretation of this figure could be to see input space as precursor chemicals, the rules as all possible synthesis pathways, and the output space as desired behavior of the synthesized materials (as this behavior may be synthesis-dependent). We note that benchmarks are typically metric-based, confining the potential output space of agents or LLMs to the correct answer (or something close to it). This encourages incremental, rather than groundbreaking innovation. At the same time, failure should not be problematic, as long as something is learned from it. Consequently, in this work, we want to explore what happens if we flip the script (Figure 1 ). Our goal is to probe the capabilities of LLM agents to interrogate a system and discover completely new knowledge [ 10 ] . Leveraging tool capabilities, we give the LLM agent a specific system to work with and ask it to provide statements about the rules governing this system without defining any other objective . This system and its rules confine the problem in a different way than a question does: we provide confined input and output spaces, leaving room for the agent to experiment and find the hidden rules. We demonstrate this through two examples. First, we play a parlor game with the agent: it is stuck on an extragalactic planet and needs to figure out the rules of the alien market through trial and error. In the second example, we give the agent access to a simulated chemical reactor. This reactor is set up with four fictional reagents A-D, which behave semi-realistically with respect to transport and temperature. Custom self-limiting and non-self-limiting interactions between chemicals and surfaces are simulated to emulate Atomic Layer Processing (ALP) chemistries (see appendix A for background). Dedicated pressure and mass change sensors provide the agent with limited, but sufficient probes into the system state, much like a human scientist would not have access to all reactor variables. 2 Methods Figure 2: a) graph for the first example (Alien market) b) graph for the second example (ALP reactor) LLM-based components are shown in blue, while deterministic components are grey. c) schematic reactor layout. The simple ReACT agent graph for the Alien market example is shown in Figure 2 a. The structure used for the ALP problem is shown in Figure 2 b, with a schematic of the chemical reactor in Figure 2 c. Specific details about the implementation, prompts and simulation framework can be found in Appendix B , C and D . 3 Results and Discussion Below, we test the capability of agents for exploring and characterizing two different systems (Alien market and ALP reactions). In both cases, we do not provide the agent with any task except to generate concise, relevant, and generalizable statements about the system under study (prompts in Appendix D ). The goal of these tests is to probe the general ability of LLM agents to explore and describe black box functions, generate hypotheses, and plan experiments, rather than testing or relying on previously absorbed (chemical) knowledge. Both tasks minimize required background knowledge: we created custom rules for the Alien Market and use abstract chemicals A-D for the ALP reaction characterization. Figure 3: Initially, most models except for gpt-5 perform poorly at discovering the rules of the Alien market (orange squares). The performance markedly improves upon requesting the agent to perform a defined number of experiments (dark dots). 3.1 Alien market We conducted two different experiments with the agent on the Alien market (Figure 2 a). We compare flagship models from openAI (gpt-5) and Google (gemini-2.5-pro) with medium-sized models (gpt-5-mini and gemini-2.5-flash) and one older medium-sized model (gemini-2.0-flash). Smaller models we tested had significant trouble using the tools and/or structured output, so they are omitted from this report. In the first experiment (orange squares in Figure 3 ), we provide the agent with the background story and instruct it to find the rules of the alien market (prompt in Appendix D ). Scoring is explained in Appendix B . Upon examination of the results, gpt-5 clearly outperforms the other models. However, it does so by conducting significantly more experiments. Other models make some observations, then provide rules that match these observations. This does not signify the superior intelligence of gpt-5; the other models simply stopped their investigation too soon. In the second set of experiments (Figure 3 , dark dots) we explicitly asked the agent to do n experiments, which significantly improved results. We note that pushing models to extend their investigation is a viable strategy to achieve meaningful and generalizable results. Additionally, some rules are intrinsically harder to find than others. This strengthens the case for requiring models to spend more time on a problem. Based on a list of 10,000 English words [ 11 ] , the probability that an item will be rejected by the market (word contains m or p) is 0.3248. The probability for a word to contain p is 0.1822 while the probability for a word to contain m is 0.1761. On the other hand, if the rule had been based on the occurrence of ’z’, the probability of stumbling over it would be only 0.0129. Even