Open Source Planning & Control System with Language Agents for Autonomous Scientific Discovery

Licong Xu, Milind Sarkar, Anto I. Lonappan, Íñigo Zubeldia, Pablo Villanueva-Domingo, Santiago Casas, Christian Fidler, Chetana Amancharla, Ujjwal Tiwari, Adrian Bayer, Chadi Ait Ekioui, Miles Cranmer, Adrian Dimitrov, James Fergusson, Kahaan Gandhi, Sven Krippendorf, Andrew Laverick, Julien Lesgour · 2025

Read Open Source Planning & Control System with Language Agents for Autonomous Scientific Discovery by Licong Xu, Milind Sarkar, Anto I. Lonappan, Íñigo Zubeldia, Pablo Villanueva-Domingo, Santiago Casas, Christian Fidler, Chetana Amancharla, Ujjwal Tiwari, Adrian Bayer, Chadi Ait Ekioui, Miles Cranmer, Adrian Dimitrov, James Fergusson, Kahaan Gandhi, Sven Krippendorf, Andrew Laverick, Julien Lesgour (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-07257.txt.

Introduction

Licong Xu and co-authors give in 2025 a free multi agent plan and run system for lone science search work. Near thirty chat agents split the work: fetch, code, read outs, and peer check. A plan chief steers the flow with no human hand in the loop at all.

Each agent owns one task kind and can run code on the spot. The set clears a doc tor grade sky task: gauge world marks from star blast data. Two test sets score the set past top chat models, in its side's claim. Code, clips, and a cloud post are all free to take.

Catalog record

CREATORLicong Xu, Milind Sarkar, Anto I. Lonappan, Íñigo Zubeldia, Pablo Villanueva-Domingo, Santiago Casas, Christian Fidler, Chetana Amancharla, Ujjwal Tiwari, Adrian Bayer, Chadi Ait Ekioui, Miles Cranmer, Adrian Dimitrov, James Fergusson, Kahaan Gandhi, Sven Krippendorf, Andrew Laverick, Julien Lesgour
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-07257

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Open Source Planning & Control System with Language Agents for Autonomous Scientific Discovery Licong Xu, Milind Sarkar, Anto I. Lonappan, Íñigo Zubeldia, Pablo Villanueva-Domingo, Santiago Casas, Christian Fidler, Chetana Amancharla, Ujjwal Tiwari, Adrian Bayer, Chadi Ait Ekioui, Miles Cranmer, Adrian Dimitrov, James Fergusson, Kahaan Gandhi, Sven Krippendorf, Andrew Laverick, Julien Lesgourgues, Antony Lewis, Thomas Meier, Blake Sherwin, Kristen Surrao, Francisco Villaescusa-Navarro, Chi Wang, Xueqing Xu, Boris Bolliet Abstract We present a multi-agent system for automation of scientific research tasks, cmbagent . The system is formed by about 30 Large Language Model (LLM) agents and implements a Planning & Control strategy to orchestrate the agentic workflow, with no human-in-the-loop at any point. Each agent specializes in a different task (performing retrieval on scientific papers and codebases, writing code, interpreting results, critiquing the output of other agents) and the system is able to execute code locally. We successfully apply cmbagent to carry out a PhD level cosmology task (the measurement of cosmological parameters using supernova data) and evaluate its performance on two benchmark sets, finding superior performance over state-of-the-art LLMs. The source code is available on GitHub 1 1 1 https://github.com/CMBAgents/cmbagent , demonstration videos are also available 2 2 2 https://www.youtube.com/@cmbagent , and the system is deployed on HuggingFace 3 3 3 https://huggingface.co/spaces/astropilot-ai/cmbagent and will be available on the cloud 4 4 4 https://cmbagent.cloud/ . † † affiliationnotice: Equal contribution 1 Introduction Rapid progress in the development of Large Language Models (LLMs) is enabling new approaches to scientific research (see, e.g., Lu et al. 2024 ; Ghareeb et al. 2025 , and references therein) . This includes quantitative disciplines grounded in numerical data, such as physics, chemistry, biology, and economics, at both fundamental and applied levels. One promising application of LLMs in astrophysics and cosmology is their use as autonomous agents to support and automate data analysis workflows. In Laverick et al. 2024 , it was demonstrated how LLM agents can be used to solve a state-of-the-art data analysis task in cosmology, measuring the values of fundamental parameters that describe the Universe using a novel dataset ( Madhavacheril et al. 2024 ) . They introduced a multi-agent system composed of several types of LLM agents specializing in different tasks (coding, retrieving information from scientific papers, using domain-specific software libraries). This multi-agent system operated with a human-in-the-loop at every step: each LLM response was reviewed by the user, who then provided guidance for the next action. Although such systems can be used as effective research assistant tools, they are limited by their reliance on continuous human input. In this work, we introduce cmbagent , a multi-agent system which can carry end-to-end research tasks with no human-in-the-loop at any point. To enable full automation, cmbagent employs a robotics-inspired Planning & Control strategy, with an agentic framework powered by AG2 ( Wu et al. 2023 ; Wang et al. 2024 ) . We first describe the system architecture and usage (Sec. 2 ) and then present a series of evaluations of the system demonstrating the system’s effectiveness in cosmological applications and beyond (Sec. 3 ), before concluding (Sec. 4 ). 2 Description of the system 2.1 Planning & Control Strategy The Planning & Control strategy in cmbagent is as follows. Given an input task, the system first goes through a Planning phase in which a plan is designed and approved. A Control phase follows, in which the plan is executed by the Control agents. The agents involved in both phases are LLMs and there is no human-in-the-loop at any point in either phase. The full Planning & Control strategy , which is described next, is illustrated in Fig. 1 . Planning phase. The input of the Planning phase is the Main Task , which is specified by the user and is typically a quantitative research task. There are two main agents in this phase: the planner , which proposes a plan, and the plan reviewer , which provides feedback on the proposed plan. The Planning phase starts with the planner proposing a first version of the plan. The plan is formed by a succession of a maximum of n_{\mathrm{steps}} steps, with each step consisting of: (i) a sub-task, (ii) a set of actions to be taken to carry out the sub-task, and (iii) an agent in charge of completing these actions. The proposed plan is then reviewed by the plan reviewer , which provides feedback on the proposed plan. This feedback is passed on to the planner , which proposes an updated plan. This planner – plan reviewer loop takes place n_{\mathrm{reviews}} times. The hyperparameters n_{\mathrm{steps}} and n_{\mathrm{reviews}} can be specified by the user. To improve performance, the output of both agents is structured by formatting agents whose sole role is to structure the text in a specific format. Also, for traceability, the structured responses of each agent (the plans and the plan feedback) are recorded into the session context by recorder agents ( plan recorder and review recorder ). Once the definitive plan is approved by the reviewer , it is stored into context and saved as a JSON file, and the total cost of the phase (in USD) is displayed and stored. This brings the Planning phase to completion and Control phase starts. Control phase. The input of the Control phase is the final plan approved in the Planning phase. The key agent of the Control phase is the controller , which distributes the assigned sub-tasks in the plan to the relevant agents. These sub-tasks are carried out by two main agents: (i) a researcher for reasoning, interpretation, and summarizing tasks; and (ii) an engineer for coding tasks (by default, in Python). When the researcher is queried, its output is formatted in Markdown by a formatting agent and then saved by an executor agent so the other agents can access it. On the other hand, when the engineer is called, a nested chat is triggered, of which only the output is kept. This nested chat is a sequence of two agents: (i) a formatting agent that structures the code written by the engineer and provides minor adjustments (e.g., formatting issues, plot labels), and (ii) an executor agent that executes the provided code block locally. The execution output is then forwarded to a post-execution interpreter agent, which decides on which agent to transfer to. A successful code execution typically triggers a transition to the control agent, which updates the necessary context variables and proceeds with the next step of the plan. On the other hand, a failed code execution may trigger different transitions. If the number of failed execution is less than n_{\mathrm{fails}} (a hyperparameter that can be specified by the user), the subsystem typically transits back to the engineer for another attempt, with suggestions provided by the interpreter . If the failure is caused by a missing Python package, the subsystem typically transits to an installer agent that runs a bash pip install command before the execution is attempted again. If the number of failed executions reaches n_{\mathrm{fails}} , the system transitions to a terminator agent and the full session ends. Upon termination of each sub-task, the cost is displayed and the context variables that have been updated by the agents are recorded

Read the complete text