DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience

Runxiang Wang, Boxiao Wang, Yifan Zhang, Jian Cheng · 2025

Read DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience by Runxiang Wang, Boxiao Wang, Yifan Zhang, Jian Cheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04282.txt.

Introduction

Runxiang Wang and co-authors build a system that discovers scientific equations from data by reasoning twice: once over the data and once over its own past attempts, in a paper from 2025. Finding a compact formula that fits measurements is a classic goal across science. Language models are good at proposing formulas but tend to lean on memorized patterns and skip careful reading of the numbers.

Their framework first writes structured notes about the data: which way variables trend, where curves bend, what correlates with what. It then watches how each proposed equation performs and reflects on the misses. That dual loop steadies the search and improves the odds of landing on the true law. Tests target robustness across varied scientific data sets.

Catalog record

CREATORRunxiang Wang, Boxiao Wang, Yifan Zhang, Jian Cheng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-04282

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience Thanks: Corresponding authors. Runxiang Wang, Boxiao Wang, Yifan Zhang, Jian Cheng Abstract Symbolic regression is a fundamental tool for discovering interpretable mathematical expressions from data, with broad applications across scientific and engineering domains. Recently, large language models (LLMs) have demonstrated strong performance in this task, leveraging embedded scientific priors and reasoning capabilities to surpass traditional methods. However, existing LLM-based approaches, such as LLM-SR, often over-rely on internal priors, lacking explicit data understanding and systematic reflection during equation generation. To address these limitations, we propose DrSR (Dual Reasoning Symbolic Regression), a framework that combines data-driven insight with reflective learning to enhance both robustness and discovery capability. Specifically, DrSR guides LLMs to analyze structural relationships—e.g., monotonicity, nonlinearity, and correlation—within the data to generate structured descriptions. Simultaneously, it monitors equation performance and establishes a feedback loop to refine subsequent generations. By integrating data understanding and generation reflection in a closed loop, DrSR enables more efficient exploration of the symbolic expression space. Experiments across interdisciplinary datasets in physics, chemistry, biology, and materials science demonstrate that DrSR substantially improves the valid equation rate and consistently outperforms both classical and recent LLM-based methods in terms of accuracy, generalization, and search efficiency—underscoring its potential for scientific equation discovery. 1 Introduction Symbolic regression (SR) makke2024interpretable aims to recover interpretable mathematical expressions from observational data and plays a critical role in scientific modeling—ranging from uncovering physical laws lemos2023rediscovering ; davis2023discovery , to modeling chemical kinetics batra2021emerging ; hernandez2019fast , to understanding financial dynamics yu2023generating ; xu2024hrft . The resulting equations support extrapolation, cross-domain transfer, and mechanistic understanding schmidt2009distilling . The core challenge in SR lies in navigating the vast, combinatorial space of mathematical expressions to identify models that both fit the data and remain interpretable. To address this, a range of algorithmic strategies have been proposed. Classical approaches based on genetic programming (GP) schmidt2009distilling ; cranmer2023interpretable evolve expression trees via mutation and crossover. Reinforcement learning (RL) methods petersen2019deep treat symbolic equation discovery as a sequential decision-making problem, using policy gradients to explore the space of symbolic programs. Transformer-based neural models biggio2021neural ; kamienny2022end have been employed to directly generate symbolic expressions by modeling equations as sequences. Figure 1 : Overview of the DrSR framework. (a) Data-aware Insight: the LLM analyzes variable relationships to derive structured, data-informed insights; (b) Inductive Idea Extraction: based on equation fitness, the LLM summarizes successful and failed patterns into an evolving idea library; (c) Equation Generation and Selection: guided by (a) and (b), the LLM generates new equation skeletons, optimizes parameters, and caches the best-performing results to inform future generations. More recently, LLMs have further advanced this area by generating candidate equations from textual prompts. For example, LLM-SR shojaee2024llm proposes new expressions conditioned on task descriptions and prior candidates, combining LLM generation with numeric optimization. However, such methods overly rely on the scientific priors embedded within LLMs while neglecting structural insights into variable relationships—effectively ignoring direct observation and analysis of the raw data. This limits the accuracy and generalizability of the generated equations. Moreover, the lack of systematic reflection on generation history often leads to the repeated production of invalid expressions (e.g., syntax errors, numerical overflows), reducing both the efficiency and stability of SR. To address these issues, a more effective SR framework is needed—one that not only incorporates scientific priors flexibly, but also analyzes data structures and learns from past generation behavior. Such an approach would better align with the scientist’s cognitive process: inducing patterns from observations, reflecting through trial and error, and efficiently navigating the vast hypothesis space to uncover interpretable and predictive models. To this end, we propose DrSR (Dual Reasoning Symbolic Regression) (Fig 1 ), a framework that augments LLM-based generation with two synergistic reasoning mechanisms: data-aware insight , which derives structural patterns from raw data, and inductive idea extraction , which reflects on generation outcomes to distill reusable strategies. Specifically, building upon the generation-evaluation loop of LLM-SR, a data-aware module prompts the LLM to extract structural patterns from a sampled subset of raw data, providing task-specific priors on variable relationships. As better-fitting equations are discovered, these insights are incrementally updated (Fig 1 (a)). Secondly, DrSR introduces an experience-based module that categorizes newly generated equations based on evaluation outcomes and prompts the model to reflect on success or failure. The resulting strategies—summarized as structured “ideas”—are stored in a dynamic idea library and reused to improve future equation generation (Fig 1 (b)). By combining data-driven interpretation with behavior-level feedback, DrSR enables continual refinement of both symbolic priors and generative heuristics, resulting in more efficient and reliable symbolic equation discovery. We evaluate DrSR on six benchmarks spanning physics, chemistry, biology, and materials science, using Mixtral-8x7B-Instruct-v0.1 jiang2024mixtral and LLaMA3.1-8B-Instruct kassianik2025llama as backbones. Compared to traditional SR methods and LLM-based baselines, DrSR achieves state-of-the-art performance across multiple metrics, including accuracy, generalization, valid expression rate, and convergence speed. 2 Preliminaries In SR, the objective is to discover a compact symbolic expression f(\mathbf{x}) that approximates an unknown mapping f^{real}:\mathbb{R}^{d}\rightarrow\mathbb{R} based on a dataset of input-output pairs D=\{(\mathbf{x}_{i},y_{i})\}_{i=1}^{n} . SR methods aim to recover underlying mathematical relationships such that f(\mathbf{x}_{i})\approx y_{i} for all i . The discovered expression is expected to not only fit the observed data accurately but also generalize well to unseen inputs while maintaining interpretability. Model performance is typically evaluated via the negative mean squared error: \text{Score}(f,D)=-\frac{1}{n}\sum_{i=1}^{n}(f(\mathbf{x}_{i})-y_{i})^{2}. Our work builds upon LLM-SR shojaee2024llm , a recent framework that utilizes LLM to generate interpretable equations from data. To contextualize our proposed DrSR, we briefly review the core workflow of LLM-SR. LLM-SR discovers equations by combining the scientific knowledge embedded in LLMs with their powerful code generation capabilities, prompting the LLM to generate equation skeletons \mathcal{F} with learnable parameters. The overall procedure consists of three key components: Hypothesis Generation. The LLM receives a structured prompt comprising: a task description \mathcal{T} , variable constraints, evaluation criteria, and reference equations. Based on this, it generates equation skeletons aligned with the physical meaning of variables. Hypothesis Evaluation. Each skeleton is fitted to data by optimizing its parameters. The resulting function is scored via the fitness metric \text{Score}(f,D) . Experience Management. A memory buffer stores high-scoring equations, which are reused as demonstrations in future prompts to improve generation quality. While LLM-SR effectively leverages prior knowledge embedded in LLMs, it suffers from two critical drawbacks: First, its understanding of data remains shallow: although LLM-SR can generate equation skeletons via language prompts and refine them through parameter optimization, it lacks direct engagement with the data itself. As a result, it fails to capture complex inter-variable relationships that are crucial for SR, limiting the reasoning capacity of LLMs. Second, it lacks a reflective mechanism to evaluate and learn from its own behavior. Without such self-correction, LLM-SR frequently produces invalid expressions—such as syntactically incorrect forms, numerical instabilities, or inconsistent variable usage—ultimately degrading performance and robustness. 3 Method To overcome these limitations, we propose DrSR (Dual Reasoning Symbolic Regression), a novel framework that enhances SR by equipping LLM with two synergistic reasoning pathways: Data-aware Insight and Inductive Idea Extraction . While retaining the generative strength of LLM-SR in leveraging scientific priors, DrSR introduces a closed-loop reasoning cycle that enables the model to both

Read the complete text