Call for Action: Towards the Next Generation of Symbolic Regression Benchmark
Guilherme Seidyo Imai Aldeia, Hengzhe Zhang, Geoffrey Bomarito, Miles Cranmer, Alcides Fonseca, Bogdan Burlacu, William G. La Cava, Fabrício Olivetti de França · 2025
Read Call for Action: Towards the Next Generation of Symbolic Regression Benchmark by Guilherme Seidyo Imai Aldeia, Hengzhe Zhang, Geoffrey Bomarito, Miles Cranmer, Alcides Fonseca, Bogdan Burlacu, William G. La Cava, Fabrício Olivetti de França (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-03977.txt.
Introduction
Guilherme Aldeia and seven co-authors refresh a bench for formula-finding tools and call the field to keep it live. Such tools hunt math forms that fit data and can be read at a glance. Bench work is hard since tools, sets, and grades all split. Their 2025 refresh nearly doubles the tool count, sharpens the grades, and draws the results in clearer charts.
No tool wins on all sets: form size, accuracy, and power cost trade off by case. The team asks the field to run the bench as a shared, live one, with set rules for tuning time, run caps, and resource size. Old tools should be cut by set rules to keep the bench fresh. Lean tuning and low-power builds get a nod as good craft.
Catalog record
| CREATOR | Guilherme Seidyo Imai Aldeia, Hengzhe Zhang, Geoffrey Bomarito, Miles Cranmer, Alcides Fonseca, Bogdan Burlacu, William G. La Cava, Fabrício Olivetti de França |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-03977
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Call for Action: Towards the Next Generation of Symbolic Regression Benchmark Guilherme Seidyo Imai Aldeia, Hengzhe Zhang, Geoffrey Bomarito, Miles Cranmer, Alcides Fonseca, Bogdan Burlacu, William G. La Cava, Fabrício Olivetti de França Abstract . Symbolic Regression (SR) is a powerful technique for discovering interpretable mathematical expressions. However, benchmarking SR methods remains challenging due to the diversity of algorithms, datasets, and evaluation criteria. In this work, we present an updated version of SRBench. Our benchmark expands the previous one by nearly doubling the number of evaluated methods, refining evaluation metrics, and using improved visualizations of the results to understand the performances. Additionally, we analyze trade-offs between model complexity, accuracy, and energy consumption. Our results show that no single algorithm dominates across all datasets. We propose a call for action from SR community in maintaining and evolving SRBench as a living benchmark that reflects the state-of-the-art in symbolic regression, by standardizing hyperparameter tuning, execution constraints, and computational resource allocation. We also propose deprecation criteria to maintain the benchmark’s relevance and discuss best practices for improving SR algorithms, such as adaptive hyperparameter tuning and energy-efficient implementations. CCS: Computing methodologies Symbolic and algebraic algorithms CCS: General and reference Empirical studies CCS: Human-centered computing Visualization design and evaluation methods 1. Introduction Symbolic regression (SR) ( Koza, 1992 ; Kronberger et al., 2024 ) is a supervised machine learning technique that searches for a mathematical expression \hat{f}(\mathbf{x},\mathbf{\hat{\theta}}) that fits a set of points \mathbf{x}_{i}\in\mathcal{X},y_{i}\in\mathcal{Y} such as y\approx\hat{f}(\mathbf{x},\mathbf{\hat{\theta}}) , where \mathbf{\hat{\theta}} is a vector of adjustable parameters. Searching for the function \hat{f} adds a degree-of-freedom to data fitting that differs from traditional parametric models that starts from a fixed f(x,\mathbf{\theta}) and tries to find \mathbf{\hat{\theta}} that minimizes a loss function. Currently, many SR algorithms implement a variation of the original genetic programming (GP) as proposed by Koza ( Koza, 1992 ; Koza, 1994 ) . Since then, it has been successfully applied to many real-world applications, such as, scientific discovery in engineering ( Liu et al., 2024 ) , healthcare ( La Cava et al., 2023 ) , physics ( Hruška et al., 2025 ) , among others ( Schmidt and Lipson, 2009 ; Kronberger et al., 2024 ; Cranmer, 2023 ; Udrescu and Tegmark, 2020 ; Russeil et al., 2024 ) . Its application in physics can be seen as “automating Kepler” —discovering analytical expressions through trial and error—, particularly evident in recent works that use SR to recover laws directly from experimental data ( Makke and Chawla, 2024 ) , underscoring its potential as a tool for automated scientific reasoning. Equation discovery is an NP-hard problem ( Virgolin and Pissis, 2022 ) , and implementations rely on various approaches to solve it, such as GP ( Schmidt and Lipson, 2010 ; La Cava et al., 2016 ; Randall et al., 2022 ; La Cava et al., 2019 ; Cava et al., 2019 ; Espada et al., 2022 ; Virgolin et al., 2021 ; Dick et al., 2020 ; de Franca and Aldeia, 2021 ; Kommenda et al., 2020 ; Zhang et al., 2022 ; Cranmer, 2023 ; de França, 2022 ) , Neural Networks ( Biggio et al., 2021 ; Sahoo et al., 2018 ) , Transformers ( Shojaee et al., 2023 ; Landajuela et al., 2022 ; Kamienny et al., 2022 ) , Bayesian models ( Jin et al., 2020 ) , iterative methods ( McConaghy, 2011 ; Kartelj and Djukanović, 2023 ; de França, 2018 ) , and even exhaustive approaches ( Kammerer et al., 2020 ; Bartlett et al., 2024 ) . In its original implementation, GP relied on generating ephemeral random constants as part of the expression, which could lead to sub-optimal exploration of the search space if these constants were not properly set. This was alleviated with the use of linear scaling ( Keijzer, 2003 ) by scaling and translating the solution before measuring the loss function. Modern implementations rely on the optimization of the numerical parameters by either enforcing the creation of a model that is linear in the parameters ( Arnaldo et al., 2014 ; de Franca and Aldeia, 2021 ; de França, 2022 ; Cava et al., 2019 ) or applying a non-linear optimization method to fit the parameters ( Dick et al., 2020 ; Kommenda et al., 2020 ) . Despite recent advances in the field of SR, La Cava et al. (2021) observed a lack of consensus regarding a standardized set of datasets and benchmarking methodologies, making it challenging to accurately establish the current state-of-the-art (SotA). Another issue is the lack of a unified programming interface and clear instructions on installing and using each approach properly, creating barriers to new experimentation and reproducibility. To address these challenges, SRBench 1 1 1 https://cavalab.org/srbench/ was proposed as a living benchmark for SR, comparing different SR implementations and standard ML regression models. This benchmark enforced the adoption of a common Python API and installation scripts within a sandbox environment, making it easier to install and evaluate different algorithms. Running a benchmark is a laborious yet important task. Not only is it necessary to compare methods, but it also serves as a tool for measuring progress in the field. While the SRBench was a significant step toward comparing and understanding the current SotA in SR, designing effective benchmarks often requires further refinements and consideration of new factors to ensure their usefulness and longevity. An effective benchmark should contain diverse subclasses of problems, identifying which algorithms perform best in each scenario, mapping problems to algorithms rather than providing a one-size-fits-all solution. Additionally, the benchmark must challenge contemporary SR algorithms while remaining computationally feasible and representative of real-world challenges, ensuring that results extend beyond artificial test cases. SRBench falls short in these aspects, as it relies on more than 200 datasets without a detailed categorization of their specific challenges. Furthermore, it presents final results using only aggregated metrics, which can hide finer details in performance comparisons. In this paper, we propose several improvements to the existing SRBench (hereafter referred to as SRBench 1.0), identifying challenges that must be addressed to develop a more refined version of the benchmark (SRBench 2.0). Additionally, this paper serves as a call to action , encouraging the community to engage in discussions on best practices for benchmarking and evaluating SR algorithms. Such involvement is crucial for the field to achieve a more mature state and drive further advancements. The primary improvements proposed include: expanding the benchmark to incorporate new SR algorithms, bringing the total to 25 ; increasing the number of independent runs from 10 to 30 to enhance the statistical significance of the results; selecting 24 datasets, divided into two tracks: (i) black-box (drawn from SRBench 1.0) and (ii) phenomenological & first-principles (sourced from ( Cranmer, 2023 ) ); and refining the reporting methodology to provide a more detailed comparison of the nuances among top-performing algorithms, rather than simply ranking them and designating a single SotA method. This call to action aims to highlight the challenges and issues encountered during benchmarking and to propose features that could streamline the process. Addressing these issues requires collaboration from method’s authors/maintainers to develop and extend a common API that facilitates the expansion of SRBench in future iterations.