Neuro-Evolutionary Approach to Physics-Aware Symbolic Regression Conference: Genetic and Evolutionary Computation Conference; July 14–18, 2025; Malaga, Spain Genetic and Evolutionary Computation Conference (GECCO ’25), July 14–18, 2025, Malaga, Spain DOI: 10.1145/3712256.3726434 ISBN: 979-8-4007-146
Jiří Kubalík, Robert Babuška · 2025
Read Neuro-Evolutionary Approach to Physics-Aware Symbolic Regression Conference: Genetic and Evolutionary Computation Conference; July 14–18, 2025; Malaga, Spain Genetic and Evolutionary Computation Conference (GECCO ’25), July 14–18, 2025, Malaga, Spain DOI: 10.1145/3712256.3726434 ISBN: 979-8-4007-146 by Jiří Kubalík, Robert Babuška (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16503.txt.
Introduction
Jiří Kubalík and Robert Babuška blend evolution and neural learning to find formulas that fit data and respect physics, in a paper from two thousand twenty-five. Formula hunting, called symbolic regression, can yield readable models. Pure neural hunts tune constants well but settle too soon on weak structures.
Their method evolves network shapes while gradient steps tune the weights. A memory trick and population shakes keep the search from stalling, so each shape needs only brief training. Three real-world problems tested the method. It beat other neural approaches on model quality, the authors report.
Catalog record
| CREATOR | Jiří Kubalík, Robert Babuška |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-16503
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Neuro-Evolutionary Approach to Physics-Aware Symbolic Regression Conference: Genetic and Evolutionary Computation Conference; July 14–18, 2025; Malaga, Spain Genetic and Evolutionary Computation Conference (GECCO ’25), July 14–18, 2025, Malaga, Spain DOI: 10.1145/3712256.3726434 ISBN: 979-8-4007-1465-8/2025/07 CCS: Computing methodologies Genetic programming CCS: Computing methodologies Genetic algorithms CCS: Computing methodologies Neural networks Jiří Kubalík, Robert Babuška Abstract . Symbolic regression is a technique that can automatically derive analytic models from data. Traditionally, symbolic regression has been implemented primarily through genetic programming that evolves populations of candidate solutions sampled by genetic operators, crossover and mutation. More recently, neural networks have been employed to learn the entire analytical model, i.e., its structure and coefficients, using regularized gradient-based optimization. Although this approach tunes the model’s coefficients better, it is prone to premature convergence to suboptimal model structures. Here, we propose a neuro-evolutionary symbolic regression method that combines the strengths of evolutionary-based search for optimal neural network (NN) topologies with gradient-based tuning of the network’s parameters. Due to the inherent high computational demand of evolutionary algorithms, it is not feasible to learn the parameters of every candidate NN topology to the full convergence. Thus, our method employs a memory-based strategy and population perturbations to enhance exploitation and reduce the risk of being trapped in suboptimal NNs. In this way, each NN topology can be trained using only a short sequence of backpropagation iterations. The proposed method was experimentally evaluated on three real-world test problems and has been shown to outperform other NN-based approaches regarding the quality of the models obtained. † † cc-license: by 1. Introduction Symbolic regression (SR) is a data-driven technique that creates models in the form of mathematical expressions. It has achieved remarkable results in various nonlinear modeling problems ( Schmidt and Lipson 2009 ; Richmond-Navarro et al. 2017 ; Udrescu and Tegmark 2020 ; Derner et al. 2020 ) . Traditionally, SR has been implemented using genetic programming (GP) ( Schmidt and Lipson 2009 ; Koza 1992 ; Staelens et al. 2013 ; Arnaldo et al. 2015 ; Bladek and Krawiec 2019 ; Burlacu et al. 2020 ) . An evolutionary algorithm refines a population of candidate solutions over multiple generations, guided by a selection mechanism that favors high-quality solutions over weaker ones. New candidate models are generated through genetic operations such as crossover and mutation. More recent GP-based methods also incorporate the loss function gradient to fine-tune the model’s internal coefficients ( Topchy et al. 2001 ; Žegklitz and Pošík 2019 ; Kommenda et al. 2020 ) . Symbolic regression offers several benefits compared to other data-driven modeling techniques. Unlike deep neural networks, which typically require large amounts of data to perform well, SR can generate accurate models even from a limited number of training samples ( Wilstrup and Kasak 2021 ; Derner et al. 2021 ) . Furthermore, SR is well suited to integrate prior knowledge about the system being modeled ( Bladek and Krawiec 2019 ; Kubalík et al. 2020 ; Kubalík et al. 2021 ) . This is valuable when the dataset does not uniformly cover the input space or when there are no data samples in certain regions of the input space. Even with a sufficiently large and diverse dataset, methods that focus solely on training error minimization can produce inaccurate models. For instance, in the case of dynamic system models, errors typically occur in steady-state behavior or incorrect models of local dynamics. Although GP-based symbolic regression methods are widely used, they have several drawbacks. One major issue is the tendency of the model to grow in size and complexity without a corresponding performance improvement, a phenomenon referred to as code bloat ( Trujillo et al. 2016 ) . In addition, these methods become inefficient for large numbers of variables and samples in the training dataset. This is because they continuously evolve and evaluate an entire population of formulas. Finally, fine-tuning the model coefficients solely through genetic operators proved to be quite challenging for the pure GP methods. To address the challenges outlined above, various methods have recently been developed that leverage neural networks (NNs) to learn analytic formulas through gradient-based optimization techniques ( Martius and Lampert 2016 ; Sahoo et al. 2018 ; Werner et al. 2021 ; Costa et al. 2021 ; Kim et al. 2021 ; Zhou and Pan 2022 ) . These methods share a common principle: analytic models are represented by a heterogeneous NN, whose units perform mathematical operations and elementary functions such as {+, -, *, /, sin, exp, etc.}. Network weights are optimized using standard gradient-based algorithms to minimize the training error while significantly reducing the number of active units, i.e., the units that contribute to the output, through regularization. The resulting NN then represents a parsimonious analytic formula. Individual approaches differ primarily in how they guide the learning process toward the final model. Another NN-based symbolic regression method for constructing physically plausible models using minimal training data and prior system knowledge was introduced in ( Kubalík et al. 2023 ) . It employs an adaptive weighting scheme for balancing multiple loss terms and an epoch-wise learning strategy to minimize the risk of getting stuck in suboptimal local optima. Unlike the aforementioned NN-based approaches, this one incorporates prior knowledge about the model sought, making it more robust for practical deployment in real-world applications. These NN-based approaches also have notable drawbacks: they are inefficient in reducing the initial redundant NN architecture to a final sparse form, and gradient-based learning is inherently prone to premature convergence to suboptimal topologies, making it difficult to transition to a better one. This paper introduces a novel neuro-evolutionary SR algorithm, EN4SR , which integrates the advantages of the two main SR approaches mentioned above. This method combines an evolutionary search for optimal neural network topologies with gradient descent-based optimization of the network weights. In addition, it employs a technique we term weight memory and regular population perturbation to improve exploration and minimize the likelihood of getting stuck in suboptimal neural network topologies. The candidate NN models generated through this process adopt the architecture and the loss functions used in gradient-based learning inspired by those introduced for the N4SR method ( Kubalík et al. 2023 ) . The main challenge is to efficiently sample and refine candidate NN topologies despite the increased computational demands that naturally arise from the nature of evolutionary algorithms. The contributions of the paper are: • Design of a neuro-evolutionary SR method that combines advantages of the gradient-free evolutionary approach to generate candidate NN topologies and the gradient-based tuning of the NN weights. • Design of a memory-based strategy to facilitate effective use of crossover and mutation operators regarding the weights of NN structures modified by these operators. • The EN4SR method proved to be as good as or better than the pure NN-based approach in terms of model accuracy and computational demands while outperforming the other method in the frequency of produced high-quality solutions. • The EN4SR method has been shown to significantly outperform the GP-based approach in terms of the quality of generated models. The EN4SR algorithm was evaluated on four test problems and compared to other existing methods, among which is the NN-based N4SR serving as the baseline for comparisons with our