[RETRACTED]Evolving Form and Function: Dual-Objective Optimization in Neural Symbolic Regression Networks
Amanda Bertschinger, James Bagrow, Joshua Bongard · 2025
Read [RETRACTED]Evolving Form and Function: Dual-Objective Optimization in Neural Symbolic Regression Networks by Amanda Bertschinger, James Bagrow, Joshua Bongard (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-17393.txt.
Introduction
Amanda Bertschinger and co-authors present a retracted 2025 paper on neural symbolic regression. The hosted text carries the retraction mark, and its claims should be read in that light. The work sought networks that write equations from data with both true form and true forecasts.
Older neural routes grade equations by symbol match alone, not by how well they predict. The team mixed gradient descent with evolutionary search so networks cut both error kinds at once. Their tests claimed sharper equations than gradient only rivals. The blend was offered as a path to better symbolic regression.
Catalog record
| CREATOR | Amanda Bertschinger, James Bagrow, Joshua Bongard |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-17393
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
[RETRACTED]Evolving Form and Function: Dual-Objective Optimization in Neural Symbolic Regression Networks Amanda Bertschinger, James Bagrow, Joshua Bongard Abstract . Data increasingly abounds, but distilling their underlying relationships down to something interpretable remains challenging. One approach is genetic programming, which ‘symbolically regresses’ a data set down into an equation. However, symbolic regression (SR) faces the issue of requiring training from scratch for each new dataset. To generalize across all datasets, deep learning techniques have been applied to SR. These networks, however, are only able to be trained using a symbolic objective: NN-generated and target equations are symbolically compared. But this does not consider the predictive power of these equations, which could be measured by a behavioral objective that compares the generated equation’s predictions to actual data. Here we introduce a method that combines gradient descent and evolutionary computation to yield neural networks that minimize the symbolic and behavioral errors of the equations they generate from data. As a result, these evolved networks are shown to generate more symbolically and behaviorally accurate equations than those generated by networks trained by state-of-the-art gradient based neural symbolic regression methods. We hope this method suggests that evolutionary algorithms, combined with gradient descent, can improve SR results by yielding equations with more accurate form and function. Figure 1. The Symbolic Regression NeuroEvolution (SRNE) method. (a) Generalized neural symbolic regression (GNSR) networks are trained to take any XY data as input and output an equation that describes said data. (b) Some networks have low symbolic loss (the difference between a generated equation’s form ( \widehat{eq} ) and that of the hidden equation ( eq ) underlying their training data (XY)) but high numeric loss (the difference between a generated equation’s predictions ( \hat{Y} ) and a hidden equation’s values ( Y )). (c) Others have high symbolic loss but low numeric loss. (d) SRNE evolves populations of data-to-equation neural networks to reduce their symbolic loss and numeric loss simultaneously to create the optimal case where networks have low symbolic and numeric loss. DOI: 10.1145/3638529.3654030 ISBN: 979-8-4007-0494-9/24/07 Conference: The Genetic and Evolutionary Computation Conference 2024; July 14–18, 2024; Melbourne, Australia CCS: Computing methodologies Neural networks CCS: Computing methodologies Genetic algorithms CCS: Computing methodologies Representation of mathematical functions 1. Introduction With the overwhelming large amount of data being collected worldwide, the ability to process and understand this data is paramount. However, raw data is generally very difficult for people to understand without the intervention of experts. These experts are generally tasked with describing the underlying relationships in the data, and often this requires finding equations that describe the dataset. Even experts may struggle with this difficult but important task. In order to help compensate for this, techniques such as regression are used to help infer these equations. Classical regression fits a predetermined "skeleton" equation to a dataset using coefficients ( Motulsky and Ransnas, 1987 ; Weisberg, 2005 ) . However, this approach still suffers from the need for expert intervention in picking the functional form of an equation to fit. To compensate for this weakness and fully automate the creation of an equation to describe a given dataset, symbolic regression (SR) can be used. SR traditionally uses genetic programming (GPSR) in order to build an equation from a corpus of mathematical operators ( Koza, 1992 ; Koza, 1994 ) . Using GPSR, equation trees are evolved using an evolutionary algorithm to fit a given dataset more and more accurately. It is an approach that can give very accurate equations, and as such GPSR has been well studied ( Keijzer, 2003 ; Keijzer, 2004 ; Schmidt and Lipson, 2008 ; Schmidt and Lipson, 2009 ; Arnaldo et al., 2014 ; Virgolin et al., 2021 ) . GPSR does have disadvantages though, primarily in the large amount of resources it requires: for each new dataset, GPSR needs to be trained from scratch. One recently proposed solution to this is neural symbolic regression (NSR), in which networks transform data into equations. Two primary types of neural symbolic regression exist: deep neural symbolic regression (DSR) ( Martius and Lampert, 2016 ; Sahoo et al., 2018 ; Kim et al., 2020 ; Costa et al., 2020 ; Petersen et al., 2019 ; Zhang et al., 2023 ; Boddupalli et al., 2023 ) and generalized neural symbolic regression (GNSR) ( Biggio et al., 2020 ; Biggio et al., 2021 ; Valipour et al., 2021 ; Kamienny et al., 2022 ; Vastl et al., 2022 ; Li et al., 2022 ; Bendinelli et al., 2023 ) . DSR methods train a network to generate one equation from one data set by utilizing neural networks’ structure and their activation functions to represent an equation. DSR is more computationally efficient than GPSR, but suffers the same drawback of requiring de novo retraining per new dataset that GPSR does. GNSR, however, leverages the scale of deep learning to pretrain a neural network on many unique dataset-equation pairs, and then uses transfer learning to allow the network to predict new equations from previously unseen datasets. Thus, though there is a large upfront cost to training a GNSR network, once it is trained it can quickly predict an equation for any given dataset. GNSR shows promise for overcoming the problems of classical regression and traditional SR techniques. However, it is known that the methods are often over-reliant on coefficient fitting after the networks generate an equation ( Bertschinger et al., 2023 ) : GNSR methods generate coefficient free equations, and a post hoc coefficient fitting step allows them to ‘cheat’ by generating symbolically inaccurate equations. When deprived of access to coefficient fitting, GNSR methods often yield equations that are still symbolically accurate (the form of generated equations are similar to that of the hidden equations used to generate the data they were trained on) or numerically accurate (the predictions of generated equations and hidden equations are similar), but not both. The difference between symbolic and numeric measures of equation accuracy is an important one: a symbolic loss metric evaluates a generated equation’s syntactic accuracy whereas a numeric loss metric evaluates a generated equation’s semantic accuracy. Importantly, an equation with low symbolic loss does not guarantee it will enjoy low numeric loss (Fig. 1 (b)) and vice versa (Fig. 1 (d)). One current problem with GNSR that is contributing to the observed inaccuracy across both kinds metrics is that all seven of the current GNSR methods ( Biggio et al., 2020 ; Biggio et al., 2021 ; Valipour et al., 2021 ; Kamienny et al., 2022 ; Vastl et al., 2022 ; Li et al., 2022 ; Bendinelli et al., 2023 ) are trained only on symbolic loss. Both symbolic and numeric metrics reveal orthogonal information about an equation’s accuracy ( Bertschinger et al., 2023 ) , and so GNSR methods need to be trained against both. However, with current techniques this has not been possible, for two reasons. First, it is challenging to backpropagate loss using two loss functions. Second, generated equations are currently numerically evaluated in a non-differentiable way. This means that their numeric loss can only be calculated during testing time, and cannot yet be employed to backpropagate numeric loss. In order to overcome this, here we present a