SynPAT: A System for Generating Synthetic Physical Theories with Data
Karan Srivastava, Jonathan Lenchner · 2025
Read SynPAT: A System for Generating Synthetic Physical Theories with Data by Karan Srivastava, Jonathan Lenchner (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-00878.txt.
Introduction
Karan Srivastava and Jonathan Lenchner present SynPAT, a 2025 tool that forges fake physics theories for testing discovery software. Programs that hunt for laws of nature need practice cases, but true theories are few. SynPAT fills the gap by building made-up worlds with known rules.
Each case ships with a set of axioms, a target result that follows from them, and noisy data that fits the result. The tool can also build flawed worlds, where the data clash with the axioms and the solver must fix its base claims. Tests with law-hunting programs show the cases sort strong tools from weak ones fairly.
Catalog record
| CREATOR | Karan Srivastava, Jonathan Lenchner |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-00878
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SynPAT: A System for Generating Synthetic Physical Theories with Data Karan Srivastava, Jonathan Lenchner Abstract Machine-assisted methods for discovering new physical laws of nature, starting from a given background theory and data, have recently emerged, and seem to hold the promise of someday advancing our understanding of the physical world. To address these needs, we have developed SynPAT, a system for generating synthetic physical theories comprising (i) a set of consistent axioms, (ii) a symbolic expression that is a consequence of the axioms and the challenge to be discovered, and (iii) noisy data that approximately match the consequence. We also generate theories that do not correctly predict the consequence. We give a detailed description of the inner workings of SynPAT and its various capabilities. We also report on our benchmarking of several open-source symbolic regression systems using our generated theories and data. 1 Introduction Machine-assisted methods for discovering new physical laws of nature, starting from a given background theory and data, have recently emerged, and seem to hold the promise of someday advancing our understanding of the physical world ai-descartes23 ; ai-hilbert24 . Given a symbolic regression problem, in other words, some data representing a phenomenon that one wishes to capture a symbolic representation of, two interesting questions emerge: 1) Can a complete and correct background theory be used to improve on the results of the symbolic regression task? 2) Can the presence of a background theory that incorrectly predicts the results of the phenomenon for which we have data, in fact, be helpful in the symbolic regression task? In practice, the presence of the background theory may actually bias us away from finding a good symbolic form. But this is the problem faced by leading scientists today in the physical sciences, particularly in physics. We have a solid theory that predicts many phenomena very accurately, but fails in some cases (e.g., data on supernova explosions and the rotation of galaxies are at odds with the equations of general relativity, and the observed abundance of lithium is at odds with the equations that we believe govern the strong and weak nuclear forces and nuclear synthesis in the primordial universe). However, the fact that there are relatively few known theories in the physical sciences has made the training, testing, and benchmarking of systems that utilize the combination of a background theory and data difficult. To address these needs, we have developed SynPAT, a system for generating synthetic physical theories comprising (i) a set of consistent axioms, (ii) a symbolic expression that is a consequence of the axioms and the challenge to be discovered, and (iii) noisy data that approximately match the consequence. We also generate theories that do not correctly predict the consequence. We give a detailed description of the inner workings of SynPAT and its various capabilities. We also report on our benchmarking of several open-source symbolic regression systems using our generated theories and data. The remainder of this paper is organized as follows. In the next section, we review related work on symbolic regression, the benchmarking of symbolic regression systems, and recent work employing theories in conjunction with data to discover new scientific laws. Following that, we give a detailed description of our theory, consequence, and data generation processes. We then discuss the case where some portion of a theory is not such a good match to the data and the generation of what we call theories with biased data. Following this, we review the benchmarks we have run using our theories, discuss code and data set availability, and end with conclusions and future work. 2 Related Work Symbolic regression refers to the idea that given some data associated with a set of variables, one can search through a space of mathematical expressions to come up with a symbolic expression from the search space that best characterizes the data in the sense of minimizing a given error metric. This idea in its full generality dates back to 1992 and the textbook on genetic programming ( koza1992 , Ch. 2.4) . However, the BACON.5 system of langley1981bacon , which discovered physical conservation laws from a small set of data-driven heuristics, dates back much further – to 1981. Linear and multilinear regression date back far earlier with the discovery of linear regression actually being credited to Newton GROPING16 . Some of the early work on symbolic regression includes korns06 ; duffy99 ; bautu95 for discovering individual equations not involving derivatives, and then the work of Bongard and Lipson bongard-pnas07 , which allows not just for the discovery of individual equations with ordinary derivatives among the regressed symbols but coupled systems of such equations. Given some data and the problem of finding one or more best-fitting governing ordinary differential equations, with initial conditions either provided or output as part of the solution, several recent approaches have utilized deep neural networks and even transformers. There is a rapidly growing literature, but some examples include NODE NODE2018 , ODEFormer d'ascoli2024odeformer , which comes with a dataset of 63 ODEs from the literature that the authors refer to as ODEBench, and the predecessor system to ODEFormer 10.5555/3618408.3618491 . Both ODEFormer d'ascoli2024odeformer and its predecessor system borrow ideas from cava2018learning , which originated the concept of learning concise representations by evolving networks of trees of operators. A nice comparison of some of the most important symbolic regression methods and systems up through 2021 is given in la2021contemporary , which also includes a large benchmark and benchmark dataset (to be discussed later). At least two commercially available symbolic regression (SR) systems exist: Eureqa (software that is now part of the DataRobot software suite), the algorithm behind which is described in doi:10.1126/science.1165893 , and the Wolfram SR system, described in wolfram2004 . Both Eureqa and Wolfram use genetic algorithms. The Eureqa system was judged to be state of the art by the AI Feynman system doi:10.1126/sciadv.aay2631 ; udrescu2020aifeynmanphysicsinspiredmethod , which is one of the three systems we describe the benchmarking of, using SynPAT, in the present work. The AI Feynman system is a “physics-inspired” symbolic regression system. It recursively applies a set of decomposition tactics and specialized solvers gleaned from the study of known physics equations, followed by polynomial fitting and brute-force search. If the problem is not directly solved, it appeals to a deep neural network (NN) based component. As usual with most symbolic regression systems, it starts with data and tries to discover a single best governing function , given user-specified variables and constants, each of which has associated units of measure. It does not support derivatives. The AI Feynman system was able to use a feedforward neural network with 6 hidden layers, and using 100,000 synthetically generated exactly matching data points per equation, recover all 100 of the equations it collated from the Feynman Lectures on Physics, volumes 1–3 feynman1963 . The results were compared to a 71% score on the same test set for the Eureqa SR system. As confessed by the authors, there is an inherent mixing of training and test data in these results, so they actually treat these 100 equations as training data for a further test on what they call 20 “bonus” equations culled from a variety of additional classical