Symbolic Density Estimation: A Decompositional Approach

Angelo Rajendram, Xieting Chu, Vijay Ganesh, Max Fieg, Aishik Ghosh · 2026

Read Symbolic Density Estimation: A Decompositional Approach by Angelo Rajendram, Xieting Chu, Vijay Ganesh, Max Fieg, Aishik Ghosh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27955.txt.

Introduction

Rajendram and four co-authors introduce AI-Kolmogorov, a framework for symbolic density estimation. The method first splits a problem by clustering or by learning a graph structure. It then fits a plain density estimate and finds the support. Last, symbolic regression turns that estimate into a formula. Tests on mixture models, normal distributions, and three exotic cases show the method can recover the true distribution or reveal useful structure in it.

Catalog record

CREATORAngelo Rajendram, Xieting Chu, Vijay Ganesh, Max Fieg, Aishik Ghosh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88668
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.27955

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Symbolic Density Estimation: A Decompositional Approach Angelo Rajendram, Xieting Chu, Vijay Ganesh, Max Fieg, Aishik Ghosh Abstract We introduce AI-Kolmogorov, a novel framework for Symbolic Density Estimation (SymDE). Symbolic regression (SR) has been effectively used to produce interpretable models in standard regression settings but its applicability to density estimation tasks has largely been unexplored. To address the SymDE task we introduce a multi-stage pipeline: (i) problem decomposition through clustering and/or probabilistic graphical model structure learning; (ii) nonparametric density estimation; (iii) support estimation; and finally (iv) SR on the density estimate. We demonstrate the efficacy of AI-Kolmogorov on synthetic mixture models, multivariate normal distributions, and three exotic distributions, two of which are motivated by applications in high-energy physics. We show that AI-Kolmogorov can discover underlying distributions or otherwise provide valuable insight into the mathematical expressions describing them. 1 Introduction Data-driven approaches are widely used to model complex nonlinear relationships between predictive features and target variables. Popular approaches tend to be black-box and do not easily yield insight into underlying patterns in the data. Symbolic Regression (SR) is a family of methods that offer an interpretable alternative by searching the space of mathematical expressions and parameters to produce models that reveal useful insights into the structure underlying the data. While SR has seen success in supervised learning tasks such as regression and differential equation discovery, extending SR to the unsupervised task of density estimation introduces unique challenges and remains largely unexplored. Classical methods for density estimation typically fall into one of two categories; parametric density estimation assumes a model for the distribution and finds the best-fitting parameters, and nonparametric density estimation avoids model assumptions but sacrifices interpretability. This work addresses the problem of Symbolic Density Estimation (SymDE) using symbolic regression to discover mathematical expressions that accurately describe continuous probability distributions. Searching the space of closed-form expressions avoids the restrictive model assumptions of parametric density estimation while retaining interpretability unlike nonparametric density estimation. The SymDE problem faces three primary challenges: (i) ensuring validity constraints on probability distributions, namely non-negativity and normalization over the support of the distribution; (ii) the curse of dimensionality; and (iii) the difficulty of discovering complex symbolic expressions from a vast search space. Related Work SR is an established area of research with several promising deep learning based approaches emerging over recent years. However, literature on the application of SR to discover probability distributions from raw samples is scarce. Aside from seminal works on SR, few works that address the SymDE are mentioned here. Symbolic Regression: SR is classically approached with genetic programming (GP) ( 17 ) , with PySR ( 8 ) currently among the state-of-the-art tools. These algorithms employ genetic operations (crossover, mutation) to evolve expressions over successive generations, evaluating them using some measure of fitness. The methods presented in this paper use PySR as the engine for SymDE, although another popular SR engine may also be used. Deep Learning-Based SR: Recent advances that apply deep learning methods to discover descriptive mathematical expressions directly from data include Deep Symbolic Regression (DSR) ( 21 ) which uses reinforcement learning (RL). Neural-Guided Genetic Programming (NGGP) ( 19 ) combines RL with GP. Transformer-based techniques include NeSymReS ( 4 ) , SymbolicGPT ( 26 ) , and E2ESR ( 16 ) . Finally, LASR ( 13 ) enhances GP by integrating learned concept libraries guided by LLMs, introducing a mechanism that brings the technique closer to the human scientific discovery process. Decomposition-Based Methods: Other methods like AI Feynman ( 25 ) and AI Feynman 2.0 ( 24 ) , integrate neural networks with recursive decomposition strategies for equation discovery, aiming to reduce the combinatorial explosion of the symbolic search space. Symbolic Regression using Control Variables ( 15 ; 7 ) also attempts to address this issue by decomposing multi-variable SR into single-variable problems. These methods directly inspire our technique of decomposing high-dimensional distributions into simpler components. Symbolic Density Estimation: Maximum-Entropy based stochastic and symbolic density estimation (MESSY) ( 23 ) is a symbolic density estimation framework that recovers analytical forms of probability density functions from sample data. It constructs a gradient-based drift–diffusion process that connects observed samples to a maximum entropy ansatz. MESSY uses symbolic regression to explore candidate basis functions for the exponent in the maximum entropy model. This approach yields tractable symbolic descriptions of densities with low bias and a cost that is linear in the number of samples and quadratic in the number of basis functions. While the maximum entropy structure ensures valid probability densities and enables recovery of multimodal distributions through refinement, it does constrain the class of expressible densities compared with fully general functional forms. 18 apply the LLM-SR (Large Language Model for Symbolic Regression) framework from 22 to benchmark problems of equation recovery in lepton angular distributions and functional forms for angular coefficients in electroweak boson production. LLM-SR treats equations as programs with mathematical operators and combines LMMs’ scientific prior knowledge with evolutionary search over equation programs. It iteratively proposed new equation skeleton hypotheses which are subsequently optimized against to find the best parameters. While it takes advantage of available scientific domain knowledge, LLM-SR is limited to the available training data which may lead to biases or gaps. Our Contributions We introduce AI-Kolmogorov, a framework for solving the SymDE problem and bridge the gap between nonparametric and parametric density estimation. While previous approaches address partial aspects of this problem, they do not search over truly general symbolic functional forms for distributions in higher dimensional settings. Symbolic regression tools cannot directly solve SymDE because the training dataset typically does not contain density labels. We explore evolutionary algorithms such as in PySR ( 8 ) which could use log-likelihood scores but find that the resultant expressions may not adhere to the non-negativity and normalization constraints that every valid distribution must satisfy. We conduct experiments that demonstrate the efficacy of AI-Kolmogorov and showcase its results on various datasets. Our evaluation includes synthetic datasets such as multivariate normal distributions and mixture models, and other problems using non-standard distributions. We present ablations studies that evaluate the utility of the decomposition techniques proposed. The results show that AI-Kolmogorov provides useful information about the structure of the underlying distributions, producing interpretable symbolic models with low residual error and effectively trading off model complexity and accuracy, demonstrating the framework’s ability to aid the discovery of interpretable mathematical expressions that describe probability distributions. 1 1 1 https://anonymous.4open.science/r/SymbolicDensityEstimation-31D4 2 Background Symbolic Regression Given a dataset \mathcal{D}=\{(\mathbf{x}_{i},y_{i})\}_{i=1}^{N} , the goal is to identify a function f such that the target variable y_{i}\approx f(\mathbf{x}_{i})\hskip 10.00002pt\forall i\in\{1,2,..,N\} . SR discovers mathematical expressions that predict the target from the feature variables while balancing model accuracy and complexity. The ideal models are accurate yet parsimonious. Unlike traditional regression that fits parameters for a fixed model class, SR instead explores a vast space of potential mathematical expressions to find the most suitable representation. The search space of expressions grows exponentially with the number of permitted operators and variables. Candidate models are constructed as expression trees from operators and variables, adhering to a context-free grammar. SR has been shown to be NP-hard ( 27 ) , but despite the computational challenges, SR offers the possibility of interpretability which is central to scientific

Read the complete text