ForwardFlow: Simulation only statistical inference using deep learning

Stefan Böhringer ∗ Department of Biomedical Data Sciences Leiden University Medical Center Einthovenweg 20, 2333 ZC Leiden, The Netherlands [email protected] ∗ corresponding author · 2026

Read ForwardFlow: Simulation only statistical inference using deep learning by Stefan Böhringer ∗ Department of Biomedical Data Sciences Leiden University Medical Center Einthovenweg 20, 2333 ZC Leiden, The Netherlands [email protected] ∗ corresponding author (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10991.txt.

Introduction

Stefan Böhringer proposes ForwardFlow, a frequentist answer to simulation-only inference. It replaces the Bayesian normalizing-flow machinery with a single summary network. Trained on simulated datasets, the network learns a summary statistic and maps it to the parameter. This solves the inverse problem. Collapsing layers shrink each dataset to its summary. Tests show finite-sample exactness and that the method stays robust with flawed data. They also show it can learn algorithms on its own. An EM algorithm for genetic data is learned without being programmed in.

Catalog record

CREATORStefan Böhringer ∗ Department of Biomedical Data Sciences Leiden University Medical Center Einthovenweg 20, 2333 ZC Leiden, The Netherlands [email protected] ∗ corresponding author
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41821
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.10991

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ForwardFlow: Simulation only statistical inference using deep learning Stefan Böhringer ∗ Department of Biomedical Data Sciences Leiden University Medical Center Einthovenweg 20, 2333 ZC Leiden, The Netherlands [email protected] ∗ corresponding author Abstract Deep learning models are being used for the analysis of parametric statistical models based on simulation-only frameworks. Bayesian models using normalizing flows simulate data from a prior distribution and are composed of two deep neural networks: a summary network that learns a sufficient statistic for the parameter and a normalizing flow that conditional on the summary network can approximate the posterior distribution. Here, we explore frequentist models that are based on a single summary network. During training, input of the network is a simulated data set based on a parameter and the loss function minimizes the mean-square error between learned summary and parameter. The network thereby solves the inverse problem of parameter estimation. We propose a branched network structure that contains collapsing layers that reduce a data set to summary statistics that are further mapped through fully connected layers to approximate the parameter estimate. We motivate our choice of network structure by theoretical considerations. In simulations we demonstrate three desirable properties of parameter estimates: finite sample exactness, robustness to data contamination, and algorithm approximation. These properties are achieved offering the the network varying sample size, contaminated data, and data needing algorithmic reconstruction during the training phase. In our simulations an EM-algorithm for genetic data is automatically approximated by the network. Simulation only approaches seem to offer practical advantages in complex modeling tasks where the simpler data simulation part is left to the researcher and the more complex problem of solving the inverse problem is left to the neural network. Challenging future work includes offering pre-trained models that can be used in a wide variety of applications. Declaration of interest: The authors declare no conflict of interests. 1 Introduction Simulation based approaches have a long tradition in statistical inference problems and have recently become more prominent [ 4 ] . Approximate Bayesian Computation (ABC) is an early example and has been employed in complex inference problems where it is difficult or impossible to evaluate the data likelihood [ 1 ] . More recently, normalizing flows have been established as an alternative for Bayesian inference [ 13 ] . To approximate the posterior distribution, normalizing flows do not need to evaluate the likelihood [ 14 ] . Normalizing flows employ deep neural networks (DNNs) to establish the normalizing mapping through a training process using simulations [ 14 ] . After training, sampling from the posterior can make use of the computationally efficient use of DNN inference [ 15 ] . In these examples, the data likelihood does not have to be evaluated which is an advantage in practical modeling. Both ABC and normalizing flows require sufficient summary statistics to properly construct the posterior distribution. For ABC, summary statistics have to be user supplied and might require deep insight into the analysis problem. Normalizing flows employ internal networks, summary networks, to automatically construct summary statistics. An additional restriction on the network structure arises for normalizing flows in that the mapping between the normal and posterior distribution has to be bijective. Another advantage of simulation based approaches is that they can implicitly account for finite sample properties by training networks on appropriate sample sizes [ 15 ] . Arguably, this is an advantage over other approximations such as variational Bayes [ 2 ] . In this paper, we investigate simulation only approaches which are based on a single network which learns sufficient summaries for the data. The focus is on offering a simulation-only approach for parametric models that can be implemented with little effort put into hyper-parameter tuning [ 16 ] . A second aspect is robustness, which can be easily added by training the estimator using contaminated data. In contrast to BayesFlow with a more complex network structure, training is faster and the summary network can be potentially also be simplified. These networks directly allow to implement frequentist inference which is explored. In a second step, Bayesian models can be recovered from frequentist models using ABC methodology. Some previous work has investigated statistical properties of deep neural networks. Results are either based in idealized assumptions ( e.g. [ 5 ] ) or concerned with model evaluation only ( e.g. [ 6 ] ). In general, such investigations face the difficulty that current DNNs are quantized at rather low numeric accuracy [ 10 ] , a fact, that is difficult to account for in theoretic investigations. We here make some effort to motivate the choice of network structure by theoretical arguments. The paper is structured as follows. In the second section the approach is described in terms of statistical models, general DNN structure, and some theoretical properties. The third section describes the network structure in detail, offering a family of models that can be tailored to the problem at hand. The next section describes simulations and is followed by a section describing results. In the final section, results are discussed and an outlook is given on future research. 2 Methods For a parametric model, let P_{,}\,\in\Theta\subset\mathbb{R}^{k} be the family of distributions. Denote with X:\textrightarrow\mathcal{X}^{N} , X=(X_{1},...,X_{N}) a random variable, where X_{i}=(X_{i1},...,X_{iK})\sim P is K -dimensional data sampled from this model. Statistically, it is of interest to solve the inverse problem, i.e. find the mapping \hat{}:\mathcal{X}^{N}\textrightarrow that maps data to an estimate of the parameter. For statistical inference, it is required to derive the distribution of \hat{}(X) . For frequentist problems, the distribution of interest is the confidence distribution [ 18 ] , i.e. the formal distribution on \Theta from which all-level confidence sets for \theta can be derived. Formally, the confidence distribution is implicitly defined by a mapping \mathrm{C}_{conf}:\mathcal{X}^{N}\to\Theta so that P(C_{conf}(X,)\ni\theta)≥\,\forall\in(0,1) , where \mathcal{X}^{N} is the event space induced by X . For a Bayesian model, the target distribution is the posterior P(\theta|X) with prior distribution \theta_{0}\sim P_{pr} , where data X is given by X_{i}\sim P_{{}_{0}} as above. The posterior can also be implicitly defined by P(\theta_{0}\in C_{post}(X,,P_{pr}))≥\,\forall\in(0,1) . ForwardFlow aims to approximate these distributions based on on estimator \hat{\theta} which is learned by a deep neural network. 2.1 Frequentist inference Denote with \hat{} the estimator \mathcal{X}^{N}\textrightarrow\Theta , \hat{\theta}:=\hat{\theta}(X) . We also assume that parameters are drawn from a training distribution governed by a dispersion parameter s\noboundary>0 , so that \sim P_{tr,s\noboundary} . Without loss of generality, we assume s\noboundary to be one-dimensional. We can then define \displaystyle\hat{}_{s\noboundary} \displaystyle:= \displaystyle\argmin_{g}E_{s\noboundary}(E((g(X)-)^{2})), \displaystyle\hat{} \displaystyle:= \displaystyle\lim_{s\noboundary\textrightarrow\infty}\hat{}_{s\noboundary} where g\in\mathcal{G} is a reasonably large class of functions. These definitions reflect the fact that \hat{} is approximated using a neural network and training data has to be generated. The final estimator is the result for a “large” dispersion parameter s\noboundary which, in practice, is chosen empirically. The class \mathcal{G} is defined by the network topology and network capacity [ 6 ] . Remark 2.1 . Under standard regularity conditions for the maximum likelihood estimator to exist, \hat{} exists. \hat{} can be interpreted as the ML estimator for as it estimates the posterior mode for the prior distribution P_{tr,s\noboundary} under a Bayesian interpretation. Then, \hat{} converges to the ML estimator as P_{tr,s\noboundary} becomes uninformative as s\noboundary\textrightarrow∞ .

Read the complete text