FNOPE: Simulation-based inference on function spaces with Fourier Neural Operators
Guy Moss, Leah Sophie Muhle, Reinhard Drews, Jakob H. Macke, Cornelius Schröder · 2025
Read FNOPE: Simulation-based inference on function spaces with Fourier Neural Operators by Guy Moss, Leah Sophie Muhle, Reinhard Drews, Jakob H. Macke, Cornelius Schröder (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22573.txt.
Introduction
Five researchers present a faster way to work backward from data to the hidden curves that produced it, in a 2025 paper. Many science simulators describe change over space and time with whole functions, not single numbers. Guessing those functions from observations is hard, and older simulation-based methods need huge computing budgets. The team's approach, called FNOPE, uses a neural operator built on wave-like math.
The method learns to estimate the odds of each candidate function given the data, at a fraction of the simulation cost of rival methods. It can give answers at any chosen spacing of points, and it can estimate plain numeric settings at the same time. Tests on standard tasks and on a hard glacier problem from earth science showed the method holds up well. The authors say it widens simulation-based inference to new fields.
Catalog record
| CREATOR | Guy Moss, Leah Sophie Muhle, Reinhard Drews, Jakob H. Macke, Cornelius Schröder |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-22573
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FNOPE: Simulation-based inference on function spaces with Fourier Neural Operators Guy Moss, Leah Sophie Muhle, Reinhard Drews, Jakob H. Macke, Cornelius Schröder Abstract Simulation-based inference (SBI) is an established approach for performing Bayesian inference on scientific simulators. SBI so far works best on low-dimensional parametric models. However, it is difficult to infer function-valued parameters, which frequently occur in disciplines that model spatiotemporal processes such as the climate and earth sciences. Here, we introduce an approach for efficient posterior estimation, using a Fourier Neural Operator (FNO) architecture with a flow matching objective. We show that our approach, FNOPE, can perform inference of function-valued parameters at a fraction of the simulation budget of state of the art methods. In addition, FNOPE supports posterior evaluation at arbitrary discretizations of the domain, as well as simultaneous estimation of vector-valued parameters. We demonstrate the effectiveness of our approach on several benchmark tasks and a challenging spatial inference task from glaciology. FNOPE extends the applicability of SBI methods to new scientific domains by enabling the inference of function-valued parameters. † † footnotetext: Code available at https://github.com/mackelab/fnope 1 Introduction Probabilistic inference of mechanistic parameters in numerical models is a ubiquitous task across many scientific and engineering disciplines. Among methods for Bayesian inference, simulation-based inference (SBI, [ 1 , 2 , 3 , 4 , 5 , 6 ] ) has emerged as a powerful approach for performing inference without requiring explicit formulation or evaluation of the likelihood. Instead, SBI only requires a simulator model which can sample from the likelihood. By training a generative model on pairs of parameters and simulation outputs, SBI can directly estimate probability distributions such as the posterior distribution. However, existing SBI methods are designed to infer a limited number of vector-valued parameters, which strongly limits their use for inferring spatially and/or temporarily varying, function-valued parameters. In these cases, parameters are commonly inferred on fixed discretizations of the domain. Despite some recent advances leveraging generative models to infer higher-dimensional posterior distributions [ 7 , 8 , 9 , 10 , 11 ] , the high-dimensional inference problems that arise from such approaches remain a challenge. Furthermore, current models need to be retrained for new discretizations of the parameters or the observations. This is particularly challenging in fields like the geosciences, where observations cannot always be made at the same locations. An alternative to using fixed discretizations is to represent the functions using a fixed set of basis functions, where the inference problem becomes inferring the basis function coefficients, as used, e.g., in [ 12 ] . However, these approaches require a good selection of basis functions and suffer from a trade-off between choosing sufficiently expressive basis sets, while maintaining a tractable number of parameters to infer. To overcome these limitations, we require methods that are capable of modeling and inferring function-valued data. Here, we propose to make use of the Fourier Neural Operator (FNO, [ 13 ] ) architecture, which operates on function-valued data, for performing SBI on function-valued parameters. Neural operators [ 14 , 15 , 16 ] combine operations on global features of function-valued data with local (typically pointwise) operations, thus capturing both global and local structures. In particular, FNOs use Fourier features to model the global structure. For smoothly varying data, the spectral power is concentrated in the lower frequency components of the spectral decomposition. This allows for a compact representation of the global structure of the data, and hence for the inference of function-valued data on high resolution discretizations. Figure 1: Overview. FNOPE approximates the posterior over function-valued parameters of a mechanistic model conditioned on function-valued observations. We use a FNO architecture with a flow matching objective to efficiently represent the function-valued parameters, enabling us to estimate extremely high dimensional posterior distributions at arbitrary discretizations of the domain. We present FNOPE (Fig. 1 ), an inference method for function-valued parameters : It trains FNO s for P osterior E stimation using a flow-matching objective [ 17 , 9 ] . FNOPE is capable of solving inference problems for spatially and temporally varying parameters, and can generalize to posterior evaluations on non-uniform, previously unseen discretizations of the parameter and observation domains. Furthermore, FNOPE can estimate additional, vector-valued parameters of the simulator. We demonstrate these features on a collection of benchmark tasks, as well as a challenging real-world task from glaciology. We compare the performance of FNOPE to SBI approaches that use fixed-discretization or basis-function representation of the parameters, and show that FNOPE outperforms these methods, especially for low simulation budgets. Thus, FNOPE enables efficient inference for high dimensional inference problems that were previously challenging or even intractable. 2 Preliminaries 2.1 Simulation-based inference SBI is designed to solve stochastic inverse problems: Given a simulator parametrized by \theta , a known prior distribution p(\theta) and an observation x\in\mathbb{R}^{N_{x}} , the goal is to infer the posterior p(\theta\mid x) for (typically) vector-valued parameters \theta\in\mathbb{R}^{N_{\theta}}. The simulator implicitly defines the model likelihood p(x\mid\theta) by allowing us to sample x\sim p(x\mid\theta) . We can construct a training dataset by sampling from the joint p(\theta)p(x\mid\theta) to construct a dataset of simulations S=\{(\theta_{i},x_{i})\}_{i=1}^{K} for a number of simulations K . Standard approaches in neural posterior estimation (NPE) approximate the posterior q^{\phi}(\theta\mid x) with a normalizing flow, which is trained by minimizing the negative log-likelihood -\mathbb{E}_{(\theta,x)\sim S}\log q^{\phi}(\theta\mid x) [ 1 , 3 ] . In contrast to this, flow-matching posterior estimation (FMPE) [ 9 ] learns a conditional velocity field v^{\phi}_{t,x}(\theta_{t}) to iteratively denoise samples from a base distribution (typically a Gaussian distribution) to the posterior distribution p(\theta\mid x) . The velocity v_{t,x}^{\phi} is trained via the flow matching objective \mathcal{L}_{\text{FMPE}}=\mathbb{E}_{t\sim\mathcal{U}[0,1],(\theta,x)\sim S,z_{t}\sim p_{t}(z_{t}\mid\theta)}||v_{t,x}^{\phi}-u_{t}(z_{t}\mid\theta)||^{2}, (1) where p_{t}(z_{t}\mid\theta) are the sample-conditional flow paths for z_{t} , and u_{t}(z_{t}\mid\theta) are the true velocity fields. The sample-conditional paths are chosen so that p_{t} and u_{t} are analytically tractable. 2.2 Fourier Neural Operators We use FNOs [ 13 ] to efficiently learn the posterior distribution of function-valued parameters. FNOs are a class of neural operators using the Fourier basis as an intermediate representation of functional data to learn mappings between function spaces. We assume to have a bounded domain D\subset\mathbb{R}^{d} , on which we define function spaces \mathcal{A}(D;\mathbb{R}^{d_{a}}) and \mathcal{B}(D;\mathbb{R}^{d_{b}} ). The goal of neural operators is to approximate some given operator \mathcal{G}:\mathcal{A}\to\mathcal{B} by a learnable operator \tilde{\mathcal{G}}^{\phi{}}:\mathcal{A}\to\mathcal{B} . In practice, the function-valued data is represented as discretizations of sample functions a_{i}\in\mathcal{A},b_{i}\in\mathcal{B} on the domain D . A single-layer FNO, \tilde{G}^{\phi}:\mathcal{A}\to\mathcal{B} , is defined by b(x)=\sigma(W^{\phi}a(x)+(\mathcal{K}^{\phi}a)(x))\quad\forall x\in D, (2) where W_{\phi} is a learnable linear operator, \sigma is a (pointwise) non-linearity, and (\mathcal{K}^{\phi}a)(x)=\mathcal{F}^{-1}(R^{\phi}(\mathcal{F}a))(x)\forall x\in\mathcal{D}. Here, \mathcal{F} and \mathcal{F}^{-1} refer to the Fourier and inverse Fourier transformation, and R^{\phi} refers to some operator acting on the Fourier modes of a . Typically, R^{\phi} is a linear transformation, and therefore corresponds to a convolution in real space with \mathcal{K}^{\phi} . But typically R^{\phi} only acts on the lower Fourier modes, discarding higher ones, and therefore gives rise to a compact representation of high-resolution data. However, as Eq. 2 includes the linear operator W^{\phi}a(x) , FNOs are still able to capture local structures. 3 Method To extend the standard SBI setting to inferring function-valued parameters, we develop FNOPE by extending FMPE with FNOs as backbone