Stochastic Process Learning via Operator Flow Matching
Yaozhong Shi, Zachary E. Ross, Domniki Asimaki, Kamyar Azizzadenesheli · 2025
Read Stochastic Process Learning via Operator Flow Matching by Yaozhong Shi, Zachary E. Ross, Domniki Asimaki, Kamyar Azizzadenesheli (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-04126.txt.
Introduction
Yaozhong Shi and three co-authors proposed in 2025 a method for learning random processes over any domain, building on neural operator tools. Their method, operator flow matching, learns the prior pattern of a whole random function rather than single values. It can give the density of values at any set of points a user picks. That makes prediction at new points possible with both a mean and a spread. In the authors' tests, the method beat leading rivals at learning random processes, fitting functions, and learning priors.
Catalog record
| CREATOR | Yaozhong Shi, Zachary E. Ross, Domniki Asimaki, Kamyar Azizzadenesheli |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-04126
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Stochastic Process Learning via Operator Flow Matching Yaozhong Shi, Zachary E. Ross, Domniki Asimaki, Kamyar Azizzadenesheli Abstract Expanding on neural operators, we propose a novel framework for stochastic process learning across arbitrary domains. In particular, we develop operator flow matching ( OFM ) for learning stochastic process priors on function spaces. OFM provides the probability density of the values of any collection of points and enables mathematically tractable functional regression at new points with mean and density estimation. Our method outperforms state-of-the-art models in stochastic process learning, functional regression, and prior learning. 1 Introduction Stochastic processes are foundational to many domains, from functional regression and physics data assimilation, to financial markets, geophysics, and black box optimization. Stochastic processes can serve as prior distributions over functions and can provide the density of any finite collection of points. Conventionally, priors over processes are often hand-designed from predefined Gaussian processes ( GP ) and their variants tuned against data, only allowing for GP regression. This is despite the fact that phenomena modeled in the natural world often do not follow Gaussianity, Fig 1 . Consequently, this hinders the flexibility and generalizability of these stochastic processes in real-world applications, leaving behind significant challenges for more general stochastic process learning ( SPL ). In SPL , the prior over the stochastic process is learned from data, i.e., historical point evaluation of past experiments. Learning the prior over the process is crucial for universal functional regression ( UFR ), which is a recently proposed Bayesian scheme for functional regression and takes GP -regression as a special case when the prior is Gaussian ( Shi et al. 2024a ) . UFR is important to scientific and engineering domains, including reanalysis, data completion-assimilation, uncertainty quantification, and black box optimization. In this paper, we introduce a novel operator learning framework termed operator flow matching ( OFM ) for learning priors over stochastic processes through the joint distribution of any collection of points. To achieve this, we theoretically and empirically generalize marginal optimal transport flow matching ( Tong et al. 2024 ) to infinite-dimensional function spaces where we map a GP into a prior over function spaces through a flow differential equation. We then derive SPL from the function space derivation and learn a prior over any collection of points. For SPL , we map any collection of pointwise evaluations of a GP to pointwise evaluations of target functions. This allows us to learn prior distributions over the more general stochastic process, hence enabling sampling values of any collection of points with their associated density and facilitating efficient UFR . We leverage this capability by extending neural operators Azizzadenesheli et al. 2024 –designed initially to map functions between infinite-dimensional spaces–to map between collections of points deploying their functional convergence properties. This serves as the essential architecture block in OFM . After learning the prior and having access to the densities, OFM is used for UFR , where given any collection of points of the underlying function, we estimate the posterior mean value of any new collection of points and efficiently sample from their posterior values using stochastic gradient Langevin dynamics (SGLD) ( Welling & Teh 2011 ) , i.e., a Gaussian sampling on the input GP space. We show that OFM outperforms state-of-the-art methods, including deep GP s, neural processes, and operator flows ( OpFlow ) ( Salimbeni & Deisenroth 2017 ; Jankowiak et al. 2020 ; Garnelo et al. 2018 ; Kim et al. 2019 ; Shi et al. 2024a ) . (a) (b) (c) (d) (e) Figure 1: Operator Flow Matching ( OFM ) regression on Navier-Stokes functional data with resolution 64\times 64 . (a) 32 random observations. (b) Ground truth sample (c) Predicted mean from OFM . (d) One posterior sample from OFM . (e) One posterior sample from best fitted GP . Our principal contribution is that OFM represents the first simulation-free continuous normalizing flow for functional regression purposes, which transports a GP to a target stochastic process and enables likelihood estimation for any collection of points. Compared with existing baselines in functional regression, OFM enjoys greater expressiveness without the architectural constraints seen in deep GP s or OpFlow , and avoids the theoretical limitations associated with neural processes (see Appendix A.11 ). We empirically show that regression with OFM outperforms existing baselines, matches classical GP regression on GP examples, and delivers exceptional performance on highly non-Gaussian functional datasets, such as those from Navier-Stokes equations and black hole simulations. 2 Related work Neural operators. Neural operators constitute a paradigm in machine learning for learning maps between function spaces, a generalization of conventional neural networks that map between finite dimensional spaces ( Li et al. 2021 ; Kovachki et al. 2023 ) . Among neural operator architectures, Fourier neural operators (FNO) ( Li et al. 2021 ) enable convolution in the spectral domain and are effective for operator learning ( Pathak et al. 2022 ; Wen et al. 2023 ; Yang et al. 2021 ; Yang et al. 2023 ; Sun et al. 2023 ; Li et al. 2023 ) . In this work, we use this as our choice of neural operators architecture. Direct function samples. There is a body of work on generative models dedicated to learning distributions over functions, such that direct sampling on the function space is possible. For example, generative adversarial neural operators ( GANO ) generalize generative adversarial nets on finite dimensional spaces to function spaces ( Rahman et al. 2022 ; Shi et al. 2024b ) , yielding a neural operator generative model that maps GP to data functions ( Azizzadenesheli et al. 2024 ) . Other works in this area have followed the success of diffusion models ( Song et al. 2021 ; Ho et al. 2020 ) in finite dimensional spaces, e.g., denoising diffusion operators generalize diffusion models to function spaces by using GP as a means to add noise and use neural operators to learn the score operator on function-valued data ( Lim et al. 2023 ; Pidstrigach et al. 2023 ; Kerrigan et al. 2023a ) . Moreover, the same principle has been deployed to generalize flow matching ( Lipman et al. 2023 ) to functional spaces ( Kerrigan et al. 2023b ) , an approach closely related to our work. However, these works on learning generative models on function spaces do not support UFR the way GP-regression does because they (i) focus solely on generating function samples, (ii) do not clarify how to model a stochastic process on point value sequential generation, and (iii) do not provide point evaluation of probability density. Stochastic processes. Earlier works on SPL have focused on hand-tuned methods in the style of GP -regression. In these cases, an expert tunes the GP parameters given a set of experimental samples. More advanced methods rely on deep GP s, in which a network of GP s is stacked on top of each other. The parameters of deep GP s are commonly optimized by minimizing the variational free energy, which serves as a bound on the negative log marginal likelihood. ( Damianou & Lawrence 2013 ; Liu et al. 2020 ) . Deep GP s have limitations in terms of learnability, expressivity, and