Tokenised Flow Matching for Hierarchical Simulation Based Inference
Giovanni Charles, Cosmo Santoni, Seth Flaxman, Elizaveta Semenova · 2026
Read Tokenised Flow Matching for Hierarchical Simulation Based Inference by Giovanni Charles, Cosmo Santoni, Seth Flaxman, Elizaveta Semenova (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20723.txt.
Introduction
Charles, Santoni, Flaxman, and Semenova address the cost of simulator runs, a key block for SBI. They explore likelihood factorisation to train from single-site runs. In likelihood factorisation sampling, they learn a per-site stand-in and assemble multi-site samples to speed inference for the posterior. Building on this, they propose TFMPE, aiding functional outputs. They offer a benchmark for SBI. They test TFMPE on this benchmark and on disease and fluid models, finding sound results while cutting cost.
Catalog record
| CREATOR | Giovanni Charles, Cosmo Santoni, Seth Flaxman, Elizaveta Semenova |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76847 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.20723
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Tokenised Flow Matching for Hierarchical Simulation Based Inference Giovanni Charles, Cosmo Santoni, Seth Flaxman, Elizaveta Semenova Abstract The cost of simulator evaluations is a key practical bottleneck for Simulation Based Inference (SBI). In hierarchical settings with shared global parameters and exchangeable site-level parameters and observations, this structure can be exploited to improve simulation efficiency. Existing hierarchical SBI approaches factorise the posterior yet still simulate across multiple sites per training sample; We instead explore likelihood factorisation (LF) to train from single-site simulations. In LF sampling we learn a per-site neural surrogate of the simulator and then assemble synthetic multi-site observations to amortise inference for the full hierarchical posterior. Building on this, we propose Tokenised Flow Matching for Posterior Estimation (TFMPE), a tokenised flow matching approach that supports function-valued observations through likelihood factorisation. To enable systematic evaluation, we introduce a benchmark for hierarchical SBI. We validate TFMPE on this benchmark and on realistic infectious disease and computational fluid dynamics models, finding well-calibrated posteriors while reducing computational cost. 1 Introduction Simulation Based Inference (SBI) enables Bayesian parameter estimation when likelihoods are intractable but simulators are available. Rather than requiring closed-form density functions, SBI methods train neural networks on simulated parameter-observation pairs to approximate posteriors directly. This makes SBI particularly well suited for calibrating complex scientific simulators where traditional MCMC methods are impractical. Many scientific applications naturally exhibit hierarchical structure, where observations arise from multiple related settings that share common properties. A hierarchical model captures this by partitioning parameters into global parameters, governing dynamics common across all settings, and local parameters, conditioning dynamics to specific observations. For example, fitting a disease transmission model to data from multiple countries requires global parameters for pathogenic properties like disease severity and local parameters for country-specific control policies 11 . In hierarchical SBI, both the number of parameters and the simulation cost required for inference grow with the number of local settings, making simulation cost a key practical bottleneck. Notation. In Bayesian Inference, we estimate a posterior distribution over some parameters \theta given observational data y : p(\theta|y)\propto p(y|\theta)p(\theta) . In Simulation Based Inference, we assume that there is no closed form solution for p(y|\theta) , however we do have a simulator at our disposal which can generate y given \theta . We consider a two-level hierarchical model where \theta is partitioned into global parameters \theta_{g}\in\mathbb{R}^{d_{g}} and local parameters \eta=(\eta_{1},\ldots,\eta_{n_{s}}) where each \eta_{s}\in\mathbb{R}^{d_{l}} parameterises site s\in\{1,\ldots,n_{s}\} . A site corresponds to a single local setting, for example a country in epidemiological modelling or a patient in clinical applications. The simulator generates observations y_{s}\in\mathbb{R}^{d_{y}} for each site s . We denote the complete set of observations across all sites as y=(y_{1},\ldots,y_{n_{s}}) . In the common situation in which the global parameters serve as the prior for local parameters, we obtain the distribution: p(\theta_{g},\eta)=p(\theta_{g})\prod_{s=1}^{n_{s}}p(\eta_{s}|\theta_{g}) . The complete hierarchical posterior thus takes the form p(\eta,\theta_{g}\mid y)\propto p(\theta_{g})\prod_{s=1}^{n_{s}}p(\eta_{s}\mid\theta_{g})\,p(y_{s}\mid\eta_{s}). (1) Note that this is the same hierarchical posterior proposed by 13 . Amortised posterior estimators are trained on a dataset of training samples \mathcal{D}=\{(\theta^{(i)},y^{(i)})\}_{i=1}^{N} to learn an approximate posterior q_{\phi}(\theta|y) . In the hierarchical setting, generating each training sample requires simulating all n_{s} sites to obtain the complete observation vector y=(y_{1},\ldots,y_{n_{s}}) . 1.1 Factorised Inference Posterior Factorisation Training sample \theta_{g} y_{1:n_{\max}} \eta_{1:n_{\max}} \theta_{g} y_{1:n_{\max}} Global Estimator ) y \mid \theta_{g} q( Local Estimator q(\eta\mid\theta_{g},y) \hat{\theta}_{g} y_{1:n_{\max}} \eta_{1:n_{\max}} Likelihood Factorisation \theta_{g} \eta_{1} y_{1} Training sample \theta_{g} \eta_{1} y_{1} Per-site Neural Surrogate q( y_{1} \mid \eta_{1} , \theta_{g} ) \theta_{g} \eta_{1:n_{s}} \hat{y}_{1:n_{s}} Posterior Estimator q(\theta_{g},\eta\mid y) p(\eta\mid\theta_{g}) Figure 1 : Comparison of posterior factorisation and likelihood factorisation approaches for hierarchical SBI. Left: Posterior-factorisation methods train from observation subsets of size at most n_{\max} , so each training sample requires n_{\max} simulator calls (with n_{\max}=1 for fully-factorised methods and 1<n_{\max}<n_{s} for partially-factorised variants). Error in the estimated global posterior can then propagate through \hat{\theta}_{g} to local estimation. Right: Likelihood factorisation trains the per-site neural surrogate q_{\phi_{l}} from exactly one simulator call per training sample, using single-site tuples (\theta_{g},\eta_{s},y_{s}) , and then generates synthetic multi-site observations ( \hat{y} ) for training a separate posterior estimator q_{\phi_{p}} . Existing factorised inference approaches factorise the posterior directly, requiring multiple simulations per training sample. When observations can be assumed i.i.d., the posterior factorises as p(\theta\mid y)\propto p(\theta)^{1-n_{s}}\prod_{s=1}^{n_{s}}p(\theta\mid y_{s}), (2) obtained by applying Bayes’ rule twice (see Appendix A.1 ). Below we summarise the most common applications of Equation 2 to Simulation Based Inference. Compositional inference. Many posterior estimation workflows exploit the factorisation in Equation 2 to train estimators on datasets of parameter-observation pairs (\theta,y_{s}) , enabling inference over arbitrarily sized observation sets ( 28 ; 4 ; 12 ) . Summary networks. Some compositional methods use permutation-invariant summary networks, such as Deep Sets ( 31 ) , to aggregate variable-sized observation sets into fixed-dimensional representations for posterior estimation. These approaches typically train by uniformly sampling observation subsets up to size n_{s} , requiring on average \nicefrac{{n_{s}}}{{2}} simulations per training sample. Factorised score estimation. Alternative compositional approaches decompose the posterior score over observation subsets and compose individual scores during sampling via annealed Langevin dynamics ( 12 ) . Fully-Factorised Neural Posterior Score Estimation (F-NPSE) trains on single observations, requiring one simulation per sample, while Partially-Factorised variants (PF-NPSE) train on subsets of size n_{\max} , requiring n_{\max} simulations per sample where 1<n_{\max}<n_{s} . Increasing n_{\max} reduces approximation error from score composition at the cost of sample efficiency. Hierarchical SBI. When applied to hierarchical models, recent methods have introduced separate estimators for global and local parameters to reduce the number of simulations required for posterior estimation ( 2 ; 17 ; 16 ; 29 ) . In these approaches, a global estimator is trained from multiple local samples to approximate p(\theta_{g}|y) , and a local estimator is trained on singular local samples to approximate p(\eta_{s}|\theta_{g},y_{s}) . 1.2 Tokenised SBI Tokenised SBI embeds the structure of probabilistic models into sequences of tokens, enabling flexible handling of function-valued observations. SimFormer ( 15 ) introduced this approach for posterior estimation, representing parameters and observations as tokens with identifiers, positional encodings, and conditioning information. We adopt tokenisation for its ability to model functional observations, data recorded at irregular, variable, or missing input coordinates, a useful technique for any application with a spatial or temporal aspect. Importantly, tokenisation applies equally to likelihood and posterior estimation, making it applicable to factorised inference strategies. Alongside the sampled value for each parameter, a tokenised estimator uses tokens which include parameter identifiers, functional inputs, dependence and conditioning information. This comes with two key benefits: (1) functional observations can be modelled to flexibly handle irregular, missing, or variable-length data, and (2) the same tokenisation scheme applies to both likelihood and posterior estimation. 1.3 Flow Matching for Posterior Estimation Flow matching trains continuous normalising flows by learning vector fields that transport samples from a base distribution to a target ( 23 ) . 6 adapted this framework to posterior estimation, learning a flow from a base distribution to p(\theta|y) via the Conditional Flow Matching objective: \mathcal{L}_{\text{FMPE}}(\phi;\theta|y)=\mathbb{E}_{t,p(\theta),p(y|\theta),p_{t}(\theta|\theta_{1})}||v_{t}(\phi,\theta_{t},y)-u_{t}(\theta_{t}|\theta_{1})||^{2} Flow matching imposes minimal architectural constraints: the learned vector field v_{t} need only be Lipschitz continuous in \theta and continuous in t for uniqueness ( 5 ) . Despite this flexibility, the continuity equation guarantees density estimation in