Fisher Score Matching for Simulation-Based Forecasting and Inference
Ce Sui, Shivam Pandey, Benjamin D. Wandelt · 2025
Read Fisher Score Matching for Simulation-Based Forecasting and Inference by Ce Sui, Shivam Pandey, Benjamin D. Wandelt (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-07833.txt.
Introduction
Ce Sui, Shivam Pandey, and Benjamin Wandelt present a way to estimate a key score used in forecasting, in a 2025 paper. The Fisher score tells how much data can teach us about a model's settings. It is normally computed from a formula for the data's likelihood, which many simulators lack. Their method learns the score from examples instead, using a neural network trained with a matching method.
A hidden-variable trick makes the learning task simple: the network just predicts scores for the hidden settings. Tests on a toy model with known truth show the learned scores match well. A cosmology simulator that can be differentiated gives the same close match. The authors note the method can work even when a simulator cannot be differentiated. That opens gradient-based forecasts and inference to a wider set of models.
Catalog record
| CREATOR | Ce Sui, Shivam Pandey, Benjamin D. Wandelt |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-07833
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fisher Score Matching for Simulation-Based Forecasting and Inference Ce Sui, Shivam Pandey, Benjamin D. Wandelt Abstract We propose a method for estimating the Fisher score—the gradient of the log-likelihood with respect to model parameters—using score matching. By introducing a latent parameter model, we show that the Fisher score can be learned by training a neural network to predict latent scores via a mean squared error loss. We validate our approach on a toy linear Gaussian model and a cosmological example using a differentiable simulator. In both cases, the learned scores closely match ground truth for plausible data-parameter pairs. This method extends the ability to perform Fisher forecasts, and gradient-based Bayesian inference to simulation models, even when they are not differentiable; it therefore has broad potential for advancing cosmological analyses. 1 Introduction Statistical inference is a core component of modern astronomical and cosmological research. A typical inference pipeline involves several key stages: identifying informative observables, constructing summary statistics, and performing parameter inference. A wide array of tools has been developed to address each of these tasks. Decades of research on cosmological structure have centered on defining informative summaries and observables, principally amongst them the power spectrum. New surveys, such as those focusing on galaxy clustering, 21cm intensity mapping, and weak lensing ( DES Collaboration et al. 2025 ; Euclid Collaboration et al. 2024 ; Square Kilometre Array Cosmology Science Working Group et al. 2020 ) motivate more sophisticated summaries ( Beyond-2pt Collaboration et al. 2024 ; Makinen et al. 2024 ; Charnock et al. 2018 ; Cheng et al. 2020 ) . These include both hand-crafted features and machine learning-based methods, aimed at extracting the maximal amount of information from increasingly complex data. To forecast constraints based on these observables and summaries, the Fisher information matrix has been widely used after being introduced to cosmology by Tegmark et al. 1997 . Given observational data summary statistics and a physical model, the next step is parameter inference. Most current cosmological analyses use Bayesian inference ( e.g. , DES Collaboration et al. 2025 ; Planck Collaboration et al. 2020 ). Traditionally, this involves assuming an explicit form for the likelihood and using MCMC algorithms for posterior sampling. Numerous MCMC packages have been developed to support this ( Torrado & Lewis 2021 ; Brinckmann & Lesgourgues 2018 ) . More recently, simulation-based inference (SBI) ( Cranmer et al. 2020 ; Alsing et al. 2019 ) has allowed approximating the likelihood implicitly using simulations. SBI methods can be more flexible and efficient than standard MCMC, and many recent tools implement SBI techniques to address a wide range of cosmological problems ( Ho et al. 2024 ; Boelts et al. 2025 ; Zhao et al. 2022 ; Lemos et al. 2023 ) . All of these steps—summary construction, Fisher forecasting, and inference—can be greatly simplified if we have access to the Fisher score function: the gradient of the log-likelihood with respect to the model parameters. The Fisher score function itself is a sufficient statistic, since it encodes the likelihood surface up to a constant factor. When evaluated at a fixed, fiducial parameter value, it serves as a locally sufficient statistic ( Heavens et al. 2017 ; Alsing & Wandelt 2018 ) and enables straightforward computation of the Fisher matrix. It serves to construct unbiased, minimum-variance estimators that saturate the Cramér-Rao information inequality, and to compute the maximum likelihood estimator. Furthermore, it opens the door to using more efficient gradient-based approaches such as Hamiltonian Monte Carlo (HMC) samplers; or to frequentist analysis avoiding prior choice, such as maximum likelihood estimators. This surprising utility of the Fisher score has motivated recent efforts to re-implement cosmological simulations using differentiable programming frameworks like JAX 1 1 1 http://github.com/jax-ml/jax ( Bradbury et al. 2018 ) that enable automatic differentiation throughout the simulation pipeline and make Fisher score computation feasible ( Li et al. 2022 ; Campagne et al. 2023 ) . In parallel, score matching—a technique used to train neural networks to approximate not the Fisher score, but the probability score \nabla_{x}\log P(x) —has seen widespread success in generative modeling, most notably in diffusion models ( Song & Ermon 2019 ; Song et al. 2020 ) . Similar techniques have also been adapted to the context of SBI and cosmological inference tasks ( Sharrock et al. 2022 ; Cuesta-Lazaro & Mishra-Sharma 2024 ; Mudur et al. 2025 ) . Here, we propose a new score matching approach for learning the Fisher score directly from simulations. We demonstrate that the learned scores are accurate for data-parameter pairs relevant during inference and can be used for both Fisher analysis and Bayesian inference. Section 2 describes our method, Section 3 presents experimental results, and Section 4 concludes with a discussion of future directions. 2 Method Given a probabilistic model P(x\mid\theta) describing a physical system, a key quantity of interest is the Fisher score, defined as: s(x,\theta)=\nabla_{\theta}\log P(x\mid\theta). (1) However, in many cases the likelihood P(x\mid\theta) is only implicitly defined through a simulator, without an analytical form or differentiable implementation, making direct computation of the Fisher score intractable. We propose a method to estimate the Fisher score using score matching. 2.1 Fisher Score Matching We assume the existence of a latent variable \theta^{*} such that the following Markov chain holds: \theta\rightarrow\theta^{*}\rightarrow x . That is, P(x\mid\theta)=\int P(x\mid\theta^{*})P(\theta^{*}\mid\theta)\,d\theta^{*}, where P(\theta^{*}\mid\theta) defines the latent model. Then we can re-write the Fisher score of the full model as an expectation of the latent model’s score as follows: \displaystyle\nabla_{\theta}\log P(x|\theta) \displaystyle=\frac{\nabla_{\theta}P(x|\theta)}{P(x|\theta)} \displaystyle=\frac{\nabla_{\theta}\int P(x|\theta^{*},\theta)P(\theta^{*}|\theta)d\theta^{*}}{P(x|\theta)} \displaystyle=\frac{\int P(x|\theta^{*})\nabla_{\theta}P(\theta^{*}|\theta)d\theta^{*}}{P(x|\theta)} \displaystyle=\int\frac{P(x|\theta^{*})P(\theta^{*}|\theta)}{P(x|\theta)}\nabla_{\theta}\log P(\theta^{*}|\theta)d\theta^{*} \displaystyle=\int P(\theta^{*}|x,\theta)\nabla_{\theta}\log P(\theta^{*}|\theta)d\theta^{*} \displaystyle=E_{P(\theta^{*}|x,\theta)}[\nabla_{\theta}\log P(\theta^{*}|\theta)]. (2) Since posterior means can be estimated by minimizing mean squared error (see Appendix A ), we can train a score estimator s(x,\theta) by minimizing the loss L=E_{p(x,\theta,\theta^{*})}[(s(x,\theta)-\nabla_{\theta}\log P(\theta^{*}|\theta))^{2}] (3) Training proceeds as follows: we sample from the joint distribution p(x,\theta,\theta^{*}) using forward simulations. For each triplet (x,\theta,\theta^{*}) , the model is asked to predict the latent score \nabla_{\theta}\log p(\theta^{*}\mid\theta) given the input (x,\theta) . Once training converges, the estimator s(x,\theta) provides a deterministic approximation of the true Fisher score \nabla_{\theta}\log p(x\mid\theta) . Since the model is essentially performing a regression task, we expect its scalability with input dimension to follow typical patterns observed in regression problems, depending on the architecture used to learn the score. 2.2 Decomposable and Indecomposable Models If we can identify a latent variable such that the Markov chain \theta\rightarrow\theta^{*}\rightarrow x holds and the gradient \nabla_{\theta}\log P(\theta^{*}\mid\theta) is tractable, this approach can be applied directly, a fact also exploited by Brehmer et al. 2020 . A simple example is the linear Gaussian model x\mid\theta\sim\mathcal{N}(\theta,\Sigma_{n}) . We can decompose this into two conditional models \theta^{*}\mid\theta\sim\mathcal{N}(\theta,\Sigma_{\theta^{*}}),\quad x\mid\theta^{*}\sim\mathcal{N}(\theta^{*},\Sigma_{n}-\Sigma_{\theta^{*}}), (4) where both covariance matrices are positive definite. The Fisher score of the full model can then be learned using only the score from the latent model. In this case, the choice of decomposition does not affect the result—the training objective will still converge to the true Fisher score of the full model. To address more complex settings where no natural latent variable is available, we introduce an auxiliary variable \tilde{\theta} and define a latent model P(\theta\mid\tilde{\theta}) , forming the Markov chain \tilde{\theta}\rightarrow\theta\rightarrow x . We simulate samples using P(\theta,x\mid\tilde{\theta})=P(x\mid\theta)P(\theta\mid\tilde{\theta}), and apply the same score matching technique to