Numerical study of high-dimensional covariance estimation and localization for data assimilation
Shay Gilpin · 2025
Read Numerical study of high-dimensional covariance estimation and localization for data assimilation by Shay Gilpin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-18299.txt.
Introduction
Shay Gilpin and co-authors test new ways to tame wide error maps in a 2025 paper. World reads, from storm lore to sea flow, feed past marks to a live sum; the weak link is the error map's wide span of posts.
The stock cure dams far pairs by their span on the map. New cures from the lore of counts shape the map by its own weave in place of span. Test worlds built to break the stock cure host the trials: one small wild sky, one two-sheet sea sky. All cures cut the miss by much. The stock cure won most rounds, if by a hair. Rich cures need more tune and more lore in hand. The authors say span still rules, for now.
Catalog record
| CREATOR | Shay Gilpin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-18299
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Numerical study of high-dimensional covariance estimation and localization for data assimilation Shay Gilpin Abstract Abstract Covariance localization is a critical component of ensemble-based data assimilation (DA) and many current localization schemes simply dampen correlations as a function of distance. Increases in computational resources, broadening scope of application for DA, and advances in general statistical methodology raise the question as to whether alternative localization methods may improve ensemble DA relative to current schemes. We carefully explore this issue by comparing distance based localization with alternative covariance localization techniques, partially those taken from the statistical literature. The comparison is done on test problems that we designed to challenge distance-based localization, including joint state-parameter estimation in a modified Lorenz ’96 model and state estimation in a two-layer quasi-geostrophic model. Across all sets of experiments, we find that while localization of any kind (with rare exceptions) can lead to significant reductions in error, traditional, distance-based localization generally leads to the largest error reduction. More general localization schemes can sometimes lead to greater error reduction, though the impacts may only be marginal and may require more tuning and/or prior information. Significance Statement Data assimilation, i.e., the fusion of observational measurements with computational models, is an essential step in numerical weather prediction. Covariance estimation — quantifying correlation between different measurements — is a key step in many data assimilation algorithms. Existing data assimilation methods often employ “localization,” which downplay correlations between distant spatial locations. In this work, we compare a standard localization method with a number of more general, alternative covariance estimation methods, in computer experiments designed to challenge the standard localization method. We find that with rare exceptions, most covariance estimation methods improve data assimilation performance. However, most alternate methods result in little or no improvement over standard localization, and the methods that show an improvement do so at an additional cost. 1 Introduction Data assimilation (DA) is a class of statistical estimation techniques that combine models with sparse and noisy observations. DA is a critical component of numerical weather prediction (NWP), and DA methods can be broadly separated into two groups: Monte Carlo based, ensemble Kalman methods (EnKF) ( Evensen, 1994 ; Burgers et al., 1998 ; Tippett et al., 2003 , e.g.,) and optimization-based/variational methods ( Talagrand and Courtier, 1987 ; Courtier and Talagrand, 1987 , e.g.,) . The most successful methods are “hybrids” that combine the Monte Carlo approach with optimization ( Hamill and Snyder, 2000 ; Lorenc, 2003 ; Zhang et al., 2009 ; Buehner et al., 2013 ; Kuhl et al., 2013 ; Poterjoy and Zhang, 2015 , e.g.,) . Collectively, we refer to EnKFs and hybrid ensemble/variational methods as “ensemble DA”. In ensemble DA, uncertainties are represented by a covariance matrix. For an ensemble of n_{e} members \mathbf{x}_{i}\in\mathbb{R}^{n_{x}} , i=1,2,\dots,n_{e} , the n_{x}\times n_{x} sample covariance matrix is \mathbf{S}=\frac{1}{n_{e}-1}\sum_{i=1}^{n_{e}}(\mathbf{x}_{i}-\overline{\mathbf{x}})(\mathbf{x}_{i}-\overline{\mathbf{x}})^{\text{T}},\quad\bar{\mathbf{x}}=\frac{1}{n_{e}}\sum_{i=1}^{n_{e}}\mathbf{x}_{i}~, (1) where \overline{\mathbf{x}} is the ensemble mean and superscript T denotes the transpose. Typically, the number of unknown state variables, n_{x} , is much larger than the ensemble size, n_{e} . In global NWP, for instance, n_{x} is on the order of 10^{9} and the ensemble size n_{e} is on the order of 10^{2} . The sample covariance is unbiased and consistent, but produces large errors when n_{e}\ll n_{x} ( Wainwright, 2019 , e.g.,) . Covariance localization aims to improve covariance estimates without increasing the number of ensemble members by tapering covariances as a function of distance ( Houtekamer and Mitchell, 2001 , e.g.,) . The justification for tapering is that atmospheric state variables decorrelate over large distances, which implies that large long-range sample covariances are likely spurious and due to sampling error. This source of error is exacerbated by the small ensemble size. Localization boosts the overall accuracy of covariance estimates by removing such spurious correlations at the cost of introducing a small bias. In atmospheric DA, it is usually implemented via the compactly-supported approximation to a Gaussian introduced by Gaspari and Cohn, 1999 , often referred to as the Gaspari-Cohn (GC) function. Covariance localization can be combined with “hybrid” estimators that linearly interpolate between the sample covariance and a time-averaged climatological covariance ( Bishop and Satterfield, 2013 ; Ménétrier and Auligné’, 2015 ; Satterfield et al., 2018 , e.g.,) . In the statistical literature, hybrid estimators are called “shrinkage” estimators (e.g., Ledoit and Wolf, 2004 ; Pourahmadi, 2013 and Section 2 ). Recent work on localization and covariance estimation also considers covariance corrections in the absence of a notion of distance ( Anderson, 2012 ) . Some schemes, for example, construct tapering matrices by raising correlations to a power, which we refer to as correlation-based localization ( Bishop and Hodyss, 2007 ; Bishop and Hodyss, 2009a ; Bishop and Hodyss, 2009b ; Lee, 2021 ; Vishny et al., 2024 ) . Much of the statistical literature is focused on thresholding techniques ( Fan and Li, 2001 ; Bickel and Levina, 2008b ; Rothman et al., 2009 ; Cai and Liu, 2011 ; Wainwright, 2019 ) . We discuss these ideas in more detail in Section 2 . We consider the question if more general localization or covariance estimations methods might improve ensemble DA compared to distance-based localization with the GC function. We take a computational approach and study this question by comparing several covariance estimation or localization techniques in the context of nonlinear cycling DA test problems of intermediate complexity. Specifically, we compare “traditional” distance-based localization, implemented via the GC function, with a more general localization scheme based on an inhomogeneous and anisotropic extension of the GC function ( Gilpin et al., 2023 ) . We further consider variants of correlation-based localization, thresholding, and hybrid estimators. The numerical experiments feature dynamical systems with intricate correlation structure (a modified Lorenz’96 model and a quasi-geostrophic flow) and joint state/parameter estimation where the notion of distance is not easily defined. In short, our experiments suggest that localization of any kind, with a few rare exceptions, already leads to a massive error reduction, with distance-based localization often resulting in the largest reduction. Alternative covariance estimation techniques may lead to additional error reduction compared to traditional localization, but overall only marginally, if at all. In this way, our experiments confirm the conclusions of a linear, non-cycling theory described by Morzfeld and Hodyss, 2023 ; Hodyss and Morzfeld, 2023 . The rest of this paper is organized as follows. Section 2 reviews the stochastic EnKF, which we use as a representative ensemble DA method in our experiments, as well as the various localization and covariance estimation methods used later in the paper. In Section 3 we compare the methods on a joint state-parameter estimation problem for a modified Lorenz ’96 model. Section 4 contains numerical results obtained with a two-layer quasi-geostrophic model. Both sets of numerical experiments feature spatially averaged observations and are designed to exhibit complex correlation structures. A summary and discussion of our results is provided in Section 5 . 2 Ensemble data assimilation, localization, and statistical covariance estimation We briefly review the stochastic EnKF, then review traditional, distance-based localization, hybrid covariance estimators/shrinkage, correlation-based localization, and thresholding. 2.1 Ensemble Kalman filters Ensemble Kalman filters (EnKFs) update a forecast ensemble using the well-known Kalman formalism. Let \mathbf{x}^{f}_{i}\in\mathbb{R}^{n_{x}} , i=1,2,\dots,n_{e} , be the forecast