Summary Statistics of Large-scale Model Outputs for Observation-corrected Outputs
Atlanta Chakraborty, Julie Bessac · 2025
Read Summary Statistics of Large-scale Model Outputs for Observation-corrected Outputs by Atlanta Chakraborty, Julie Bessac (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-15845.txt.
Introduction
Atlanta Chakraborty and Julie Bessac present a framework that corrects large physics model outputs with sparse real-world observations, in a 2025 paper. Physics models capture broad patterns in space and time but carry bias, while sensors catch local detail at few spots. Model outputs are also huge, which makes them hard to work with.
Their method, Sig-PCA, pairs summary statistics drawn from model output with local sensor data inside a neural network. Working with reduced summaries lets the network stay small. Tests on surface temperature and surface wind data show the corrected output tracks the sensors closely. The method can even fix the full spread of values and their links across space and time.
Catalog record
| CREATOR | Atlanta Chakraborty, Julie Bessac |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-15845
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Summary Statistics of Large-scale Model Outputs for Observation-corrected Outputs Atlanta Chakraborty, Julie Bessac Abstract Physics-based models capture broad spatial and temporal dynamics, but often suffer from biases and numerical approximations, while observations capture localized variability but are sparse. Integrating these complementary data modalities is important to improving the accuracy and reliability of model outputs. Meanwhile, physics-based models typically generate large outputs that are challenging to manipulate. In this paper, we propose Sig-PCA, a space-time framework that integrates summary statistics from model outputs with localized observations via a neural network (NN). By leveraging reduced-order representations from physics-based models and integrating them with observational data, our approach corrects model outputs, while allowing to work with dimensionally-reduced quantities hence with smaller NNs. This framework highlights the synergy between observational data and statistical summaries of model outputs, and effectively combines multisource data by preserving essential statistical information. We demonstrate our approach on two datasets (surface temperature and surface wind) with different statistical properties and different ratios of model to observational data. Our method corrects model outputs to align closely with the observational data, specifically enabling to correct probability distributions and space-time correlation structures. 1 Introduction Physics-based models, such as atmospheric models, typically simulate physical systems and are driven by mathematical representations of the system, which require computational treatment to output simulations. While they provide consistent outputs across space and time, they are often subject to model biases arising from approximations in physical processes, numerical approximations, and unresolved processes from parameterization schemes ( Danforth et al. 2007 ; Palmer et al. 2009 ; Christensen and Zanna 2022 ) . Their high-dimensional outputs create computational difficulties in movement, storage, analysis, and visualization, raising the needs for data compression Jian et al. 2024 , surrogates and dedicated analysis tools such as in-situ treatment ( Dutta et al. 2021 ) . In contrast, measurement observations capture localized, model-independent information, but are typically sparse in space and time and subject to measurement errors. Observational data are essential for validating and improving model outputs, improving their representations, and understanding local phenomena, especially in complex environments, where topography or local environments significantly influence patterns. To improve model outputs accuracy and realism, techniques to directly correct them have been proposed in various settings such as post-processing of forecasts ( Glahn and Lowry 1972 ; Raftery et al. 2005 ; Vannitsem et al. 2021 ) or bias correction ( Maraun 2019 ) . Bias correction aims at matching the statistical properties of physics-based model outputs to the observation ones. Statistical methods have been extensively used in that field. Quantile-matching and its extensions provide an empirical transformation from the model output distribution to the observational one ( Maraun 2016 ; Cannon 2018 ) . Similarly, statistical parametric models have been built to correct physics-based model outputs, leveraging various statistical features and interactions across data sources or variables, ( Bessac et al. 2018 ; Majumder et al. 2024 ) . However, including spatial, multivariate or multi-source information remains a statistical challenge, and general techniques are lacking due to limitations in statistical methods for complex dependencies and high-dimensional settings, ( Cressie and Johannesson 2008 ; Salvana and Genton 2020 ) . In a similar vein, downscaling enhances the resolution of model outputs when scales are mismatching while capturing finer-scale statistical properties. Recent advances in deep learning have lead to a profusion of literature in downscaling, from generative adversarial networks and their stochastic extension ( Stengel et al. 2020 ; Annau et al. 2023 ; Daust and Monahan 2024 ) to normalizing flows ( Groenke et al. 2021 ; McDonald et al. 2022 ; Bailie et al. 2024 ) . However, most of these methods require the entire gridded model outputs to learn corrective models. To address the challenge of handling large datasets, unsupervised dimension reduction techniques such as principal component analysis (PCA) ( Von Storch and Zwiers 2002 ; Hannachi et al. 2007 ) and its variants, such as non-linear ones ( Monahan 2000 ) , have been widely used to extract dominant spatial and temporal modes from large fields. Deep learning methods have also been employed in dimension reduction, such as variational autoencoders (VAE) providing a probabilistic low-dimensional embedding of complex high-dimensional fields, ( Kingma and Welling 2022 ) . However, their interpretation remains an active area of research, e.g. in disentangling and enhancing the interpretability of VAE latent space for scientific data ( Ganguli et al. 2024 ) . Meanwhile, implicit neural representations (INR) offer a complementary approach by encoding complex spatial and temporal fields as continuous functions parameterized by NN, as exemplified in ( Abdul Qayyum et al. 2024 ) in a wind velocity super-resolution context. In addition to dimension reduction, data compression such as error-bounded lossy compression ( Di et al. 2025 ) has gained significant traction for storage and movement of scientific data, including interests in atmospheric model outputs and their statistics ( Poppick et al. 2020 ; Underwood et al. 2022 ; Gong et al. 2023 ) . In the meantime, specific compression techniques for atmospheric data have been developed by Klöwer et al. 2021 and Huang and Hoefler 2023 based on information theory and NN representations respectively. In the statistical context, Guinness and Hammerling 2018 have proposed a statistical compression and decompression method that stores Fourier coefficients as essential summary statistics, along with a decompression statistical model describing the conditional distribution of the full dataset based on these summaries. In this paper, we propose a novel approach, called Sig-PCA, to integrate space-time reduced representations of numerical model outputs with observational data in order to correct model outputs. To the best of our knowledge, this is the first attempt to bias-correct physics-based model outputs from reduced representations of the former. Specifically, we propose a NN-based approach that inputs space-time summary statistics rather than the full-resolution of model outputs, and generates outputs matching measurement data, hence correcting the model outputs in space and time. By processing low-dimensional representations derived from model outputs with observations, we enable a computationally efficient integration of multisource data, while capturing meaningful patterns (probability distribution and space-time correlation structures) without handling the entire model output complexity. Path signatures are used to generate summary statistics in an unsupervised fashion, as opposed to deep-learning based dimension reduction models requiring training, hence ensuring more tractable computations and interpretability. Moreover, the proposed NN are inputted with deep kriging basis functions Chen et al. 2022 , allowing to extend the learned observation-based correction to a spatial neighborhood of the considered measurement stations. Finally, the method captures the spatiotemporal non-stationarity of the data via the summary statistics and deep-kriging. The paper is outlined as follows: Section 2 details the proposed methodology, beginning with an introduction to path signatures, followed by the presentation of the proposed NN-based corrective method, Sig-PCA. Section 3 presents the datasets (surface temperature and surface wind speed) comprised of model outputs and observations. Each data application presents a different type of model data (reanalysis and numerical weather prediction), a different ratio of model to observation data, and different statistical properties as temperature exhibits smoother spatiotemporal behaviors than surface wind speed. Sections 4 and 5 demonstrate the application of our approach to these datasets and provide statistical validation. Finally, Section 6