Imputing Missing Long-Term Spatiotemporal Multivariate Atmospheric Data with CNN-Transformer Machine Learning

Jiahui Hu, Wenjun Dong, Alan Z. Liu · 2025

Read Imputing Missing Long-Term Spatiotemporal Multivariate Atmospheric Data with CNN-Transformer Machine Learning by Jiahui Hu, Wenjun Dong, Alan Z. Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01141.txt.

Introduction

Gaps in atmospheric records, caused by equipment limits and harsh weather, hinder the study of how the air moves and changes. Jiahui Hu, Wenjun Dong, and Alan Z. Liu propose CT-MVP, a learning model that fills in missing values across place, time, and height. It handles several variables at once, including temperature and winds.

The model joins convolutional networks, which pick up local patterns, with transformers, which track links over long spans of time and height. Training used a thirteen-year data set from a whole-atmosphere climate model with global coverage. Tests under varied gap patterns check how well the filled values hold up. Hu and the team aim for complete records that support stronger analysis of atmospheric processes.

Catalog record

CREATORJiahui Hu, Wenjun Dong, Alan Z. Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-01141

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Imputing Missing Long-Term Spatiotemporal Multivariate Atmospheric Data with CNN-Transformer Machine Learning Jiahui Hu, Wenjun Dong, Alan Z. Liu Abstract Abstract Continuous physical domains are important for scientific investigations of dynamical processes in the atmosphere. However, missing data—arising from operational constraints and adverse environmental conditions—pose significant challenges to accurate analysis and modeling. To address this limitation, we propose a novel hybrid Convolutional Neural Network (CNN)–Transformer machine learning model for multivariable atmospheric data imputation, termed CT-MVP. This framework integrates CNNs for local feature extraction with transformers for capturing long-range dependencies across time and altitude. The model is trained and evaluated on a testbed using the Specified Dynamics Whole Atmosphere Community Climate Model with thermosphere and ionosphere extension (SD-WACCM-X) dataset spanning 13 years, which provides continuous global coverage of atmospheric variables, including temperature and zonal and meridional winds. This setup ensures that the ML approach can be rigorously assessed under diverse data-gap conditions. The hybrid framework enables effective reconstruction of missing values in high-dimensional atmospheric datasets, with comparative evaluations against traditional methods and a simple transformer. The results demonstrate that CT-MVP achieves superior performance compared with traditional approaches, particularly in cases involving extended periods of missing data, and slightly outperforms a simple transformer with the same hyper-parameters. Plain Language Scientists need continuous data to study how the atmosphere behaves, but real-world measurements often contain gaps due to instrument limitations or poor observing conditions. To address the challenge, we developed a hybrid machine learning model called CT-MVP. The model combines two powerful techniques: convolutional neural networks, which capture local patterns, and transformers, which learn long-term trends across time and altitude. We trained and tested CT-MVP on 13 years of global atmospheric data from the state-of-the-art numerical atmospheric model, which provides complete and continuous simulations. This setup allowed us to robustly evaluate the model under diverse conditions. Our results show that CT-MVP outperforms traditional methods, particularly when large sections of data are missing, which indicates machine learning can be a promising tool for reconstructing atmospheric datasets. 1. Introduction The Earth’s atmosphere is a highly dynamic system, with interactions between different layers driving key processes that regulate climate and space weather Barry & Chorley (2009) . These interactions occur across a wide range of spatial and temporal scales, influencing large-scale circulation, energy transfer, and variability in the upper atmosphere. Continuous atmospheric datasets are essential for enhancing our understanding of coupled dynamical processes across atmospheric layers. However, long-term observational records often contain substantial gaps due to operational faults, sensor malfunctions, adverse environmental conditions, or under-sampling. These missing values pose challenges to the statistical analysis, introducing bias in physical interpretation, and degrading downstream applications such as forecasting and data assimilation. Traditional approaches, such as linear interpolation methods that estimate missing values from adjacent time steps, are only effective for imputing short gaps Betancourt et al. (2023) . More advanced interpolation-based techniques, including kriging and polynomial fitting, remain widely used in environmental science because of their simplicity and interpretability Larson et al. (2023) . Other Ensemble-based or advanced statistical approaches can outperform classical interpolation when applied to long-term ecological datasets. However, the ensemble methods are built based on smoothness or locality assumptions, and often fail when faced with long missing intervals or nonlinear cross-variable dependencies, where atmospheric processes are strongly coupled and variability spans multiple scales across altitude and time. Recent advances in Machine Learning (ML) offer alternative solutions to the data gap challenges across a variety of applications Platias & Petasis (2020) ; Emmanuel et al. (2021) ; Teegavarapu (2024) . ML-based imputation methods such as random forests and k-Nearest Neighbors (kNN) have been applied to meteorological datasets, showing greater effectiveness than traditional methods when the proportion of missing data is high Doreswamy & Manjunatha (2017) . Approaches that combine dense layers with convolutional neural network (CNN) have further improved performance by capturing temporal dependencies and spatial patterns across multiple missing variables (e.g., temperature, wind speed, precipitation) measured at stations from the National Climatic Data Center (NCDC). More recently, transformer-based models have demonstrated strong capabilities in capturing long-range dependencies. For instance, Ayub & Jamil (2024) applied a transformer‑based model to univariate time‑series data aggregated at hourly, daily, and monthly frequencies, achieving significant improvements over classical approaches such as mean and KNN imputation. Innovations such as missing-position encoding Wi et al. (2024) and heterogeneous node embeddings have been specifically designed to improve imputation in multivariate, irregularly spaced time series. By leveraging the capabilities of neural networks to learn both local features and long-range dependencies, advanced techniques that combine different ML architectures can achieve higher accuracy and robustness. The integration of meteorological factors into pollution prediction models has highlighted the value of incorporating domain-specific characteristics to improve imputation performance, as demonstrated by the CNN–LSTM hybrid model for airborne particle forecasting Samal et al. (2021) . Furthermore, frameworks that integrate convolutional layers with transformer-based architectures have shown superior performance in reconstructing missing air quality datasets Cui et al. (2023) . Wang et al. Wang et al. (2025) developed a CNN–Transformer model with a customized loss function to predict high-resolution PM_{2.5} from sparse mobile monitoring data, demonstrating the feasibility of fusing local spatial filters with long-range temporal attention for environmental prediction. Similarly, Hou et al. Hou et al. (2025) proposed a CNN–Transformer model to interpolate missing meteorological variables on the Tibetan Plateau, reporting that the hybrid design significantly outperformed conventional machine learning and statistical interpolation methods. These studies underscore the growing recognition that hybrid architectures can balance fine-scale feature extraction with long-range dependency modeling. However, most prior work has focused on either urban pollutant mapping or single-site meteorological interpolation, often with limited variables or restricted spatiotemporal coverage. Inspired by recent ML methods for meteorological data imputation, we develop a hybrid CNN–Transformer framework tailored to the time–altitude domain. The model combines convolutional encoders for local feature extraction with transformer layers enhanced by rotary embeddings Su et al. (2024) , which is expected to improve the capture of long-range temporal–vertical dependencies. Our task is distinct as it targets multivariate atmospheric data (e.g., temperature, zonal and meridional wind), where gaps impact not only statistical fidelity but also the physical interpretability of wave propagation and mesospheric tides. This design ensures that reconstructions are not only numerically accurate but also scientifically meaningful to preserve gradients, oscillations, and variability that underpin physical coupling across atmospheric layers. Because spatiotemporal observations for training are not sufficient to assess model performance, we use the Specified Dynamics Whole Atmosphere Community Climate Model with thermosphere and ionosphere extension (SD-WACCM-X) as a controlled testbed, where artificial gaps are introduced to mimic realistic observational gaps of varying duration. This setup allows us to robustly test model skill in reconstructing missing multi-variate atmospheric data. We benchmark CT-MVP against widely used linear interpolation method and advanced statistical imputation approaches, including Rauch–Tung–Striebel (RTS) Kalman filtering and smoothing Särkkä (2008) and Principal Component Analysis (PCA) Abdi & Williams (2010) , as well as a simple transformer Vaswani et al. (2017) . The results demonstrate that CT-MVP achieves higher reconstruction accuracy than traditional methods, slightly better than a simple transformer, particularly in extended-gap cases. These findings highlight the potential of machine learning frameworks for atmospheric data imputation, with strong prospects for application to real-world observational records. 2. CNN-Transformer Multi-Variable imPutation

Read the complete text