WeatherMesh-3: Fast and accurate operational global weather forecasting

Haoxing Du, Lyna Kim, Joan Creus-Costa, Jack Michaels, Anuj Shetty, Todd Hutchinson, Christopher Riedel, John Dean · 2025

Read WeatherMesh-3: Fast and accurate operational global weather forecasting by Haoxing Du, Lyna Kim, Joan Creus-Costa, Jack Michaels, Anuj Shetty, Todd Hutchinson, Christopher Riedel, John Dean (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-22235.txt.

Introduction

Haoxing Du and seven co-authors present WeatherMesh-3, a machine learning system for global weather forecasts, in a twenty twenty-five paper. Classic weather models are slow and costly to run. WeatherMesh-3 uses a transformer, the same broad design as large language models, tuned for weather data.

Two ideas drive the gains. A latent rollout lets the model roll a forecast forward in its inner space, for any length, with no need to decode each step. A modular build blends many real-time weather analyses into the start state. The system makes a fourteen-day global forecast in twelve seconds on one consumer GPU, and beats standard models on error.

Catalog record

CREATORHaoxing Du, Lyna Kim, Joan Creus-Costa, Jack Michaels, Anuj Shetty, Todd Hutchinson, Christopher Riedel, John Dean
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-22235

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

WeatherMesh-3: Fast and accurate operational global weather forecasting Haoxing Du, Lyna Kim, Joan Creus-Costa, Jack Michaels, Anuj Shetty, Todd Hutchinson, Christopher Riedel, John Dean Abstract We present WeatherMesh-3 (WM-3), an operational transformer-based global weather forecasting system that improves the state of the art in both accuracy and computational efficiency. We introduce the following advances: 1) a latent rollout that enables arbitrary-length predictions in latent space without intermediate encoding or decoding; and 2) a modular architecture that flexibly utilizes mixed-horizon processors and encodes multiple real-time analyses to create blended initial conditions. WM-3 generates 14-day global forecasts at 0.25-degree resolution in 12 seconds on a single RTX 4090. This represents a > > 100,000-fold speedup over traditional NWP approaches while achieving superior accuracy with up to 37.7% improvement in RMSE over operational models, requiring only a single consumer-grade GPU for deployment. We aim for WM-3 to democratize weather forecasting by providing an accessible, lightweight model for operational use while pushing the performance boundaries of machine learning-based weather prediction. 1 Introduction Numerical Weather Prediction (NWP) systems are fundamental to modern society, providing critical guidance for public safety, economic planning, and climate adaptation. However, operating NWP systems poses a high computational cost, particularly for developing regions and smaller organizations seeking to deliver essential real-time forecasting products. Recent advances in machine learning-based weather prediction (MLWP) have demonstrated the potential to alter this paradigm ( Kurth et al. 2023 ; Chen et al. 2023 ; Price et al. 2023 ; Lang et al. 2024a ) . Models such as GraphCast or Aurora have shown that neural architectures can match or exceed the accuracy of traditional NWP systems while reducing computational requirements by several orders of magnitude ( Lam et al. 2023 ; Bodnar et al. 2024 ) . However, current AI approaches still face several limitations for operational use. First, despite promising results, MLWP models must continue to demonstrate improved forecast skill relative to operational NWP systems in real-time settings. Second, these models often produce excessively blurred forecasts, a direct consequence of architectural and training decisions that introduce accumulating error and can limit operational utility. Third, while less computationally demanding than traditional NWP, MLWP systems still require substantial computational resources and expertise to operate, creating barriers to widespread adoption. We present WeatherMesh-3 (WM-3), an operational transformer-based global forecasting system that introduces advances to directly mitigate these limitations. WM-3 dramatically reduces both hardware requirements and operational complexity while improving state-of-the-art MLWP performance. 2 Methods 2.1 Model architecture WeatherMesh-3 takes a single state of global weather as input to predict the state of global weather any integer number of hours later. It represents the weather state on a latitude-longitude grid at 0.25 degree resolution, with surface and atmospheric variables at each grid point. Like Pangu-Weather ( Bi et al. 2023 ) , WM-3 makes heavy use of the vision transformer architecture ( Dosovitskiy 2020 ) , and treats pressure level as a third dimension. See Appendix A.2 for details on the model architecture. Encoder-processor-decoder architecture WM-3 consists of three high-level components: encoder, processor, and decoder. The encoder has convolution layers that map the weather state from the 0.25-degree grid to a learned, lower-resolution (2-degree) latent space. The processor then acts on the latent space any number of times, with each processor step corresponding to either one or six hours of forward time evolution. Finally, a decoder with deconvolution layers translates the latent space back into a 3D grid in physical space. A schematic of the model architecture is shown in Figure 1 . Figure 1: (a) A schematic of the encoder-processor-decoder architecture. (b) Illustration of the difference in attention window location between SWIN and NATTEN. WM-3 uses a latent rollout to produce a forecast for an arbitrary integer number of hours of forecast lead time in a single forward pass. It achieves this by invoking a combination of the six-hour and one-hour processors repeatedly until the target lead time, using a simple greedy schedule. This avoids the typical MLWP rollout method of encoding a weather state, rolling forward six hours, decoding to the physical space, then re-encoding to take another six hour step forward. Keeping the atmospheric state in latent space between rollouts has three main advantages: 1) avoids extraneous encoding and decoding compute or error accumulation processes; 2) enables mixed-horizon prediction tasks in training (see Section 2.2 ); 3) allows modularity in every model component by working with a shared latent space. NATTEN The backbone of the WM-3 processors is a series of neighborhood attention-based transformer (NATTEN) blocks ( Hassani et al. 2023 ) . In earlier versions of WeatherMesh, we experimented with the SWIN architecture ( Liu et al. 2021 ) , and found that NATTEN significantly improves WM-3 performance. The neighborhood attention mechanism provides a better inductive bias to the model for learning atmospheric physics due to its consistent locality of attention for information transfer between patches (Figure 1 b). In addition, thanks to the performance of the NATTEN library with Fused Neighborhood Attention kernels ( Hassani et al. 2024 ) , NATTEN is faster and has a lower memory footprint than our prior SWIN implementations. 2.2 Training Pre-training WeatherMesh-3 undergoes a two-stage training procedure: pretraining on ERA-5 reanalysis data, then fine-tuning on analysis data to create an operational version for real-time use. WM-3 is pretrained on ERA-5, ECMWF’s reanalysis dataset, from 1979 to 2022, with 2020 left out of the training data for testing per Rasp et al. 2024 . The six-hour processor is trained first to predict the state 6 and 12 hours later. The target timestep is then lengthened up to 120 hours (5 days, or 20 calls to the six-hour processor) according to a schedule. Once the six-hour processor is trained, we freeze the weights in the encoder and decoder and train a one-hour processor. We use Distributed Shampoo ( Shi et al. 2023 ) for training on multiple GPUs. See Appendix A.3 for details on the training setup and A.5 for compute optimizations. Operationalization In order to make WM-3 suitable for operational use, we replace the pretrained encoder with two new encoders that ingest analyses from European Centre for Medium-Range Weather Forecasts (ECMWF)’s Integrated Forecasting System (IFS) and the National Oceanic and Atmospheric Administration (NOAA)’s Global Forecast System (GFS). See Figure 4 for details. WeatherMesh has been running operationally since March 2024. 3 Results Accuracy Operational WM-3 achieves high accuracy relative to current operational models. To evaluate WM-3, we calculate latitude-weighted RMSE scores for our operational forecasts against ERA-5 and compare with IFS HRES over our evaluation period in 2024 (see Appendix A.4 ). In Figure 2 , we present the evaluation scorecards of WM-3. WM-3 outperforms HRES on all but one of 690 targets evaluated (geopotential at 50 hPa at 10 days), with a notable 37.7% improvement over HRES on 2-meter temperature at 1 day lead time. Figure 2: (a) Scorecard comparing WM-3 vs. IFS HRES across all pressure levels and surface variables. (b) Scorecard comparing WM-3 vs. AIFS. MLWP models outperforming HRES on RMSE should be evaluated with the perspective that HRES has not been optimized for RMSE as a metric. While it is common to compute RMSE for HRES against its own analysis in literature ( Lam

Read the complete text