DEF: Diffusion-augmented Ensemble Forecasting Journal: UNSURE
David Millard, Arielle Carr, Stéphane Gaudreault, Ali Baheri · 2025
Read DEF: Diffusion-augmented Ensemble Forecasting Journal: UNSURE by David Millard, Arielle Carr, Stéphane Gaudreault, Ali Baheri (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07324.txt.
Introduction
Weather forecasts made by neural networks give one answer, but forecasters also need a spread of likely outcomes. David Millard and co-authors present a method that adds that spread with a diffusion model. The model creates varied starting conditions for the forecast. It can run again and again, and a guidance term sets how strong the variation is. Any single-answer neural forecaster can gain a range of outcomes this way.
Tests used a global weather dataset at coarse grid size, built from records since the nineteen sixties. The spread-out system built up less error over long forecasts. Its range of outcomes also stayed reasonable. The approach frees ensemble design from methods built only for physics-based solvers.
Catalog record
| CREATOR | David Millard, Arielle Carr, Stéphane Gaudreault, Ali Baheri |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-07324
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DEF: Diffusion-augmented Ensemble Forecasting Journal: UNSURE David Millard, Arielle Carr, Stéphane Gaudreault, Ali Baheri Abstract We present DEF ( D iffusion-augmented E nsemble F orecasting), a novel approach for generating initial condition perturbations. Modern approaches to initial condition perturbations are primarily designed for numerical weather prediction (NWP) solvers, limiting their applicability in the rapidly growing field of machine learning for weather prediction. Consequently, stochastic models in this domain are often developed on a case-by-case basis. We demonstrate that a simple conditional diffusion model can (1) generate meaningful structured perturbations, (2) be applied iteratively, and (3) utilize a guidance term to intuitivey control the level of perturbation. This method enables the transformation of any deterministic neural forecasting system into a stochastic one. With our stochastic extended systems, we show that the model accumulates less error over long-term forecasts while producing meaningful forecast distributions. We validate our approach on the 5.625 ∘ ERA5 reanalysis dataset, which comprises atmospheric and surface variables over a discretized global grid, spanning from the 1960s to the present. On this dataset, our method demonstrates improved predictive performance along with reasonable spread estimates. 1 Introduction In 2023, a machine learning model surpassed the best operational numerical weather prediction models. Pangu-Weather [ 1 ] , a 3D Earth-Specific Transformer, demonstrated state-of-the-art accuracy in the medium-range forecast (10–15 days). More remarkably, it achieved this in a fraction of the computational cost–producing forecasts in minutes on a GPU, whereas traditional models require custom hardware processing petaflops of operations over the course of hours. Since then, the machine learning weather prediction community has expanded, leading to the development of more advanced models such as GraphCast [ 2 ] , FourCastNet [ 3 ] , and FuXi [ 4 ] . Despite their architectural differences, they all share one trait: deterministic forecasting. Two common approaches for introducing stochasticity to the trajectories of a forecast are ensemble methods and condition perturbation. Ensemble methods involve using multiple models with the same input to generate different trajectories, achieved by varying model parameters. While powerful, these methods are extremely expensive, particularly in machine learning, where training a model can take weeks to months. Condition perturbation, on the other hand, involves perturbing known conditions to generate plausible alternative states that the system could have experienced. These perturbations are both effective and cost-efficient, as they account for the inherent uncertainty in observations. However, the primary challenge lies in designing an effective perturbation strategy. Many modern condition perturbation techniques employ data assimilation methods, which leverage known observations and error estimates to generate conditions that theoretically maximize exploration of the forecasting space. While theoretically compelling, these methods require both a significant history of observational data and a large number of ensemble members to fully explore the space. The machine learning weather prediction (MLWP) community has addressed the challenge of ensemble forecasting on a case-by-case basis. Following the success of GraphCast, GenCast [ 5 ] was introduced–the same GraphCast model trained as a conditional diffusion model. FourCastNet demonstrated a stochastic approach by employing an ensemble of deterministic forecasts. Other techniques, such as Monte Carlo dropout, conformal prediction, and Bayesian neural networks, offer a more general framework for uncertainty quantification within deep learning but face several practical limitations when applied to the sensitivity and complexities of modern MLWP architectures. This discrepancy arises because, unlike many other fields, neural forecasting systems are trained to be highly sensitive to their inputs. In this study, we present an approach that successfully incorporates all three aspects: low complexity, fast speed, and high transferability. Our method is modular in design, leverages the well-studied and widely understood conditional diffusion methodology, and, after applying various speedups from modern diffusion literature, can generate hundreds of perturbations within minutes. Utilizing this technique, any deterministic model can effectively generate a large ensemble of members without requiring modifications to inference or training, aside from simply perturbing the initial conditions and autoregressing upon them. Our Contributions. The contributions of this paper are threefold: 1. We develop a novel method for condition perturbation by performing conditional generation on the input conditions themselves. This method requires significantly less computational power, as the model generates within its own state rather than predicting the next state. 2. We demonstrate that the method can be applied iteratively to perform "random walks" within a condition, offering more exploration power and better distribution estimation of the extremely large trajectory space. 3. We show that the guidance term, the scalar parameter \omega used within classifier free guidance, intuitively controls the strength of the perturbation, directly regulating the level of uncertainty quantification and enabling the controlled exploration of the state space by informed users. Paper Organization. The remainder of this paper is structured as follows. Section 2 is a literature review covering recent techniques used for uncertainty quantification. Section 3 establishes foundational concepts in ensemble forecasting, neural forecasting systems, and denoising diffusion probabilistic models. Section 4 presents the methodology, detailing our dataset, models, and training/inference procedures. Section 5 presents our numerical experiments/results, Section 6 discusses key findings, and Section 7 outlines future work and directions enabled by this study. 2 Related Work Model ensembling is the most popular method for ensemble forecasting. Within the context of deep learning, the method relies on training multiple deterministic members and using their distribution of predictions. Due to the randomness of training deep learning methods, differing random seeds produce slightly different predictive models, and due to the sensitivity of forecasting models, they can lead to diverging results in the long term. FourCastNet [ 3 ] proposed the use of 1,000 deterministic members to generate an effective ensemble that was able to compete with the ECMWF [ 6 ] , which has 30-50 members due to the significant computational demand of numerical weather prediction (NWP) [ 7 ] schemes. Modern perspectives are now considering huge ensembles of 1,000 to 10,000 members [ 8 ] utilizing spherical Fourier neural operators [ 9 ] , an extension of FourCastNet that operates on the spherical coordinates of Earth rather than grid interpolations. While these approaches are very promising, they all suffer from a lack of probabilistic formulation and typically require a significant number of models to ensure the empirical distribution is large enough to effectively capture the uncertainty of the dynamics, resulting in an ensemble underdispersion. One established machine learning architecture for generating probabilistic forecasts is Bayesian neural networks [ 10 ] . These networks modify the conventional neural network framework by learning distributions of weights rather than point estimates. This modification fundamentally alters the architecture and training procedures but demonstrates significant advantages when modeling inherently uncertain data. Recent research has specifically examined the application of this approach for predicting North Atlantic sea surface temperature using a convolutional LSTM architecture [ 11 ] . The investigators demonstrated that through their probabilistic formulation, the direct point estimate of the prediction exhibited superior performance on average compared to an equivalent model with standard deep learning weight parameterization. Furthermore, the study presents a quantitative comparison between a conventional deep learning ensemble and their SVGD ensemble derived from the Bayesian convolutional LSTM. Their analysis reveals that the SVGD methodology mitigates the ensemble collapse phenomenon frequently observed in standard deep learning ensembles, wherein ensemble members produce nearly