WxC-Bench: A Novel Dataset for Weather and Climate Downstream Tasks
Rajat Shinde, Christopher E. Phillips, Kumar Ankur, Aman Gupta, Simon Pfreundschuh, Sujit Roy, Sheyenne Kirkland, Vishal Gaur, Amy Lin, Aditi Sheshadri, Udaysankar Nair, Manil Maskey, Rahul Ramachandran · 2024
Read WxC-Bench: A Novel Dataset for Weather and Climate Downstream Tasks by Rajat Shinde, Christopher E. Phillips, Kumar Ankur, Aman Gupta, Simon Pfreundschuh, Sujit Roy, Sheyenne Kirkland, Vishal Gaur, Amy Lin, Aditi Sheshadri, Udaysankar Nair, Manil Maskey, Rahul Ramachandran (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02780.txt.
Introduction
Rajat Shinde and co-authors release WxC-Bench, a ready-made dataset collection for training AI on weather and climate problems. Model builders have plenty of algorithms but few clean, prepared datasets, and each weather task eats data at a different scale. Their collection bundles tasks spanning neighborhood storms to continent-wide patterns: turbulence for aviation, hurricane strength and track, finding past weather twins, gravity wave modelling, and writing weather reports in words.
The dataset is public, with code to prepare it, and the paper documents its structure and runs baseline checks. The authors pitch it as shared ground where general weather AI can be trained and compared fairly.
Catalog record
| CREATOR | Rajat Shinde, Christopher E. Phillips, Kumar Ankur, Aman Gupta, Simon Pfreundschuh, Sujit Roy, Sheyenne Kirkland, Vishal Gaur, Amy Lin, Aditi Sheshadri, Udaysankar Nair, Manil Maskey, Rahul Ramachandran |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-02780
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
WxC-Bench: A Novel Dataset for Weather and Climate Downstream Tasks Rajat Shinde, Christopher E. Phillips, Kumar Ankur, Aman Gupta, Simon Pfreundschuh, Sujit Roy, Sheyenne Kirkland, Vishal Gaur, Amy Lin, Aditi Sheshadri, Udaysankar Nair, Manil Maskey, Rahul Ramachandran Abstract High-quality machine learning (ML)-ready datasets play a foundational role in developing new artificial intelligence (AI) models or fine-tuning existing models for scientific applications such as weather and climate analysis. Unfortunately, despite the growing development of new deep learning models for weather and climate, there is a scarcity of curated, pre-processed machine learning (ML)-ready datasets. Curating such high-quality datasets for developing new models is challenging particularly because the modality of the input data varies significantly for different downstream tasks addressing different atmospheric scales (spatial and temporal). Here we introduce WxC-Bench (Weather and Climate Bench), a multi-modal dataset designed to support the development of generalizable AI models for downstream use-cases in weather and climate research. WxC-Bench is designed as a dataset of datasets for developing ML-models for a complex weather and climate system, addressing selected downstream tasks as machine learning phenomenon. WxC-Bench encompasses several atmospheric processes from meso- β \beta (20 - 200 km) scale to synoptic scales (2500 km), such as aviation turbulence, hurricane intensity and track monitoring, weather analog search, gravity wave parameterization, and natural language report generation. We provide a comprehensive description of the dataset and also present a technical validation for baseline analysis. The dataset and code to prepare the ML-ready data have been made publicly available on Hugging Face - https://huggingface.co/datasets/nasa-impact/WxC-Bench . Background & Summary Figure 1: Illustration showing the applicability of the proposed WxC-Bench in the overall AI/ML workflows after converting raw non-ML ready datasets to ML ready datasets. The ML-ready datasets can then be directly used for training new AI models or fine-tuning pre-trained AI models. The WxC-Bench dataset can be used with AI models relevant to downstream applications across multiple spatial and temporal scales and resolutions. Weather and climate variability influences all spheres of human activity including the economy, transport, and energy production. Extreme events such as hurricanes, wildfires, droughts, and floods often result in catastrophic damage to human lives and capital. Since 1980, the U.S. alone has incurred damages worth more than $2.5 trillion from weather and climate disasters [ 1 ] . Current forecast models can reliably predict atmospheric extremes only up to 14-15 days in advance with forecast skill decreasing with time 1 1 1 https://www.whitehouse.gov/wp-content/uploads/2023/04/PCAST_Extreme-Weather-Report_April2023.pdf . In the past, such limits of predictability were believed to be improved mostly by recording more comprehensive set of observations, and through improving traditional models. However, the past couple of years have witnessed the emergence of a radically new, AI-driven approach. Artificial Intelligence driven weather forecasting models deliver accurate weather predictions more quickly and cost-effectively than traditional numerical models. Notable examples of these models are FourCastNet [ 2 ] , Graphcast [ 3 ] , FengWu [ 4 ] , Pangu [ 5 ] , and MetNet [ 6 ] . Machine Learning (ML) models also hold potential for longer-term subseasonal-to-seasonal modeling through efficient ensemble generation or otherwise [ 7 ] . Additionally, ML-based methods are being explored to improve climate prediction [ 8 ] , development of ML-driven parameterizations [ 9 , 10 , 11 , 12 ] , bias correction [ 13 ] , and understanding climate change impacts [ 14 , 15 ] . Recently, AI-Numerical Weather Prediction (NWP) models, primarily using transformer-based architectures, blended with Fourier Neural Operators [ 2 ] , Clifford Neural Operators [ 16 ] (NOs), and Graph Neural Networks have been introduced. These techniques aim to learn complex, nonlinear mappings required for approximating Partial Differential Equations (PDEs) that capture the inherent atmospheric physics. Unlike traditional NWP models, which are widely used to study weather extremes, ozone recovery, air quality monitoring, parameterization development, boundary layer turbulence, etc., these pioneering models are typically trained on large volumes of data, but only to perform one specific task: medium-range weather forecasting. This can severely limit their broader applicability, explaining the rising push to develop general-purpose AI models. Recent models such as Prithvi WxC [ 17 ] , ClimaX [ 18 ] , Aurora [ 19 ] , and Prithvi [ 20 ] have been developed as a generalist AI-system capable of performing multiple tasks. These generalist models are termed foundation models (FMs) for earth sciences. FMs [ 21 ] , which were first proposed in the fields of natural language processing (NLP) and computer vision (CV), are general-purpose AI models that replace specialized task-specific models across a wide range of applications. FMs are trained in an unsupervised manner on a large volume of datasets. Within atmospheric sciences, Prithvi WxC FM [ 17 ] , a 2.3 billion parameter model has been designed for weather and climate research with a focus on applying it to multiple downstream tasks. The emergence of weather and climate FMs like Prithvi WxC, ClimaX, etc., also poses questions about their generalization over several scientific applications and inherent uncertainties. Therefore, to empower a systematic benchmarking and performance evaluation of such models, there is a need for a unified ML ready dataset that spans over multiple spatial and temporal scales. WxC-Bench, the ML-ready dataset presented in this study, empowers such evaluations by facilitating the application of FMs to support downstream tasks and allowing a scoring of their performance. In this context, Table 1 outlines a few prominent ML-ready datasets that already exist. These include WeatherBench [ 22 ] and WeatherBench2 [ 23 ] exclusively for consistently scoring AI forecasts. Another dataset, ClimateNet [ 24 ] , provides a collection of expert-labeled tropical cyclones and atmospheric rivers. On longer timescales, ClimateBench [ 25 ] and ClimSim [ 26 ] both provide curated climate model datasets for training climate emulators. Additionally, data processing tools for processing data into diverse ML tasks, such as Climatelearn [ 27 ] have been developed. The main contribution of this paper is an extension of our previous work [ 28 ] with detailed analysis and comprehensive evaluation of the datasets. Table 1: Recently proposed weather and climate data benches and their key features Paper Title Year of Publication Key Features WeatherBench [ 22 ] , WeatherBench 2 [ 23 ] 2020, 2024 Dataset for data-driven medium-range weather forecasting (specifically 3–5 days). Dataset curated from ERA5 re-gridded to lower resolutions (5.625° (32 × 64 grid points), 2.8125° (64 × 128 grid points), and 1.40525° (128 × 256 grid points) to be used for deep learning approaches. ClimateNet [ 24 ] 2020 Dataset trained on expert-labeled climate data via ClimateContours, enables precise weather pattern segmentation, which utilizes output data from the 25 km Community Atmospheric Model (CAM5.1). This is a curated expert-labeled dataset which is trained on the DL segmentation model. ClimateBench [ 25 ] 2022 Dataset curated from a suite of Earth System Models participating in the Coupled Model Intercomparison Project 6 (CMIP6), covering a variety of global warming scenarios as yearly averages, at the native grid resolutions ( \sim 2 ∘ ). ClimSim [ 26 ] 2024 Dataset compiled using the E3SM-MMF multi-scale climate simulator and stored every 20 minutes for 10 years, at an approximate grid-spacing of 1.5 ∘ using a cubed-sphere unstructured grid. Curated for