Sonny: Breaking the Compute Wall in Medium-Range Weather Forecasting
Minjong Cheon · 2026
Read Sonny: Breaking the Compute Wall in Medium-Range Weather Forecasting by Minjong Cheon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21284.txt.
Introduction
Cheon introduces Sonny, an efficient transformer for medium-range weather forecasting. Deep learning forecasters now rival operational systems, but many need huge training compute. Sonny uses a two stage StepsNet design. A narrow slow path first models large scale atmospheric dynamics. A full width fast path then adds thermodynamic interactions. Exponential moving average during training steadies the rollout. On WeatherBench2, Sonny shows robust medium-range skill and trains on one A40 GPU in about five and a half days.
Catalog record
| CREATOR | Minjong Cheon |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18594 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.21284
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Sonny: Breaking the Compute Wall in Medium-Range Weather Forecasting Minjong Cheon Abstract Weather forecasting is a fundamental problem for protecting lives and infrastructure from high-impact atmospheric events. Recently, data-driven weather forecasting methods based on deep learning have demonstrated strong performance, often reaching accuracy levels competitive with operational numerical systems. However, many existing models rely on large-scale training regimes and compute-intensive architectures, which raises the practical barrier for academic groups with limited compute resources. Here we introduce Sonny, an efficient hierarchical transformer that achieves competitive medium-range forecasting performance while remaining feasible within reasonable compute budgets. At the core of Sonny is a two-stage StepsNet design: a narrow slow path first models large-scale atmospheric dynamics, and a subsequent full-width fast path integrates thermodynamic interactions. To stabilize medium-range rollout without an additional fine-tuning stage, we apply EMA during training. On WeatherBench2, Sonny yields robust medium-range forecast skill, remains competitive with operational baselines, and demonstrates clear advantages over FastNet, particularly at extended tropical lead times. In practice, Sonny can be trained to convergence on a single NVIDIA A40 GPU in approximately 5.5 days. 1 Introduction Weather forecasting is essential for protecting lives, infrastructure, and economic systems from high-impact atmospheric events [ 14 , 1 , 3 ] . As climate variability intensifies, improving both forecast skill and accessibility has become a central scientific and societal priority [ 14 ] . Recent advances in data-driven Numerical Weather Prediction (NWP) show that CNN-based models, Graph Neural Networks (GNNs), and Transformers can match, and in some settings exceed, the skill of traditional physics-based systems [ 8 , 7 , 2 , 18 , 16 ] . However, many Large Weather Models (LWMs) still depend on computationally intensive training pipelines that require large-scale TPU/GPU infrastructure [ 2 , 16 , 10 ] . This training-time compute barrier can substantially limit broader academic participation in developing next-generation global forecasting models [ 15 ] . To address this accessibility gap, we target a compact model that can be trained with limited compute while still delivering reliable medium-range global forecasts. Existing studies have primarily emphasized either maximum skill at large scale or isolated efficiency gains on specific setups [ 16 , 4 , 13 , 5 ] , leaving a gap for a compute-accessible baseline that jointly prioritizes forecast quality, training stability, and reproducibility in a single end-to-end framework. To fill this gap, we present Sonny, an efficient Small Weather Models (SWM) based on the StepsNet architecture [ 12 ] . Sonny is designed to achieve competitive medium-range forecast skill at practical training cost, and it is developed as a reproducible baseline for broader academic use. Recent efficient SWM efforts, including ArchesWeather [ 9 ] and FastNet [ 10 ] , also highlight the importance of balancing forecast quality with training efficiency under constrained compute. Our key contributions are threefold. First, we introduce Sonny, an efficient SWM for medium-range global weather forecasting under limited compute budgets. Second, we provide a practical training recipe aimed at stable medium-range rollout without expensive large-scale infrastructure. Third, we demonstrate on WeatherBench2 that Sonny achieves competitive performance against HRES and AI baselines while remaining trainable on a single NVIDIA A40 GPU in approximately 5.5 days. 2 Method 2.1 Data We trained and evaluated Sonny using the WeatherBench2 (WB2) dataset [ 21 ] with 6-hourly temporal resolution and 1.5^{\circ} spatial resolution ( 121\times 240 ). Our input includes four surface variables and five atmospheric variables defined on 13 pressure levels (50, 100, 150, 200, 250, 300, 400, 500, 600, 700, 850, 925, and 1000 hPa). We split the data chronologically into 1979–2018 for training, 2019 for validation, and 2022 for testing. The full variable list is summarized in Table 1 . Table 1: List of ERA5 atmospheric variables used in Sonny, corresponding short names, vertical pressure levels in hectopascals (hPa), and standard units of measurement. Variable name Short name Vertical levels (hPa) Units Zonal wind U 1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 50 m/s Meridional wind V 1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 50 m/s Temperature T 1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 50 K Specific humidity Q 1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 50 kg/kg Geopotential Z 1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 50 m 2 /s 2 2m temperature T2m - K Mean sea level pressure MSLP - Pa Surface air pressure SP - Pa 10m zonal wind U10 - m/s 10m meridional wind V10 - m/s 2.2 Randomized Dynamics Forecasting Sonny was trained to predict the atmospheric dynamics \Delta_{\delta t}=X_{\delta t}-X_{0} rather than absolute states, representing the evolution of weather conditions over a time interval \delta t . Unlike conventional models that rely on a fixed interval (e.g., 6 hours), we used a randomized dynamics forecasting objective. During training, \delta t was sampled from a discrete uniform distribution P(\delta t)\sim\mathcal{U}\{6,12,24\} hours [ 17 ] . 2.3 Pressure-Weighted Loss Function To prioritize variables with the highest societal impact, we employed a physics-based pressure-weighted loss. This function uses atmospheric pressure as a proxy for density, assigning higher weights to variables near the Earth’s surface [ 17 ] . The final loss function incorporated both this pressure weighting w(v) and a latitude-weighting factor L(i) to account for the non-uniformity of the spherical grid: \mathcal{L}(\theta)=\mathbb{E}\left[\frac{1}{VHW}\sum_{v=1}^{V}\sum_{i=1}^{H}\sum_{j=1}^{W}w(v)L(i)\left(\Delta\hat{v}_{ij}^{\delta t}-\Delta v_{ij}^{\delta t}\right)^{2}\right] 2.4 Efficient Training via EMA One of bottlenecks in state-of-the-art weather models is the extensive fine-tuning required to stabilize long-term iterative forecasts. In Sonny, we bypassed this computationally expensive fine-tuning phase by implementing an Exponential Moving Average (EMA) during the primary training stage [ 20 , 22 , 11 ] . We applied a decay factor of 0.9 to the model weights, which effectively smooths the optimization landscape and enhances the model’s robustness. This approach served as a highly efficient alternative to traditional fine-tuning, allowing the model to achieve superior stability in iterative rollouts without additional temporal overhead. In our setup, the full training process converged in approximately 5.5 days on a single NVIDIA A40 GPU, with peak GPU memory usage of about 32 GB. 2.5 Sonny Sonny is a hierarchical weather forecasting model based on the StepsNet architecture, designed to exploit the distinct physical characteristics of atmospheric variables. The model’s core is the Variable-Aware Embedding module, which decouples input variables into two physically informed categories: a Dynamics group ( U,V,Z,P ) and a Thermodynamics group ( T,Q ). This separation is motivated by the physical hierarchy of the atmosphere, where kinematic drivers (dynamics) dictate the large-scale evolution that thermodynamic states subsequently follow. This variable-aware separation facilitates a dual-phase processing pipeline. In Step 1 (Slow Path), the Dynamics group is embedded into a d_{1} -dimensional space and processed through a specialized sequence of N_{1} Transformer blocks. By restricting the hidden dimension to d_{1} , Sonny can afford a deeper network architecture to distill the fundamental atmospheric backbone without the quadratic memory overhead typically associated with processing all variables simultaneously. This stage focuses on capturing long-range spatial dependencies and the structural evolution of the pressure fields. Subsequently, these refined dynamic features ( y_{1} ) are concatenated with the raw Thermodynamics embeddings