MP-MoE: Matrix Profile-Guided Mixture of Experts for Precipitation Forecasting
Huyen Ngoc Tran 2 , Dung Trung Tran 2 , Hong Nguyen 3 , Xuan Vu Phan 2 , Nam-Phong Nguyen 2 · 2026
Read MP-MoE: Matrix Profile-Guided Mixture of Experts for Precipitation Forecasting by Huyen Ngoc Tran 2 , Dung Trung Tran 2 , Hong Nguyen 3 , Xuan Vu Phan 2 , Nam-Phong Nguyen 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25046.txt.
Introduction
Tran and co-authors forecast heavy rain in Vietnam with a mixture-of-experts model guided by pattern matching. Raw weather models drift in the tropics, where terrain and storms are complex. Standard fixes score each time point alone, which punishes small timing slips twice. MP-MoE adds a matrix profile loss that scores whole rain shapes, not single points. This helps the model pick the right expert and keeps storm shapes intact. On two river basin datasets, it beats raw forecasts and rivals on heavy rain scores.
Catalog record
| CREATOR | Huyen Ngoc Tran 2 , Dung Trung Tran 2 , Hong Nguyen 3 , Xuan Vu Phan 2 , Nam-Phong Nguyen 2 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29809 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.25046
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MP-MoE: Matrix Profile-Guided Mixture of Experts for Precipitation Forecasting Huyen Ngoc Tran 2 , Dung Trung Tran 2 , Hong Nguyen 3 , Xuan Vu Phan 2 , Nam-Phong Nguyen 2 Abstract Precipitation forecasting remains a persistent challenge in tropical regions like Vietnam, where complex topography and convective instability often limit the accuracy of Numerical Weather Prediction (NWP) models. While data-driven post-processing is widely used to mitigate these biases, most existing frameworks rely on point-wise objective functions, which suffer from the “double penalty” effect under minor temporal misalignments. In this work, we propose the Matrix Profile-guided Mixture of Experts (MP-MoE), a framework that integrates conventional intensity loss with a structural-aware Matrix Profile objective. By leveraging subsequence-level similarity rather than point-wise errors, the proposed loss facilitates more reliable expert selection and mitigates excessive penalization caused by phase shifts. We evaluate MP-MoE on rainfall datasets from two major river basins in Vietnam across multiple horizons, including 1-hour intensity and accumulated rainfall over 12, 24, and 48 hours. Experimental results demonstrate that MP-MoE outperforms raw NWP and baseline learning methods in terms of Mean Critical Success Index (CSI-M) for heavy rainfall events, while significantly reducing Dynamic Time Warping (DTW) values. These findings highlight the framework’s efficacy in capturing peak rainfall intensities and preserving the morphological integrity of storm events. I Introduction Precipitation forecasting plays a vital role in mitigating natural hazards, optimizing reservoir operations, and ensuring sustainable agricultural planning. Traditionally, Numerical Weather Prediction (NWP) models [ 1 , 7 , 15 ] have been globally adopted as the standard for short-term rainfall forecasting due to their ability to simulate atmospheric dynamics. Various operational frameworks, such as the Weather Research and Forecasting (WRF) system [ 20 ] , have served as the primary physical foundation for these efforts by solving complex fluid dynamics and thermodynamic equations under specific initial and boundary conditions. However, despite their widespread applications, they are inherently constrained by the chaotic nature of the atmosphere and their sensitivity to initial perturbations. Such limitations often manifest as systematic biases and spatial displacement errors, particularly within tropical regions characterized by complex topography, such as Vietnam. Recently, statistical post-processing has emerged as a well-established framework to calibrate raw model outputs, encompassing a wide range of methodologies. These approaches span from classic probabilistic techniques such as Bayesian Model Averaging (BMA) [ 16 ] and Quantile Regression Averaging (QRA) [ 3 ] to more recent advanced data-driven models including XGBoost [ 4 ] and Long Short-Term Memory (LSTM) [ 11 ] . While these methods have successfully improved calibration for general weather conditions and captured complex nonlinear dependencies, they often struggle to accurately resolve rapidly evolving convective systems or extreme rainfall onsets. One of the fundamental limitations is that they rely on pointwise loss functions like Mean Squared Error (MSE) or Mean Absolute Error (MAE). This dependency triggers the well-known “double penalty” problem, where a forecast that correctly captures a storm’s structural evolution but is slightly displaced in time is penalized twice: once for the displacement and once for the perceived false alarm [ 6 , 24 ] . Mathematically, such an optimization landscape incentivizes models to produce overly smoothed, blurred predictions to minimize variance, resulting in the systematic suppression of peak rainfall intensities, a phenomenon commonly referred to as “peak-shaving” [ 17 ] . To bridge these critical gaps, we propose the Magnitude-Aware Matrix Profile Mixture of Experts (MP-MoE). This neural post-processing framework shifts the optimization paradigm from strict point-wise accuracy to structural fidelity by integrating a shape-aware objective into a dynamic expert selection mechanism. Unlike traditional methods, MP-MoE leverages temporal subsequence similarity to mitigate the double penalty effect while preserving the physical magnitude of extreme events. The key contributions of this work are summarized as follows: • A hybrid loss function is introduced that integrates MSE with a structural similarity penalty derived from an unnormalized MP distance [ 25 ] . This mechanism acts as a “soft” time-lag correction, rewarding models that capture the true physical evolution of rainfall. • A gating network is proposed that uses large-scale GFS variables to dynamically assign weights to specific WRF experts based on meteorological regimes. This architecture allows the system to adaptively switch strategies between stable stratiform and convective rainfall regimes. • Comprehensive experiments are conducted in two real-world hydrological basins in Vietnam, demonstrating that MP-MoE significantly reduces Dynamic Time Warping (DTW) values [ 13 ] and reconstructs peak intensities. II Related Work Rainfall forecasting has traditionally relied on NWP models, which provide a rigorous physical foundation but frequently exhibit systematic biases due to simplified atmospheric parameterizations and coarse spatial resolutions [ 1 , 22 ] . To mitigate these discrepancies, statistical post-processing methods, such as BMA and QRA, have been widely implemented to refine raw NWP outputs. While effective in improving probabilistic calibration, these frameworks often struggle to capture the complex, non-linear dependencies inherent in extreme meteorological events, particularly during rapid storm onsets where atmospheric conditions shift abruptly [ 9 , 22 ] . Recent advances in deep learning have accelerated the shift toward data-driven rainfall forecasting models. Early architectures advanced data-driven forecasting by transitioning from temporal modeling with LSTM to capturing joint spatiotemporal dependencies via ConvLSTM and PredRNN [ 19 , 23 ] . These foundations paved the way for high-capacity frameworks such as GraphCast [ 14 ] , Earthformer [ 8 ] , and Pangu-Weather [ 2 ] , which excel at extracting multi-scale patterns from atmospheric data. Despite their success, a persistent challenge is that many such models rely predominantly on point-wise objective functions like MSE. For highly non-stationary rainfall processes, these losses tend to penalize temporal misalignment excessively, leading to the well-known “peak-shaving” effect where intense precipitation events are smoothed or underestimated [ 6 ] . To address the limitations of point-wise accuracy, structural distance measures such as DTW and Soft-DTW have been proposed to alleviate this issue by allowing elastic temporal matching between predicted and observed rainfall sequences [ 18 , 5 ] . Nevertheless, these methods suffer from quadratic computational complexity and may introduce non-physical temporal warpings, limiting their practicality for operational hydrological forecasting. While the MP paradigm has recently emerged as an efficient framework for subsequence similarity search in data mining [ 25 , 21 ] , its potential as a structural penalty mechanism to guide expert selection remains largely unexploited. Unlike DTW’s unrestricted temporal warping, MP identifies structurally similar patterns within a constrained temporal search range, thereby preserving the morphological characteristics of rainfall events while maintaining a much lower constant-factor overhead in practical execution. In parallel with loss function optimization, the MoE architecture has received increasing attention for managing heterogeneous data regimes. Unlike static ensembles that apply fixed weights to model outputs, MoE utilizes a dynamic gating mechanism to route inputs to specialized experts based on the current context [ 12 , 10 ] . While MoE has shown promise in general time-series tasks, its application in NWP post-processing remains under-explored, particularly regarding the use of structural similarity to guide the gating process. Our approach bridges this gap by integrating MP-derived guidance into an MP-MoE architecture. By optimizing the selection process through a redefined structural loss, our framework ensures adaptive expert selection while preserving the integrity of rainfall