Stable Attention Response for Reliable Precipitation Nowcasting DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 4544 CCS: Applied computing Earth and atmospheric sciences CCS: Computing methodologies Computer vision
Penghui Wen, Zexin Hu, Sen Zhang, Patrick Filippi, Xiaogang Zhu, Allen Benter, Thomas Bishop, Zhiyong Wang, Kun Hu · 2026
Read Stable Attention Response for Reliable Precipitation Nowcasting DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 4544 CCS: Applied computing Earth and atmospheric sciences CCS: Computing methodologies Computer vision by Penghui Wen, Zexin Hu, Sen Zhang, Patrick Filippi, Xiaogang Zhu, Allen Benter, Thomas Bishop, Zhiyong Wang, Kun Hu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13181.txt.
Introduction
Wen and eight co-authors trace precipitation nowcasting errors to shaky attention-response energy across samples. Bad forecasts pair with larger energy variance across heads and layers, and theory shows cross-sample variability spreads through self-attention, raising a lower bound on prediction error. Their HARECast setup models head-wise attention-response energy and steadies it with a group-wise penalty that cuts cross-sample swings. Built into a standard pipeline with reconstruction branches and a diffusion-based predictor, it leads SEVIR and MeteoNet benchmarks.
Catalog record
| CREATOR | Penghui Wen, Zexin Hu, Sen Zhang, Patrick Filippi, Xiaogang Zhu, Allen Benter, Thomas Bishop, Zhiyong Wang, Kun Hu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64936 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13181
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Stable Attention Response for Reliable Precipitation Nowcasting DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 4544 CCS: Applied computing Earth and atmospheric sciences CCS: Computing methodologies Computer vision Penghui Wen, Zexin Hu, Sen Zhang, Patrick Filippi, Xiaogang Zhu, Allen Benter, Thomas Bishop, Zhiyong Wang, Kun Hu Abstract . Precipitation nowcasting remains challenging due to the highly localized, rapidly evolving, and heterogeneous nature of atmospheric dynamics. Although recent methods increasingly adopt attention-based architectures in both unimodal and multimodal settings, they mainly emphasize stronger representation learning and prediction capacity, while paying less attention to the stability of attention responses across samples. In this work, we show that cross-sample instability of attention-response energy is an important and previously underexplored source of forecasting unreliability. Empirically, inaccurate forecasts are associated with larger attention-response energy variance across heads and layers. Theoretically, we show that cross-sample variability can propagate through self-attention, and enlarge a lower bound on prediction error. Based on this insight, we propose HARECast, a Head-wise Attention Response Energy-regulated framework for precipitation nowcasting. HARECast explicitly models head-wise attention-response energy and stabilizes it through a group-wise regularization objective that reduces cross-sample fluctuations. The proposed formulation is generic and applicable to both unimodal and multimodal nowcasting architectures. We instantiate HARECast in a standard forecasting pipeline with reconstruction branches and a diffusion-based predictor, and evaluate it on commonly used benchmarks–SEVIR and MeteoNet. Experimental results demonstrate that HARECast achieves state-of-the-art performance. Code will be made publicly available. 1. Introduction Precipitation nowcasting aims to predict near-future weather evolution from recent observation and has become increasingly essential in daily planning, transportation, and disaster management ( 34 ) . Despite substantial progress in deep forecasting models, reliable nowcasting remains challenging because atmospheric dynamics are highly localized, rapidly evolving, and strongly heterogeneous across samples ( 26 ) . As a result, even small spatial deviations in predicted rainfall patterns can noticeably degrade forecast quality. To address such complex spatio-temporal variability, recent nowcasting methods increasingly adopt attention-based ( 23 ) architectures. This trend appears in both unimodal settings, such as radar-based forecasting ( 31 ; 7 ) , and multimodal settings that combine radar with satellite observations ( 30 ; 29 ) . Attention is attractive because it can adaptively focus on informative regions and capture long-range interactions that are difficult to model with purely convolutional designs. As a result, improving representational capacity, generative prediction or cross-modal fusion has become a major direction in modern nowcasting approaches. However, we argue that a more fundamental issue has been underexplored: the stability of the attention response itself . In nowcasting, samples often differ substantially in intensity, spatial coverage, displacement patterns, and cross-source consistency ( 30 ; 34 ) , as shown in Figure 1 (b). Under such regime changes, the responses of attention heads may fluctuate sharply across samples. This instability already arises in unimodal forecasting and can become even more pronounced when heterogeneous modalities are introduced. As a result, attention-based models may exhibit unstable internal responses across samples, creating a previously overlooked bottleneck for reliable forecasting. Figure 1. (a) Visualization of the variance of attention-response energy across samples within a batch, shown for different heads and layers of PiMMNet ( 29 ) . An accurately predicted batch exhibit consistently lower variance, while an inaccurately predicted batch shows higher variance. (b) Examples of precipitation patterns exhibiting rapid and large spatial variations, which lead to strong cross-sample differences and high attention-response variance. We study this phenomenon from both empirical and theoretical perspectives. Empirically, we observe a clear correlation between prediction quality and the cross-sample variance of attention-response energy: accurate batch tend to exhibit more stable head responses, while inaccurate batch are often associated with larger energy fluctuations across heads and layers, as shown in Figure 1 (a). Theoretically, we show that cross-sample variability in the input can propagate through self-attention and induce variance in the attention response, which enlarges a lower bound on the prediction error. These results suggest that expressive attention alone is insufficient for reliable nowcasting, the attention response should also remain stable across samples . Motivated by this observation, we propose HARECast , a H ead-wise A ttention R esponse E nergy-regulated diffusion framework for precipitation nowcasting. The central idea of HARECast is to explicitly model and stabilize the energy of attention-induced responses. Specifically, we define head-wise attention response energy (HARE) to quantify the effective response strength of each attention head, and use it to organize heads into groups with different response patterns. We then introduce a group-wise HARE stabilization objective that regularizes attention-response energy toward the mean response of each head group,thereby reducing cross-sample fluctuations and stabilizing forecasting behavior. Importantly, this formulation is general: it can be applied to both unimodal and multimodal nowcasting network architectures, and is especially useful when heterogeneous modalities introduce additional response instability. We instantiate HARECast within a standard nowcasting framework by combining the proposed HARE stabilization mechanism with modality-aware reconstruction branches and a diffusion-based predictor for future radar forecasting. This design allows the method to be integrated into common forecasting architectures while delivering state-of-the-art performance on benchmark datasets including SEVIR ( 24 ) and MeteoNet ( 14 ) . Our main contributions are summarized as follows: • We show that cross-sample instability of attention-response energy is an important and previously underexplored failure mode in precipitation nowcasting. • We provide empirical and theoretical evidence that cross-sample variability can propagate through self-attention and increase a lower bound on prediction error, motivating explicit stabilization of attention-response energy. • We propose HARECast , a head-wise attention response energy-regulated framework for nowcasting, and demonstrate state-of-the-art performance on benchmark datasets. 2. Related Work We briefly review related work on unimodal and multimodal precipitation nowcasting, and attention energy analysis. Unimodal Precipitation Nowcasting. Deep precipitation nowcasting methods have evolved from deterministic spatiotemporal prediction models to probabilistic and hybrid generative frameworks. Early deterministic models learn spatiotemporal dynamics for mean-centric precipitation ( 20 ) : PhyDNet ( 12 ) incorporates physics-guided disentanglement to separate precipitation dynamics from residual information; FACL ( 28 ) replaces L2 with a signal-oriented spatiotemporal loss; AlphaPre ( 15 ) factorizes phase and amplitude to decouple motion from intensity. PercpCast ( 7 ) incorporates perceptual constraints through a rectified flow model to achieve better perceptual performance. More recent probabilistic models introduce latent variables to represent the intrinsic uncertainty of future weather, enabling finer depiction of small-scale processes ( 19 ) : DGMR uses a GAN with separate spatial/temporal discriminators ( 19 ) . NowcastNet ( 34 ) injects physical priors into a GAN. Prediff adds knowledge-guided sampling ( 9 ) . DuoCast ( 26 ) proposes a dual-diffusion framework for precipitation forecasting that decomposes predictions into low- and high-frequency components modeled in orthogonal latent subspaces. A theoretical analysis proves that this frequency decomposition reduces prediction error. Hybrid models combine deterministic backbones for large-scale evolution with probabilistic modules for local variability, balancing accuracy and diversity: CasCast ( 11 ) and DiffCast ( 31 ) exemplify this pairing of trajectory prediction with stochastic refinement. These methods have substantially improved forecasting quality, but their main focus remains on predictive capacity, uncertainty modeling, or generative realism, rather than on the cross-sample stability of