DINOv3 as a Frozen Encoder for CRPS-Oriented Probabilistic Rainfall Nowcasting
Luciano Araujo Dourado Filho, Almir Moreira da Silva Neto Anthony Miyaguchi,, Rodrigo Pereira David, Rodrigo Tripodi Calumby, Lukáš Picek · 2025
Read DINOv3 as a Frozen Encoder for CRPS-Oriented Probabilistic Rainfall Nowcasting by Luciano Araujo Dourado Filho, Almir Moreira da Silva Neto Anthony Miyaguchi,, Rodrigo Pereira David, Rodrigo Tripodi Calumby, Lukáš Picek (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-10894.txt.
Introduction
Luciano Araujo Dourado Filho and five co-authors propose a lean recipe for probabilistic rainfall nowcasting, the short-horizon prediction of accumulated rain with honest uncertainty. Their system keeps a large pretrained satellite vision encoder completely frozen, avoiding the cost of training a weather model from scratch.
A video projector built on a vision transformer reads the encoder's tokens. A lightweight probabilistic head turns them into a discrete distribution over four-hour accumulated rainfall, trained end to end against the ranked probability score. On the Weather4Cast 2025 benchmark, the system posted a continuous ranked probability score of about three and a half. That is roughly a 26 percent effectiveness gain over the best 3D-UNet baseline the authors trained. They present frozen-encoder nowcasting as both competitive and computationally cheap.
Catalog record
| CREATOR | Luciano Araujo Dourado Filho, Almir Moreira da Silva Neto Anthony Miyaguchi,, Rodrigo Pereira David, Rodrigo Tripodi Calumby, Lukáš Picek |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-10894
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DINOv3 as a Frozen Encoder for CRPS-Oriented Probabilistic Rainfall Nowcasting Luciano Araujo Dourado Filho, Almir Moreira da Silva Neto Anthony Miyaguchi,, Rodrigo Pereira David, Rodrigo Tripodi Calumby, Lukáš Picek Abstract This paper proposes a competitive and computationally efficient approach to probabilistic rainfall nowcasting. A video projector (V-JEPA Vision Transformer) associated to a lightweight probabilistic head is attached to a pre-trained satellite vision encoder (DINOv3 - SAT493M) to map encoder tokens into a discrete empirical CDF (eCDF) over 4-hour accumulated rainfall. The projector-head is optimized end-to-end over the Ranked Probability Score (RPS). As an alternative, 3D-UNET baselines trained with an aggregate Rank Probability Score and a per-pixel Gamma-Hurdle objective are used. On the Weather4Cast 2025 benchmark, the proposed method achieved a promising performance, with a CRPS of 3.5102, which represents ≈ \approx 26% in effectiveness gain against the best 3D-UNET. 1 Introduction Short-term precipitation forecasting (“nowcasting”) underpins risk-aware decisions in urban drainage and flood mitigation, aviation safety, road traffic management, and renewable energy balancing. Urban flash-flood control and sewer operations depend on reliable exceedance probabilities at neighborhood scale; airlines and air-traffic controllers require accurate, rapidly updating assessments of convective activity; grid operators need short-horizon rain and cloud dynamics to schedule reserves and balance wind/solar variability [ 14 ] . Meeting these operational needs requires models that deliver both high spatial/temporal resolution and calibrated uncertainty. Despite rapid progress, reliable high-resolution forecasts from heterogeneous observations remain challenging, particularly under spatial and temporal distribution shifts [ 12 , 11 ] .. Recent learning-based nowcasting systems—ranging from ConvLSTM video predictors to multi-resolution neural weather models—have advanced the state of the art at fine lead times [ 15 , 17 , 4 ] . In parallel, probabilistic evaluation via proper scoring rules such as the Continuous Ranked Probability Score (CRPS) has become standard, rewarding calibrated predictive distributions rather than point estimates [ 6 , 8 ] . We study the cumulative-rainfall track in a cross-sensor benchmark setting that stresses generalization. The target is the four-hour accumulation (16 slots at 15-minute cadence) averaged over a 32{\times}32 radar window, while inputs are recent multi-band satellite frames. This configuration forces super-resolution from satellite to radar scale and exposes models to domain shifts in space and time. Prior work around the Weather4cast line of benchmarks has shown that formulating geospatial forecasting as video prediction yields strong baselines and controlled stress tests under spatiotemporal and domain distribution shifts. [ 7 , 2 ] . Our approach takes a pragmatic view of foundation models for Earth observation: reuse a large satellite-pretrained vision encoder as a frozen “world-model” prior, and learn only a narrow probabilistic projector for downstream nowcasting. Concretely, we use a frozen DINOv3 ViT-L/16 encoder trained on satellite imagery [ 16 ] and compare lightweight transformer heads that map latent tokens to a categorical distribution over four-hour rainfall-accumulation bins. The probabilistic heads are trained with a discrete CRPS loss computed from cumulative probabilities (an eCDF), aligning optimization with the evaluation metric. To contextualize transfer versus full end-to-end learning, we also train compact, fully learnable 3D-UNet baselines that operate on the spatiotemporal satellite stack, instantiated with (i) a discrete-CRPS eCDF head and (ii) a zero-inflated Gamma (Gamma-Hurdle) head whose samples are aggregated into an empirical CDF of the four-hour total [ 13 , 3 ] . This design allows a controlled comparison of (frozen encoder + narrow head) against a from-scratch spatiotemporal convolutional alternatives, under a scoring rule that foregrounds calibration. Contributions. (i) We present a focused case study of frozen DINOv3 features for precipitation nowcasting from satellite context. (ii) We introduce a simple, calibration-oriented eCDF head trained with a discrete CRPS objective aligned to evaluation. (iii) We benchmark against compact 3D-UNet variants with both eCDF and hurdle-Gamma probabilistic heads, clarifying the trade-offs between sample efficiency, parameter count, and probabilistic quality. 2 Data The dataset is comprised of two years of 11 band satellite image radiance (visible, water vapor, and infrared) and OPERA high resolution rain rates in mm/hr at quarter hour frequency. The rain rates are six times the resolution of the satellite imagery meaning a 32x32 grid in radar resolution corresponds to an approximate 6x6 patch in the satellite resolution. For each training sample we extract a co-registered satellite context (4 frames, 11 bands, last hour) padded with context to match the corresponding 32\times 32 radar window used by scoring. This preserves the native super-resolution relationship between satellite context and radar target. During training we sample the radar data-cube with rain-fall biased random crops and apply random rotation and reflection during training. During validation, we center or evenly tile the cropping procedure for determinism. Table 1: Summary Statistics of the Validation Data Split Statistic Mean Min p50 p95 Max Masking (%) 0.0000 0.0000 0.0000 0.0000 0.0000 Zero-Inflation (%) 88.8463 0.0916 99.8718 100.0000 100.0000 Aggregate Target (mm) 0.3434 0.0000 0.0012 2.4084 4.1663 Non-Zero Mean (mm) 0.3073 0.0000 0.2354 0.9859 1.6961 Non-Zero Median (mm) 0.2327 0.0000 0.2200 0.7100 1.1700 Non-Zero p95 (mm) 0.7607 0.0000 0.2900 2.9515 4.7800 Non-Zero Max (mm) 1.7536 0.0000 0.2950 7.9700 10.5800 We compute aggregate statistics over the validation radar data cubes in Table 1 , demonstrating a high degree of zero-inflation. Rain fall in this split is rare, with a median per-sample median of 99.9% zero-valued pixels. The median per-sample target rain fall rate is 0.0012 mm/hr, while the non-zero-inflated value is 0.235mm/hr. The challenge is then learning how to learn for a target signal that is averaged over a sparse spatio-temporal cube. 2.1 Training Objective Figure 1: Training pipeline illustration for the DINO-V-JEPA architecture. The target variable y_{i,t} , is defined as the average of ground-truth rain rate ( R_{t}\in\mathds{R} ) for each pixel (m,n) over a 32\times 32\ (H\times W) grid, averaged within the t-th time-slot ( t\in[t_{0},t_{0}+15] ). y_{i}=\frac{1}{4}\sum_{t=t_{0}}^{t_{0}+15}\left[\frac{1}{H\times W}\sum_{m=1}^{H}\sum_{n=1}^{W}R_{t,m,n}\right] (1) We propose training the models over a discrete version of the Continuous Probability Rank Score (CRPS) [ 18 ] metric, namely Rank Probability Score (RPS), described in Equation 2 . RPS(F,y)=\sum_{k=1}^{K}\big(F_{k}-\mathds{1}(k\geq y)\big)^{2} (2) Let P_{\phi}(y_{i}\mid z_{i}) be the probability distribution of the target variable (spatially-averaged cumulative rainfall) conditioned on a latent representation of the input frames z_{i}=\{f_{\theta}(x_{i,t})\}_{t=t_{o}}^{t_{0}+3} : the set of embeddings extracted for an arbitrary geographical region i in each time instant, ranging from t=t_{0} to t_{0}+3 – where t_{0} indicates an initial time-step. f_{\theta}(x_{i,t}) denotes the encoding function (e.g., 3D-UNET, DINOv3, V-JEPA, etc) that maps an input satellite frame x_{i,t}\in\mathds{R}^{11\times 32\times 32} into an arbitrary representation. Then F_{k} (Equation 3 ), represents the cumulative distribution function (CDF) at bin k , where p_{k} denotes the predicted probability mass from P_{\phi}(y\mid z) at bin k . F_{k}(P_{\phi}(y_{i}\mid z_{i}))=\sum_{j=1}^{k}p_{j} (3) The training objective with the standard RPS loss then amounts to find the set of model parameters \phi from a P_{\phi}(y\mid z) whose CDF minimizes the squared distance to the target CDF of rainfall values (Equation 4 ). For the DINO-V-JEPA model (illustrated in Figure 1 ), for example, P_{\phi}(y_{i}\mid z_{i}=\{f_{\theta}(x_{i,t})\}_{t=t_{o}}^{t_{0}+3}) is a categorical distribution over the K bins, where f_{\theta} represents the frozen DINOv3, and \phi denotes the V-JEPA projector parameters. For the 3D-UNET variant, for example, P_{\phi}(y_{i}\mid z_{i}) denotes the Gamma-Hurdle distribution parameterized by the per-pixel outputs of the model head whereas \phi encompasses both the parameters of the feature extractor/encoder f_{\theta}