Does Your Wildfire Prediction Model Actually Work, or Just Score Well?

Yangshuang Xu, Yuyang Dai, Liling Chang, Qi Wang, Yushun Dong · 2026

Read Does Your Wildfire Prediction Model Actually Work, or Just Score Well? by Yangshuang Xu, Yuyang Dai, Liling Chang, Qi Wang, Yushun Dong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18911.txt.

Introduction

Yangshuang Xu and four colleagues build WILDFIRE-FM, the first foundation model pretrained for wildfire prediction from weather, fire observations, topography, vegetation and static data. General Earth models are trained for broad atmospheric goals, not fire forecasting. But a specialized backbone alone does not fix evaluation: sparse fire events make transfer verdicts sensitive to matching rules and settings. The authors add a fixed-contract evaluation framework with separate checks for matching-rule and head-selection effects, showing wildfire transfer verdicts depend strongly on evaluation design.

Catalog record

CREATORYangshuang Xu, Yuyang Dai, Liling Chang, Qi Wang, Yushun Dong
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87075
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2605.18911

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Does Your Wildfire Prediction Model Actually Work, or Just Score Well? Yangshuang Xu, Yuyang Dai, Liling Chang, Qi Wang, Yushun Dong Abstract Wildfire prediction is important for early warning and resource allocation, yet existing Earth foundation models (Earth FMs) are pretrained for general atmospheric and geophysical objectives rather than wildfire forecasting. To address this gap, we introduce Wild FIRE -FM , the first foundation model pretrained specifically for wildfire prediction using weather, active-fire observations, topography, vegetation, and static environmental data. However, introducing a domain-specific backbone alone does not solve the evaluation problem: wildfire events are sparse in space and time, making transfer conclusions highly sensitive to matching rules and evaluation settings. To address this problem, we introduce a fixed-contract evaluation framework with two controlled checks: a fixed-output check for matching-rule effects and a fixed-feature check for head-selection effects. Under matched contracts, we compare Wild FIRE -FM with ten Earth-FM baselines across occupancy, spread, retrieval, and regression tasks. Our results show that wildfire transfer conclusions depend strongly on evaluation design and task formulation. We hope this framework and Wild FIRE -FM provide a foundation for future wildfire-specific Earth-FM research and benchmarking. https://huggingface.co/RAI-Lab/Wildfire-FM 1 Introduction Wildfire prediction is critical for early warning and resource allocation in disaster response [ 10 ; 7 ] . As extreme fire events grow more frequent and severe, accurate forecasting of wildfire occurrence and spread is becoming increasingly important [ 29 ; 13 ] . Recent Earth foundation models (Earth FMs), pretrained on large-scale atmospheric and geophysical data [ 2 ; 35 ; 23 ] , provide transferable representations for Earth-system dynamics and have shown strong performance across weather and remote-sensing tasks. However, wildfire dynamics depend on complex interactions among weather, vegetation, topography, fuel conditions, and active-fire behavior, which are not explicitly modeled during pretraining in existing general-purpose Earth FMs. This mismatch raises a natural question: can representations learned for general atmospheric or geophysical objectives transfer reliably to wildfire forecasting, and how should we measure that transfer? Answering this question requires solving two intertwined problems. The first is that existing Earth FMs are not pretrained specifically for wildfire dynamics, but instead adapted to wildfire tasks after general-purpose pretraining. To address this limitation, we introduce Wild FIRE -FM , the first foundation model pretrained specifically for wildfire prediction using fire-relevant multimodal data, including regional weather dynamics, active-fire observations, topography, vegetation, and static environmental context. By incorporating wildfire-specific signals directly during pretraining, Wild FIRE -FM learns representations aligned with the physical processes underlying fire behavior rather than relying on transfer from general atmospheric or geophysical objectives. The second problem is evaluation: even with a domain-specific model, reliably comparing Wild FIRE -FM against transferred general-purpose Earth FMs remains difficult. Wildfire events are sparse in space and time [ 6 ; 9 ] , making transfer conclusions highly sensitive to three sources of evaluation variability. First, matching rules determine what counts as a correct prediction. Early warning systems tolerate spatial offsets that post-fire damage assessment cannot, so different matching rules can produce substantially different F1 scores from the same model outputs [ 6 ; 9 ] . Second, head-selection metrics determine which lightweight adapter is chosen on top of a frozen representation. Ranking metrics such as PR-AUC and decision metrics such as F1 can favor different heads from the same frozen features [ 20 ; 37 ] . Third, wildfire task forms operate over different prediction units and metric families. Occupancy prediction, spread forecasting, burned-area regression, and analog retrieval therefore produce scores that are not directly comparable even under the same backbone [ 34 ; 8 ] . Related protocol sensitivity has also been observed in active learning [ 18 ] and selective classification [ 37 ] . We show that these effects become particularly severe in wildfire transfer evaluation, where sparse events and heterogeneous task forms amplify evaluation instability [ 19 ] . Figure 1 : Overview of Wild FIRE -FM and Evaluation Protocol in this paper. This evaluation instability makes reliable comparison between Wild FIRE -FM and existing Earth FMs fundamentally difficult. Standard geospatial benchmarks such as GEO-Bench [ 14 ] , WeatherBench2 [ 31 ] , WILDS [ 12 ] , SustainBench [ 41 ] , and TorchGeo [ 36 ] standardize datasets, splits, and metrics, but do so for tasks with dense, balanced labels where matching-rule sensitivity is not a primary concern. Wildfire studies such as FireCast [ 30 ] and Next Day Wildfire Spread [ 11 ] apply the same report-and-compare paradigm directly to sparse fire events without controlling for matching-rule choice, head-selection metric, or task-form comparability, the three sources of instability identified above. As a result, scores reported under different implicit protocol choices are not directly comparable, even when the underlying predictions are identical. Based on the limitations of prior work , our contributions are as follows (see Figure 1 ). • Wildfire-specific foundation model. We introduce Wild FIRE -FM , the first foundation model pretrained specifically for wildfire prediction using multimodal wildfire data spanning weather, active-fire observations, topography, vegetation, and static environmental context. Unlike general Earth FMs adapted after pretraining, Wild FIRE -FM learns wildfire representations directly from fire-relevant processes during pretraining. • Fixed-contract evaluation framework. We formulate wildfire Earth-FM transfer as a fixed-contract evaluation problem, defining a contract \mathcal{C}=(\mathcal{T},M,\Lambda,\Omega,\mathcal{A}) that specifies the task, metric, matching rule, evaluation scope, and lightweight-head family before comparison. We introduce two controlled checks: a fixed-output check for matching-rule effects and a fixed-feature check for head-selection effects, enabling evaluation artifacts to be separated from representation quality. • Systematic wildfire transfer benchmark. Under fixed contracts, we compare Wild FIRE -FM against ten general-purpose Earth FMs across six wildfire task forms. Our results show that wildfire transfer conclusions are highly sensitive to evaluation design and strongly task-dependent across occupancy, spread, retrieval, and regression settings. 2 Wild FIRE -FM Reference Backbone Wild FIRE -FM is a wildfire-specialized regional backbone trained on fire-relevant multimodal data for wildfire prediction. Existing general-purpose Earth FMs are pretrained for atmospheric and geophysical objectives [ 15 ] , or for remote-sensing objectives [ 32 ] , so wildfire-relevant information enters only indirectly through those objectives. In contrast, Wild FIRE -FM is trained with weather, active-fire observations, topography, vegetation, and static environmental context, so its representation is learned from inputs tied directly to wildfire behavior. This design makes Wild FIRE -FM a strong wildfire-specific backbone whose features are shaped by signals directly relevant to fire occurrence and spread. It provides a task-aligned regional model trained directly for wildfire prediction. It also serves as an empirical anchor for interpreting how transferred Earth FMs behave under matched evaluation contracts. This section describes the data resources and training strategy used to build Wild FIRE -FM as an in-domain reference backbone. The fixed-contract protocol used to compare it with transferred Earth FMs is defined separately in Section 3 . 2.1 Data Resources We group the resources by their role in the study: dynamic weather inputs, occupancy supervision, static context, and event-level resources for supporting tasks. Source and terms-of-use notes for the external data and model assets used in this study are summarized in Appendix Table 16 . Dynamic weather inputs. The weather inputs come from a California regional dataset built from NOAA High-Resolution

Read the complete text