Rolling-Origin Validation Reverses Model Rankings in Multi-Step PM10 Forecasting: XGBoost, SARIMA, and Persistence Journal: International Journal of Forecasting

Federico García Crespi, Eduardo Yubero Funes, Marina Alfosea Simón · 2026

Read Rolling-Origin Validation Reverses Model Rankings in Multi-Step PM10 Forecasting: XGBoost, SARIMA, and Persistence Journal: International Journal of Forecasting by Federico García Crespi, Eduardo Yubero Funes, Marina Alfosea Simón (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20315.txt.

Introduction

Garcia and co-authors compare ways to judge multi-step PM10 air pollution forecasts. Many studies use one fixed split and skip the simple persistence baseline. They test XGBoost, SARIMA, and persistence on 2,350 daily readings from southern Europe. Under a fixed split, XGBoost looks strong one to seven days ahead. Under rolling-origin updates, the rankings reverse. SARIMA keeps positive skill at all ranges, while XGBoost often fails to beat persistence. Static splits, they warn, can overstate real usefulness.

Catalog record

CREATORFederico García Crespi, Eduardo Yubero Funes, Marina Alfosea Simón
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33732
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.20315

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Rolling-Origin Validation Reverses Model Rankings in Multi-Step PM10 Forecasting: XGBoost, SARIMA, and Persistence Journal: International Journal of Forecasting Federico García Crespi, Eduardo Yubero Funes, Marina Alfosea Simón Abstract (a) Many air quality forecasting studies report gains from machine learning, but evaluations often use static chronological splits and omit persistence baselines, so the operational added value under routine updating is unclear. (b) Using 2,350 daily PM10 observations from 2017 to 2024 at an urban background monitoring station in southern Europe, we compare XGBoost and SARIMA against persistence under a static split and a rolling-origin protocol with monthly updates. We report horizon-specific skill and the predictability horizon, defined as the maximum horizon with positive persistence-relative skill. Static evaluation suggests XGBoost performs well from one to seven days ahead, but rolling-origin evaluation reverses rankings: XGBoost is not consistently better than persistence at short and intermediate horizons, whereas SARIMA remains positively skilled across the full range. (c) For researchers, static splits can overstate operational usefulness and change rankings. For practitioners, rolling-origin, persistence-referenced skill profiles show which methods stay reliable at each lead time. 1 Introduction Air quality forecasting is operationally useful only if predictions remain informative at the time scales required for intervention. For PM 10 , this requirement is particularly stringent. Under Directive 2008/50/EC, Member States must limit daily exceedances of the 24-hour mean concentration threshold of 50 \mu g m -3 , making short-term forecasting relevant for episode management, traffic regulation, industrial mitigation, and public health advisories [ 2 , 5 , 7 ] . In this context, the practical question is not merely whether a model reduces average forecast error, but whether it provides actionable skill beyond simple temporal inertia over the one- to seven-day horizon relevant to operational decision-making. A large body of literature has addressed PM forecasting using statistical, machine learning, and deep learning approaches. Despite this methodological diversity, the operational meaning of reported gains remains difficult to assess because evaluation is often based on a single static train–test split [ 6 ] and summarized through aggregate error measures such as RMSE, MAE, or R^{2} . These metrics are informative but incomplete: they do not show whether a model consistently improves upon a naive persistence forecast, nor how forecast usefulness changes across the multi-step horizon. This limitation is especially important in highly autocorrelated environmental series, where strong short-lead performance may reflect temporal inertia more than genuine predictive skill. As a result, model rankings obtained under static evaluation may be misleading from an operational perspective, particularly when forecasting systems are used under sequential information arrival and repeated forecast origins. In this setting, temporal validation design is not a secondary implementation detail but part of the substantive forecasting question. Recent work has also examined direct and recursive multi-step PM 10 forecasting with deep neural architectures in Southern Spain, showing that convolutional models can perform well for short-horizon prediction and that recursive forecasting may remain usable only under limited background conditions [ 3 ] . However, this line of work remains primarily architecture-centered and accuracy-centered. It does not directly address whether comparative conclusions are stable under leakage-safe rolling-origin evaluation, whether apparent gains persist relative to a persistence benchmark, or how long predictive usefulness remains operationally meaningful across horizons. A second limitation is that most comparative studies focus on performance at one predefined horizon or report multi-step errors without identifying the horizon beyond which predictive skill becomes negligible. Yet for operational forecasting, horizon dependence is central. Forecast usefulness should be assessed not only by absolute error but also by how long a model preserves positive skill relative to a minimal benchmark. Persistence is especially relevant in this setting because it represents the simplest plausible forecast in short-memory pollution series. Evaluating models against persistence makes it possible to distinguish genuine predictive value from improvements that are statistically detectable but operationally trivial. This study addresses these issues by reframing multi-step PM 10 forecasting as a problem of operational predictability under realistic temporal evaluation. Using daily PM 10 series from an urban background station in southeastern Spain, we compare three forecast families with different complexity profiles: persistence, SARIMA, and XGBoost. The comparison is conducted under a leakage-safe rolling-origin evaluation with train-only preprocessing, so that model assessment remains aligned with the information actually available at forecast time. This paper makes three main contributions. First, it provides a reproducible evaluation design for multi-step PM 10 forecasting that distinguishes genuine forecast skill from artifacts introduced by static validation and non-causal preprocessing. Second, it introduces an operational interpretation of forecast usefulness through the predictability horizon, H ∗ , defined as the maximum horizon at which a model retains positive persistence-relative skill. Third, it empirically shows that model rankings are not invariant to evaluation design: under rolling-origin evaluation, XGBoost — which appeared uniformly superior under static evaluation — failed to systematically outperform persistence at short and intermediate horizons, whereas SARIMA maintained positive skill throughout, reversing the apparent ranking. This ranking sensitivity is not a marginal technical detail, but a substantive result for forecasting methodology and practice. The remainder of the paper is organized as follows. Section 2 describes the dataset, forecasting protocols, models, and evaluation metrics. Section 3 presents the empirical case study results. Section 4 interprets the findings from an operational forecasting perspective, with emphasis on persistence-relative skill and H ∗ . Section 5 concludes with implications for PM 10 forecasting and the broader evaluation of environmental time-series models. 2 Methodology This study evaluates multi-step operational PM 10 forecasting under temporally realistic validation protocols. The objective is not only to compare predictive accuracy between models but also to determine the forecast horizon at which each model retains useful skill relative to a persistence benchmark. To this end, the experimental design combines daily PM 10 observations, competing forecasting models with different complexity profiles, and temporal evaluation schemes that distinguish static holdout assessment from rolling-origin deployment conditions. 2.1 Study Design The empirical design is structured around three methodological questions. First, how do forecast rankings change when evaluation moves from a conventional static split to rolling-origin evaluation? Second, to what extent do more flexible models improve upon a simple persistence benchmark across lead times from one to seven days ahead? Third, at which horizon does the model skill become operationally negligible? To answer these questions, we compare three forecast families: persistence, SARIMA, and XGBoost. The comparison is conducted under alternative temporal validation schemes and is evaluated using both absolute error metrics and persistence-relative skill. This design allows for the separation of raw predictive performance from operational usefulness under realistic forecasting conditions. 2.2 Study Area and Data The empirical illustration uses daily PM 10 concentration data from an urban background monitoring station in Elche (Alicante, southeastern Spain). This site is representative of a Mediterranean urban environment where PM 10 dynamics are influenced by local emissions, synoptic variability, resuspension processes, and episodic regional transport. Such conditions make the station a suitable test case for multi-step forecasting under heterogeneous and non-stationary environmental forcing. The target variable is the daily mean PM 10 concentration. Forecast experiments are defined for horizons h=1,\dots,7 days ahead. The modeling pipeline uses only information available up to the forecast origin for each prediction so that the evaluation remains consistent with operational deployment.

Read the complete text