Timeseries Foundation Models for Mobility: A Benchmark Comparison with Traditional and Deep Learning Models

Anita Graser · 2025

Read Timeseries Foundation Models for Mobility: A Benchmark Comparison with Traditional and Deep Learning Models by Anita Graser (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03725.txt.

Introduction

Anita Graser, writing alone, benchmarks time series foundation models for mobility forecasts in this 2025 paper. City bike and crowd flows are classic forecast targets. New foundation models promise strong results with no retraining on a fresh task.

The study pits TimeGPT against classic statistical tools and deep learning models. Two bike-sharing datasets, from New York City and Vienna, supply the test series. Forecasts run at one hour, twelve hours, and one day ahead. Scores show where the foundation model shines and where it lags older tools. The zero-shot strength is real but bounded, Graser concludes, and task setup still matters.

Catalog record

CREATORAnita Graser
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE9 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-03725

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Timeseries Foundation Models for Mobility: A Benchmark Comparison with Traditional and Deep Learning Models Anita Graser Abstract . Crowd and flow predictions have been extensively studied in mobility data science. Traditional forecasting methods have relied on statistical models such as ARIMA, later supplemented by deep learning approaches like ST-ResNet. More recently, foundation models for time series forecasting, such as TimeGPT, Chronos, and LagLlama, have emerged. A key advantage of these models is their ability to generate zero-shot predictions, allowing them to be applied directly to new tasks without retraining. This study evaluates the performance of TimeGPT compared to traditional approaches for predicting city-wide mobility timeseries using two bike-sharing datasets from New York City and Vienna, Austria. Model performance is assessed across short (1-hour), medium (12-hour), and long-term (24-hour) forecasting horizons. The results highlight the potential of foundation models for mobility forecasting while also identifying limitations of our experiments. 1. Introduction Accurate mobility forecasting is essential for urban transport operations and planning. Traditionally, time series forecasting has relied on statistical models such as AutoRegressive Integrated Moving Average (ARIMA) and Seasonal ARIMA (SARIMA), along with machine learning and deep learning architectures. Recent advances in time series foundation models, such as Chronos ( Ansari et al. 2024 ) , Lag-Llama ( Rasul et al. 2024 ) , and TimeGPT ( Garza et al. 2024 ) , have demonstrated the potential of large-scale pre-trained models for forecasting. These foundation models leverage knowledge across diverse datasets, providing zero-shot and few-shot forecasting capabilities. However, their effectiveness in mobility prediction remains underexplored. This study benchmarks TimeGPT against classical and deep learning models for predicting bike-sharing flow and bicycle availability in urban environments. The primary objectives are to: (1) gain intuition about TimeGPT’s ability to predict mobility time series, (2) compare its performance with classical machine learning (ML) and deep learning (DL) models, and (3) analyze how different forecasting horizons impact predictive accuracy. 2. Related Work Mobility forecasting has been widely studied using both traditional and deep learning approaches. To assess developments in flow forecasting over the past decade, we reviewed openly accessible papers citing the seminal ST-ResNet paper ( Zhang et al. 2017 ) and referencing the BikeNYC dataset using a targeted Google Scholar search query ( https://scholar.google.at/scholar?hl=en&as_sdt=2005&sciodt=0%2C5&cites=10585174952970430867&scipsc=1&q=bikenyc&btnG= ). Papers that provide comparable BikeNYC experiments include ( Mourad et al. 2019 ; Xiong et al. 2022 ) . Their results are included in the following comparisons. Multiple other papers mention BikeNYC but use different datasets from the NYC bicycle sharing system ( Du et al. 2020 ; Ye et al. 2021 ; Jiang et al. 2021 ) or don’t provide enough information to ensure that the experiments are comparable ( Parvathi et al. 2024 ) . Recent literature also introduces explicitly spatiotemporal foundation models such as UrbanGPT ( Li et al. 2024 ) , UniST ( Yuan et al. 2024a ) , and UrbanDiT ( Yuan et al. 2024b ) . However, to our knowledge, none have publicly released their model weights, limiting reproducibility and further research. 3. Experimental Setup Our experiments build upon the ST-ResNet framework ( Zhang et al. 2017 ) . Since the original code repository ( https://github.com/lucktroy/DeepST/tree/master/scripts/papers/AAAI17 ) is no longer available, we leveraged a publicly available implementation https://github.com/topazape/ST-ResNet . The complete experimental setup is available at https://github.com/anitagraser/ST-ResNet/blob/main/experiment.ipynb . This study uses two datasets: BikeNYC and BikeVIE, summarized in Table 1 . BikeNYC has been extensively studied, while BikeVIE provides a less-explored dataset from Vienna, Austria for evaluating generalization. The BikeVIE timeseries were preprocesssed by resampling the number of available bicycles to the maximum hourly value, with a cutoff in mid-August to avoid a longer data collection gap as well as the less busy autumn and winter period. Table 1. Datasets BikeNYC BikeVIE Description: Hourly bike flows in New York City Hourly bike-sharing station data from Vienna Source: https://1drv.ms/f/s!Akh6N7xv3uVmhOhCtwaiDRy5oDVIug https://www.opendataportal.at/katalog/de/dataset/sharedmobility-ai-tensorflow-datasets No. timeseries: 16x8=128 grid cells 120 stations Time span: 2014-04-01 to 2014-09-30 2019-05-07 to 2019-12-31 (cut off at 2019-08-15) Benchmark models include: • Baseline models : AutoARIMA, Seasonal Naive, Historical Average • Foundation model : TimeGPT Example forecasting results are provided in Figure 4 and Figure 5 for the baseline models and TimeGPT, respectively. Performance is evaluated using Root Mean Square Error (RMSE) for 1-hour, 12-hour, and 24-hour forecasts, with the last ten days used for backtesting via a rolling window approach. 4. Results and Discussion In this section, we present the results obtained for the BikeNYC and BikeVIE datasets. For BikeNYC, we compare TimeGPT to model performance results found in the literature as well as to our three baseline models. For BikeVIE, we rely solely on our three baseline models since, to the best of our knowledge, there are no comparable results in the literature. 4.1. BikeNYC Results Table 2 summarizes the results for the BikeNYC dataset. TimeGPT outperforms ARIMA, DeepST, ST-ResNet, PredNet, and PredRNN for 1-hour forecasts (Figure 1 ). Only ASTIR ( Mourad et al. 2019 ) reports notably better results. Figure 1. BikeNYC results. Comparing model results for a horizon of 1 hour. For longer horizons, TimeGPT remains competitive to the baselines but converges to the SeasonalNaive performance at 24 hours (Figure 2 ). Notably, SeasonalNaive outperforms AutoARIMA for 12-hour and 24-hour forecasts, suggesting strong seasonal effects. Figure 2. BikeNYC results. Comparing model results for different forecast horizons. Table 2. BikeNYC results Model Horizon RMSE ARIMA ( Zhang et al. 2016 ) 1 10.56 SARIMA ( Zhang et al. 2016 ) 1 10.07 VAR ( Zhang et al. 2016 ) 1 9.92 DeepST ( Zhang et al. 2016 ) 1 7.43 ST-ResNet ( Zhang et al. 2017 ) 1 6.33 AFCM ( Mourad et al. 2019 ) 1 5.64 ASTIR ( Mourad et al. 2019 ) 1 4.18 PredNet ( Xiong et al. 2022 ) 1 7.45 PredRNN ( Xiong et al. 2022 ) 1 5.99 CLSTAN ( Xiong et al. 2022 ) 1 5.65 HistoricAverage 1/12/24 10.78 SeasonalNaive 1/12/24 8.93 AutoARIMA 1 7.18 AutoARIMA 12 10.55 AutoARIMA 24 10.42 TimeGPT 1 5.70 TimeGPT 12 7.62 TimeGPT 24 8.93 4.2. BikeVIE Results Table 3 summarizes the results for the BikeVIE dataset. TimeGPT struggles with longer horizons, with RMSE more than doubling from 1-hour to 12-hour forecasts. While TimeGPT marginally outperforms AutoARIMA at 12 and 24 hours, the results indicate significant forecast uncertainty. The poor performance of SeasonalNaive, even compared to the simple HistoricAverage, suggests the presence of irregular station restocking patterns that affect predictability. Figure 3. BikeVIE results. Comparing model results for different forecast horizons. Table 3. BikeVIE results Model Horizon RMSE HistoricAverage 1/12/24 7.04 SeasonalNaive 1/12/24 7.87 AutoARIMA 1 2.26 AutoARIMA 12 5.10 AutoARIMA 24 6.20 TimeGPT 1 2.32 TimeGPT 12 4.84 TimeGPT 24 5.67 5. Conclusion and Future Work These results indicate that time series foundation models like TimeGPT hold promise for mobility forecasting. However, a key limitation is the potential exposure of foundation models to datasets used in evaluation, raising concerns about true generalization. Future studies should evaluate foundation models on previously unpublished datasets. Since deep learning models benefit from additional training data, it remains likely that domain-specific deep learning models will outperform foundation models when sufficiently trained. However, in data-sparse scenarios, foundation models present a viable forecasting option. Future research should furthermore explore the impact

Read the complete text