TSFMAudit: Data Contamination Auditing in Forecasting Time Series Foundation Models

Hongkai Li, Shifeng Xie, Lefei Shen, Zhuo Li, Mouxiang Chen, Xiaobin Zhang, Han Fu, Jianling Sun, Xiaoxue Ren, Chenghao Liu · 2026

Read TSFMAudit: Data Contamination Auditing in Forecasting Time Series Foundation Models by Hongkai Li, Shifeng Xie, Lefei Shen, Zhuo Li, Mouxiang Chen, Xiaobin Zhang, Han Fu, Jianling Sun, Xiaoxue Ren, Chenghao Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26161.txt.

Introduction

Li and nine colleagues audit pretraining contamination in time series foundation models, a first. Such models train on vast corpora, raising the worry that test sets were seen in pretraining, inflating scores. Auditing is hard because time series signals vary smoothly, differ widely, and often lack corpus records. TSFMAudit formalises the problem and probes adaptation after fine-tuning: contaminated sets show fast loss reduction with smaller backbone movement. Tested on six foundation models and 187 sets with known training sources, the method beats ten adapted baselines.

Catalog record

CREATORHongkai Li, Shifeng Xie, Lefei Shen, Zhuo Li, Mouxiang Chen, Xiaobin Zhang, Han Fu, Jianling Sun, Xiaoxue Ren, Chenghao Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65907
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26161

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TSFMAudit: Data Contamination Auditing in Forecasting Time Series Foundation Models Hongkai Li, Shifeng Xie, Lefei Shen, Zhuo Li, Mouxiang Chen, Xiaobin Zhang, Han Fu, Jianling Sun, Xiaoxue Ren, Chenghao Liu Abstract Time series foundation models (TSFMs) are increasingly pretrained on large corpora, raising concerns that evaluation datasets may have been exposed during pretraining and thus yield overly optimistic performance estimates. Auditing such contamination is challenging in time series because signals are continuous and heterogeneous, and often lack corpus documentation. To the best of our knowledge, this is the first work to study pretraining contamination auditing for TSFMs. We formalize the problem of pretraining contamination auditing for TSFMs and propose TSFMAudit , a method based on probe adaptation dynamics. Our key intuition is that contamination manifests as unusually efficient adaptation: after a fine tuning probe, contaminated datasets tend to exhibit faster loss reduction with smaller backbone movement. We evaluate TSFMAudit on 6 TSFMs and 187 datasets using documented training source evidence as supervision, and compare against 10 competitive baselines adapted from the LLM literature. \svgpath figures/images/ 1 Introduction Time series foundation models (TSFMs) have recently attracted growing attention because of their strong cross domain capabilities across applications such as finance, transportation, and health ( 24 ; 23 ; 36 ; 33 ) . Although there have been promising attempts to train TSFMs on purely synthetic data, current state-of-the-art models still rely heavily on large scale real world corpora collected from diverse sources ( 19 ; 43 ) . At the same time, widely used evaluation benchmarks, are themselves assembled from multiple public sources ( 1 ; 34 ) . This shared multi-source pretraining evaluation pipeline creates a substantial risk of contamination: data used for evaluation may already have been exposed during pretraining ( 30 ) . Such exposure can artificially inflate downstream performance , and thereby bias our assessment of TSFM generalization ( 30 ; 1 ; 13 ) . More importantly, contamination may arise unintentionally ( 30 ; 44 ) . In time series data, the same underlying signal can reappear under rescaling, renaming, rewindowing, or other preprocessing changes while preserving nearly identical content ( 30 ; 18 ) . For example, the Monash archive’s Elecdemand dataset is a 1/1000 -scaled subset of Australian Electricity Demand ( 30 ) . To ensure the fairness and validity of TSFM benchmarking, we therefore need efficient methods for contamination auditing ( 30 ; 1 ; 12 ; 10 ) . Contamination auditing has been studied extensively in large language models (LLMs) and related domains. Popular families of methods include overlap detection, which searches for direct or near-duplicate benchmark content, memorization extraction, which tests whether a model can reproduce training examples, perplexity-based tests, which flag unusually low model loss, and membership inference attacks, which calibrate a score for deciding whether an example was used in training, often using shadow models, i.e., auxiliary models trained on controlled data splits ( 45 ; 38 ; 8 ; 39 ; 6 ) . However, these methods do not transfer naturally to TSFMs because time series data are continuous, heterogeneous, and often only partially documented at the corpus level ( 41 ; 11 ; 35 ; 17 ; 19 ) . For instance, leakage can indeed lead to abnormally low loss, and methods such as 12 exploit this signal for contamination detection; however, a smooth seasonal series may also yield loss far below the normal generalization level even when the model has never seen it before, making static loss or perplexity-style criteria unreliable ( 26 ; 22 ) . Likewise, membership inference attacks typically require multiple shadow models to calibrate membership scores, but reproducing TSFM pretraining at that scale is expensive ( 39 ; 6 ; 20 ) . These limitations call for a contamination auditing method designed specifically for TSFMs. We make the following empirical observation: under a fine tuning probe, contaminated datasets tend to exhibit faster loss reduction and require less backbone movement than clean datasets as shown at Fig. 1 and confirmed by the feature ablation in Table 4 . Motivated by this phenomenon, we propose TSFMAudit , a contamination auditing framework for TSFMs based on learning dynamics. TSFMAudit extracts probe time signals such as loss reduction and parameter displacement to infer contamination risk. We summarize our contributions as follows: • To the best of our knowledge, this is the first work to study pretraining contamination auditing for TSFMs. We formalize the dataset level audit objective for contamination. • We propose TSFMAudit 1 1 1 Code is available at: https://github.com/kkevin117/TSFMAudit . , a contamination auditing framework specifically designed for TSFMs, which infers contamination risk from probe time adaptation dynamics. • We evaluate TSFMAudit on 6 TSFMs using their reported pretraining corpora, GIFT-Eval ( 1 ) , and the TIME ( 34 ) benchmark, covering 187 datasets in total. Compared with 10 contamination auditing baselines adapted from the LLM literature, TSFMAudit achieves consistently stronger detection performance. 2 Related Work TSFMs and Benchmarks Contamination Concerns. Numerous recent works on time series forecasting have moved from small, dataset-specific evaluations toward standardized large-scale benchmarks such as M4, LSF, and the Monash archive, and, with the rise of time series foundation models, further toward broad zero-shot evaluation suites such as GIFT-Eval, TSFM-Bench, and TIME 28 ; 18 ; 42 ; 1 ; 24 ; 34 . In parallel, TSFMs themselves are increasingly trained on large heterogeneous corpora collected from diverse real-world and synthetic sources, reflecting the field’s ambition to build universal forecasters across domains 2 ; 11 ; 41 ; 3 . However, this development has also raised a new concern about benchmark validity: many modern benchmarks are assembled by reusing public datasets, evaluation data may overlap with, or remain implicitly connected to, pretraining corpora, leading to hidden information leakage and overly optimistic estimates of zero-shot generalization 1 ; 30 . This problem is particularly acute in time series, where fragmented naming and versioning, weak semantic transparency, and strong temporal dependence make contamination harder to detect than in other modalities 30 . These concerns have recently motivated a new wave of benchmark design centered on fresher data sources and stricter evaluation protocols intended to reduce leakage and restore the credibility of TSFM comparison 1 ; 34 . Contamination Auditing in LLMs. Large language models have motivated a broad literature on contamination auditing. Existing methods mainly fall into four families: corpus-side overlap detection based on n-gram or token-level matching, memorization or extraction attacks that test whether benchmark content can be reproduced, perplexity-based methods that treat unusually low likelihood as evidence of prior exposure, and membership inference attacks (MIA) such as LiRA that estimate whether a sample was included in training 5 ; 31 ; 8 ; 7 ; 12 ; 38 ; 32 ; 6 . Despite the rich literature on contamination auditing in LLMs, these methods do not transfer directly to TSFMs. Overlap-based and extraction-based approaches rely on discrete tokens or verbatim matching, whereas time series are continuous values and the same underlying signal may reappear after rescaling, resampling, rewindowing, or renaming, making exact matching both brittle and incomplete 45 ; 22 ; 30 . Perplexity- or loss-based criteria are also especially unreliable in time series, because many datasets are

Read the complete text