Towards Foundation Auto-Encoders for Time-Series Anomaly Detection Conference: ; August 25, 2024; Barcelona, Spain

Gastón García González, ( † \dagger ) Pedro Casas, Emilio Martínez, Alicia Fernández · 2025

Read Towards Foundation Auto-Encoders for Time-Series Anomaly Detection Conference: ; August 25, 2024; Barcelona, Spain by Gastón García González, ( † \dagger ) Pedro Casas, Emilio Martínez, Alicia Fernández (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-01875.txt.

Introduction

Gastón García González and co-authors chase a foundation model for time-series data in this 2025 paper. Foundation models, trained once on huge data, reshaped language and vision work. The team asks whether the same idea can serve anomaly spotting in streams such as network traffic, where odd patterns flag faults or attacks.

Their model, called FAE, builds on variational auto-encoders paired with widened convolutional networks. It trains on large amounts of single-channel time-series data to learn normal shapes and rhythms. The hope is a model that works out of the box on new data sets, with no fresh training. Early tests span several fields, including a live mobile operator data set and a well-known contest benchmark, and the authors present them as first steps toward that goal.

Catalog record

CREATORGastón García González, ( † \dagger ) Pedro Casas, Emilio Martínez, Alicia Fernández
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-01875

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Foundation Auto-Encoders for Time-Series Anomaly Detection Conference: ; August 25, 2024; Barcelona, Spain Gastón García González, ( † \dagger ) Pedro Casas, Emilio Martínez, Alicia Fernández Abstract . We investigate a novel approach to time-series modeling, inspired by the successes of large pretrained foundation models. We introduce FAE (Foundation Auto-Encoders), a foundation generative-AI model for anomaly detection in time-series data, based on Variational Auto-Encoders (VAEs). By foundation, we mean a model pretrained on massive amounts of time-series data which can learn complex temporal patterns useful for accurate modeling, forecasting, and detection of anomalies on previously unseen datasets. FAE leverages VAEs and Dilated Convolutional Neural Networks (DCNNs) to build a generic model for univariate time-series modeling, which could eventually perform properly in out-of-the-box, zero-shot anomaly detection applications. We introduce the main concepts of FAE, and present preliminary results in different multi-dimensional time-series datasets from various domains, including a real dataset from an operational mobile ISP, and the well known KDD 2021 Anomaly Detection dataset. 1. Introduction While the literature offers a plethora of traditional statistical models for anomaly detection in time-series data, they often struggle with the non-stationary, non-linear, and noisy nature of real data, leading to suboptimal predictions. In recent years, there has been a surge in the adoption of modern deep learning-based approaches for time-series anomaly detection ( Pang et al. 2021 ; Schmidl et al. 2022 ) , owing to their ability to handle complex dependencies and generate realistic data sequences. Generative AI methodologies, in particular, have gained attention for their performance in time-series modeling ( Lai et al. 2018 ; Yoon et al. 2019 ; García González et al. 2021 ) . In this paper, we focus on devising a Generative AI model capable of matching or even surpassing the performance of conventional time-series modeling methods without the need for training on the specific target dataset - a concept known as Zero-Shot Learning (ZSL). ZSL is a problem setup in deep learning where, at test time, a learner observes samples from classes which were not observed during training, and needs to predict the class that they belong to. We therefore investigate if a model pretrained on multiple time-series data can learn temporal patterns useful for accurate forecasting on previously unseen time-series. For doing so, we use as starting point the DC-VAE model ( García González et al. 2023 ) , a deep-learning-based, unsupervised, and multivariate approach to real-time anomaly detection in MTS data, based on popular Variational Auto-Encoders (VAEs) ( Kingma and Welling 2013 ) . VAEs are generative AI models that learn the underlying distribution of the data and can generate new samples from this distribution. In the context of time-series data, VAEs can capture latent representations of temporal patterns and generate sequences that exhibit similar characteristics, making them powerful for generalization and ZSL. VAEs learn a low-dimensional latent space representation of the input data, which captures the underlying structure of the data in a compressed form. By learning meaningful representations, VAEs can generalize well to unseen data points that lie within the same distribution as the training data, supporting generalization to new instances. As generative models, VAEs can generate new samples from the learned latent space distribution. potentially enabling ZSL, as the model can produce samples that belong to unseen classes or categories without explicitly training on them. By sampling from the latent space, VAEs can generate diverse and realistic data points even for classes not present in the training set. We introduce and investigate FAE (Foundation Auto-Encoders), a foundation generative-AI model for anomaly detection in time-series data, based on VAEs. FAE uses DC-VAE’s network architecture ( García González et al. 2023 ) , originally designed for multivariate anomaly detection. In particular, it leverages VAEs and Dilated Convolutional Neural Networks (DCNNs) to build a generic model for univariate time-series modeling , which could eventually perform properly in out-of-the-box, zero-shot anomaly detection applications. The reasons for moving from multivariate to univariate time-series analysis are twofold: from an architectural point of view, we become independent of the spatial dimensionality of a MTS dataset – i.e., we fix the spatial input dimensionality to one – and can therefore apply exactly the same architecture without any modifications; from an analytics perspective, while a univariate model is at a disadvantage compared to a multivariate model – i.e., it loses access to cross-correlational information, which we have shown might be critical for better data modeling ( García González et al. 2023a ; García González et al. 2023 ) – the univariate version puts the focus exclusively on the temporal behavior of the data, which is exactly the target of the generalization we are looking for – we want a model that generalizes across the temporal dimension and not necessarily the spatial one, which varies among different problems. We introduce the main concepts and ideas behind FAE, along with its network architecture, and present preliminary results in the analysis of two different datasets: (1) a multi-dimensional network monitoring dataset – TELCO ( García González et al. 2023b ) , collected from an operational mobile Internet Service Provider (ISP), recently released openly to the research community; and (2) a large-span dataset of 250 multi-domain time series released for the KDD2021 yearly data contest ( Keogh et al. 2021 ) . The remainder of the paper is organized as follows: Section 2 presents an overview of the related work. In Section 3 we describe the FAE model and its underlying network architecture. Section 4 reports the preliminary results obtained with FAE on the analysis of time-series from TELCO and KDD2021, with a particular focus on generalization and zero-shot modeling. Discussion on the potential of FAE, along with its limitations, is presented in Section 5 . Finally, Section 6 concludes the paper. 2. Related Work The diversity of data characteristics and types of anomalies results in a lack of universal anomaly detection models. Modern approaches to time-series anomaly detection based on deep learning technology have flourished in recent years ( Pang et al. 2021 ; Schmidl et al. 2022 ) . Due to their data-driven nature and achieved performance in multiple domains, generative models such as VAEs and Generative Adversarial Networks (GANs) ( Goodfellow et al. 2014 ) have gained relevance in the anomaly detection field ( Zavrak and Iskefiyeli 2020 ; Zenati et al. 2018 ; Chen et al. 2021 ; Donahue et al. 2016 ; Li et al. 2019 ; Geiger et al. 2020 ; García González et al. 2021 ) . VAEs ( Kingma and Welling 2013 ; Doersch 2016 ; Kingma and Welling 2019 ) represent a powerful and widely-used class of models to learn complex data distributions. Unlike GANs, a potential limitation of VAEs is the prior assumption that latent sample representations are independent and identically distributed. While this is the most common assumption followed in the literature, there is ongoing research on the benefits of accounting for covariances between samples in time to improve model performance ( Casale et al. 2018 ; Girin et al. 2019 ; Fortuin et al. 2020 ; Ramchandran et al. 2021 ) . For example, while the original work ( Kingma and Welling 2013 ) assumes that the prior over the parameters and

Read the complete text