Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning

1 st Bahareh Golchin, 2 nd Banafsheh Rekabdar · 2025

Read Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning by 1 st Bahareh Golchin, 2 nd Banafsheh Rekabdar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-02999.txt.

Introduction

Bahareh Golchin and Banafsheh Rekabdar published a paper in 2025 on spotting anomalies in streams of data over time, such as sensor feeds and market series. Older detectors need hand tuning and freeze when new kinds of oddity appear. The authors combine three learning ideas so the system can keep adapting with very few labeled examples.

A memory network first learns the shape of normal sequences, and a variational model compresses them into a compact code. A reinforcement learner then decides which points deserve a human label, so each label teaches the most. Tests on real data sets beat existing methods, according to the authors. The design targets control rooms where oddities are rare, costly, and always changing form.

Catalog record

CREATOR1 st Bahareh Golchin, 2 nd Banafsheh Rekabdar
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-02999

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning 1 st Bahareh Golchin, 2 nd Banafsheh Rekabdar Abstract A novel approach to detecting anomalies in time series data is presented in this paper. This approach is pivotal in domains such as data centers, sensor networks, and finance. Traditional methods often struggle with manual parameter tuning and cannot adapt to new anomaly types. Our method overcomes these limitations by integrating Deep Reinforcement Learning (DRL) with a Variational Autoencoder (VAE) and Active Learning. By incorporating a Long Short-Term Memory (LSTM) network, our approach models sequential data and its dependencies effectively, allowing for the detection of new anomaly classes with minimal labeled data. Our innovative DRL-VAE and Active Learning combination significantly improves existing methods, as shown by our evaluations on real-world datasets, enhancing anomaly detection techniques and advancing time series analysis. I Introduction Detecting anomalies in time series plays a key role in different areas, such as data centers, sensor networks, cyber-physical systems, and finance [ 1 , 2 , 3 , 4 ] . Manual tuning of parameters and features and specific data properties is required in most of the existing methods in the literature. Furthermore, due to large amounts of data, manually identifying anomalies in time series data: 1) takes a lot of time and labor, and 2) is likely to be influenced by human errors. Therefore, an automated system is needed to detect anomalies in extensive time series data [ 5 ] . Detecting anomalies presents two main challenges: First, because anomalies are rare, it is difficult to train models effectively to spot them. Second, the fact that real-world data often changes over time adds another layer of complexity. In response to the scarcity of labeled data, many anomaly detection algorithms have been suggested typically unsupervised, which do not require labeled data. These algorithms are usually based on specific assumptions about anomaly patterns observed in the data. However, these assumptions may not always align with real-world situations, leading to high false-positive rates. This mismatch arises from varying user interests and definitions of anomalies [ 6 , 7 ] . Supervised methods are highly effective when sufficient labeled data is available but face difficulties in scenarios with limited or no labels. These methods assume that the underlying distribution stays the same, even when labels are available. If the distribution changes, they need to be retrained [ 8 ] . Next, the approach utilized to detect anomalies in time series data is semi-supervised learning. This method is not suitable for situations where there are many different types of anomalies because it only works with a small number of known anomalies, and it can miss new anomalies in data that has not been labeled. Thus, this approach cannot detect new types of anomalies [ 9 ] . To tackle the issues mentioned above, namely the problem of weakly-supervised anomaly detection in time series data, we utilize Deep Reinforcement Learning (DRL). The exploration versus exploitation dilemma is crucial in Reinforcement Learning (RL) [ 10 ] . Taking actions that are best based on what is already known is called exploitation in the literature. However, exploration is trying new things to find better actions. The agent must balance between acting optimally with current knowledge and seeking more knowledge. The DRL used in our study is developed to efficiently utilize a small portion of labeled anomalous data ( D_{la} ). This approach extensively explores a large pool of unlabeled data ( D_{u} ), which detects new classes of anomalies not covered by the labeled data. This exploration is enhanced by the integration of a Variational Autoencoder (VAE), which powers the DRL framework. Furthermore, recognizing the high cost and scarcity of fully labeled data in real-world scenarios, active learning has been integrated into our system which enables our RL agent to 1) explore the environment and accumulate experience effectively, and 2) make informed queries based on this experience during its exploration. Long Short-Term Memory (LSTM) network is the core of our DRL agent. This 1) simulates sequential time series data, and 2) extracts the long-term dependencies between activities [ 11 ] . To summarize, our key contributions include the following: • To the best of our knowledge, the combination of DRL-VAE and an Active Learning approach (i.e., RLVAL) to detect anomalies in time series data proposed in this paper is one of the first studies in the literature. • We investigate the use of LSTMs to enhance the robustness of time series data modeling and integrate these networks into our DRL framework. • We evaluate our approach on two time series datasets (i.e., Yahoo and KPI). Our results demonstrate that RLVAL surpasses previous state-of-the-art methods. In what follows, we lay out the structure of the rest of this paper. In Section II, we review the studies in the literature which is related to our work. Moreover, the background in anomaly detection is explored. Next, our proposed framework is detailed in Section III. Section IV discusses the implementation, including a comprehensive examination of the datasets used. Finally, we conclude our study in Section V. II Related Work Recently, detecting anomalies has been the focus of numerous methodologies. Broadly, we can classify these methods as follows. 1) Statistical-based methods, and 2) machine learning-based approaches. II-A Statistical-based Methods Statistical-based models involve building a model from given datasets, and then, using mathematical tests to decide if the unseen data fits the proposed model. Kernel function-based methods directly learn normal behavior from the training data [ 12 , 13 ] . Parametric statistical models, such as Gaussian, regression, and logistic regression models, assume the underlying distribution of normal data follows an existing distribution [ 14 ] . However, these methods assume that the normal behavior fits an existing distribution which is not fair in practice. II-B Machine Learning-based Methods Machine learning-based approaches use labeled training data to differentiate between normal and abnormal data, achieving this through either classification or clustering techniques. Common algorithms include Bayesian networks [ 15 ] , support vector machines [ 16 ] , rule-based systems [ 17 ] , and neural networks [ 18 ] . Clustering algorithms, such as k-means are also used to detect anomalies [ 19 ] . Next, for complex time series data, specialized techniques and algorithms have been developed. Examples include 1) Skyline [ 20 ] , which detects anomalies in real-time, and 2) Twitter’s package, which detects anomalies where seasonality and trends are present [ 21 ] . Contextual anomaly detection, exemplified by ContextOSE [ 22 ] , focuses on capturing local information rather than global patterns. Hierarchical temporal memory (HTM), seen in projects like Numenta and Numenta TM, stores and recalls temporal and spatial patterns [ 23 ] . Along with the advancement of deep learning, to detect anomalies through time series data, Recurrent Neural Networks (RNN) or LSTM models have been developed. These models learn from normal training data to predict future values and detect anomalies based on prediction errors. Variants based on autoencoders have also been explored [ 24 ] . Recently, RL has attracted attention to detect anomalies in time series data. The reason is that it has a generic framework, and it

Read the complete text