CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streams
Ashok Devireddy, Shunping Huang · 2025
Read CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streams by Ashok Devireddy, Shunping Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-21273.txt.
Introduction
Sensors pour out data streams that drift over time, and yesterday's anomaly detector quietly goes stale. Ashok Devireddy and Shunping Huang present CALM, a framework that keeps retraining itself as the stream flows. A forecasting model flags odd points; the system then fine-tunes on fresh data in a closed loop. A large language model acts as judge, deciding which flagged points are noise and which mark a real shift worth learning from.
Built on a distributed processing engine, the system runs in near real time. On a broad public benchmark, the ever-updated model beat the frozen starting model on most datasets. The team argues that self-refresh, guided by a judging model, is the way to watch changing streams.
Catalog record
| CREATOR | Ashok Devireddy, Shunping Huang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-21273
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streams Ashok Devireddy, Shunping Huang Abstract The detection of anomalies in non-stationary time-series streams is a critical but challenging task across numerous industrial and scientific domains. Traditional models, trained offline, suffer significant performance degradation when faced with concept drift, where the underlying statistical properties of the data change over time. This paper introduces CALM (Continuous, Adaptive, and LLM-Mediated), a novel, end-to-end framework for real-time anomaly detection designed to address this challenge. CALM is built on the Apache Beam distributed processing framework and leverages the TimesFm foundation model for forecasting-based anomaly detection. The framework’s novelty lies in two core contributions. First, it implements a closed-loop, continuous fine-tuning mechanism that allows the anomaly detection model to adapt to evolving data patterns in near real-time. Second, it introduces an “LLM-as-a-Judge” component—a Large Language Model that provides semantic, context-aware judgments on detected anomalies to curate a high-quality training dataset, deciding whether an anomaly represents transient noise or a meaningful pattern shift. We evaluate CALM on the comprehensive TSB-UAD benchmark. Our results demonstrate that the continuously fine-tuned model improves the ROC AUC score in most datasets compared to the static, pre-trained base model, validating the efficacy of our adaptive, LLM-guided approach to maintaining high-performance anomaly detection in dynamic streaming environments. 1 Introduction Time-series anomaly detection is a fundamental task with profound implications across a wide array of domains, including financial markets, industrial manufacturing, and IT systems monitoring. The timely identification of anomalous events—observations that deviate significantly from expected behavior—can prevent substantial economic losses, preempt system failures, and ensure operational reliability. As data generation becomes increasingly continuous and voluminous, the analysis of these time-series streams has become paramount. A primary challenge in real-world streaming environments is the phenomenon of concept drift , where the statistical properties of the data stream evolve over time. This non-stationarity invalidates the core assumption of many classical machine learning models, which are trained on static, historical datasets. A model that performs well at deployment time may see its accuracy degrade rapidly as new patterns emerge, a problem that necessitates a shift from static modeling to adaptive, online learning paradigms. The recent advent of large-scale, pre-trained foundation models for time-series, such as TimesFm, represents a significant advancement. Trained on vast and diverse datasets comprising billions of data points, these models exhibit impressive zero-shot forecasting capabilities, providing a powerful baseline for various downstream tasks. However, their generalist nature, while a strength for broad applicability, can be a limitation in specialized, dynamic domains. Research suggests that while these models can be adapted for anomaly detection, they may not outperform specialized approaches without further tuning, particularly when faced with domain-specific concept drift ( Fu et al. 2024 ) . This paper proposes that the solution lies not in choosing between a generalist foundation model and a specialized, adaptive one, but in architecting a system that achieves the best of both. While significant research has advanced scalable stream processing ( Akidau et al. 2015 ) , the power of time-series foundation models ( Liang et al. 2024 ) , and the reasoning capabilities of LLMs ( Zheng et al. 2023 ) , these powerful paradigms have largely evolved in parallel. The critical gap, which this paper addresses, lies in their architectural synthesis. We argue that the next frontier in intelligent systems is not a better isolated algorithm, but a more integrated, self-correcting architecture. The true innovation is a novel architectural synthesis of three powerful and distinct technological paradigms: (1) scalable, stateful stream processing for robust data handling; (2) time-series foundation models for powerful, out-of-the-box forecasting; and (3) Large Language Models (LLMs) as sophisticated reasoning agents. By integrating these components into a cohesive, closed-loop system, we create a new class of intelligent infrastructure capable of autonomous adaptation. To this end, we introduce CALM (Continuous, Adaptive, and LLM-Mediated), a holistic framework that leverages the zero-shot power of a foundation model as a starting point but integrates it into a fully automated, adaptive pipeline that specializes it over time. The CALM framework makes the following primary contributions: 1. A Scalable Streaming Architecture: An end-to-end anomaly detection system architected on Apache Beam, specifically designed to handle high-throughput, out-of-order data streams. It employs stateful processing primitives to ensure correctness and robustness in a distributed environment ( Akidau et al. 2015 ; Apache Beam Community 2024 ) . 2. The LLM-as-a-Judge Mechanism: A novel component that functions as an online, semantic concept drift detector. Moving beyond the typical use of LLMs for offline evaluation ( Patronus AI 2025 ) , our judge actively curates the fine-tuning dataset by providing reasoned judgments on anomalies, distinguishing transient noise from genuine pattern shifts to guide the model’s adaptation ( Feki 2023 ) . 3. A Closed-Loop Continuous Fine-Tuning Pipeline: A fully automated MLOps loop ( Feki 2023 ; Google Cloud 2024b ) that collects data deemed valuable by the LLM Judge, batches it for training, triggers a fine-tuning job, and enables the dynamic, in-flight deployment of the improved model. This allows the system to autonomously adapt to concept drift without human intervention. We validate the CALM framework through a rigorous experimental evaluation on the TSB-UAD benchmark, demonstrating that our adaptive, LLM-guided approach yields significant performance improvements over a static, pre-trained model. 2 Related Work 2.1 Taxonomy of Time-Series Anomaly Detection A time series is a sequence of data points indexed chronologically ( Salehi et al. 2024 ) . Such series can be univariate , consisting of a single variable observed over time, or multivariate , comprising multiple variables recorded concurrently ( Liang et al. 2024 ) . Anomalies within these series are broadly categorized into three types: point anomalies , which are single data points deviating from the norm; contextual anomalies , which are normal in a global sense but abnormal within their local context; and collective anomalies , which are sequences of data points that, as a group, represent an anomalous pattern ( Chandola et al. 2009 ; Hawkins 1980 ) . The methodologies for detecting these anomalies are diverse, but a common taxonomy classifies them into four main categories: forecasting-based, reconstruction-based, representation-based, and hybrid methods ( Salehi et al. 2024 ; Blázquez-García et al. 2021 ) . The CALM framework falls squarely within the forecasting-based paradigm. The fundamental principle of this approach is to train a model to predict future values of the time series based on its recent history. The deviation between the model’s forecast and the actual observed value, often termed the prediction error or residual, serves as the anomaly score ( Laptev et al. 2015 ; Malhotra et al. 2015 ) . A data point is flagged as an anomaly if this error surpasses a predefined threshold ( Blázquez-García et al. 2021 ; Hundman et al. 2018 ) . Our work refines this by using the model’s quantile forecasts to establish a dynamic and robust statistical threshold. 2.2 The Landscape of Time-Series Foundation Models Inspired by the transformative success of LLMs in Natural Language Processing (NLP), a recent paradigm shift has occurred in time-series analysis towards the use of large, pre-trained foundation models, often called Time-Series Foundation Models (TSFMs) ( Liang et al. 2024 ; Wen and Liang