DeepFilter: A Transformer-style Framework for Accurate and Efficient Process Monitoring
Hao Wang, Zhichao Chen †, Licheng Pan, Xiaoyu Jiang, Yichen Song, Qunshan He, Xinggao Liu † · 2025
Read DeepFilter: A Transformer-style Framework for Accurate and Efficient Process Monitoring by Hao Wang, Zhichao Chen †, Licheng Pan, Xiaoyu Jiang, Yichen Song, Qunshan He, Xinggao Liu † (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-01342.txt.
Introduction
Hao Wang and co-authors rework the transformer for watching industrial processes in this 2025 paper. Process monitoring demands sharp accuracy on log data at low cost. Standard attention blocks blur the meaning of log entries and chew through compute, so the team revised the mechanism itself.
Their framework, DeepFilter, filters attention to grasp context better while running leaner. The authors present it as a straightforward yet flexible baseline: useful for teams starting a monitoring project and for those upgrading existing systems. By fixing attention rather than piling on modules, they aim for a design practitioners can adopt without exotic machinery, balancing precision against the strict budgets of plant-floor computing.
Catalog record
| CREATOR | Hao Wang, Zhichao Chen †, Licheng Pan, Xiaoyu Jiang, Yichen Song, Qunshan He, Xinggao Liu † |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-01342
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DeepFilter: A Transformer-style Framework for Accurate and Efficient Process Monitoring Hao Wang, Zhichao Chen †, Licheng Pan, Xiaoyu Jiang, Yichen Song, Qunshan He, Xinggao Liu † Abstract Impact statement The process monitoring task is characterized by stringent demands for accuracy and efficiency. Current transformer-based methods, characterized by self-attention for temporal fusion, exhibit limitations in accurately understanding the semantic context and efficiently processing monitoring logs, rendering them inadequate for process monitoring. To address these limitations, we introduce DeepFilter, which revises the self-attention mechanism to improve both accuracy and efficiency. As a straightforward yet versatile approach, DeepFilter provides an instrumental baseline for practitioners in process monitoring, whether initiating new projects or enhancing existing capabilities. Interested readers can implement DeepFilter following the code in the supplementary file. The monitoring logs are available and daily updated on https://data.rmtc.org.cn/gis/PubIndex.html . I Introduction Monitoring the quality variable through advanced time-series analysis techniques is paramount for ensuring operational safety in industrial automation [ 1 , 2 ] . These techniques are widely employed in process engineering [ 3 ] , manufacturing [ 4 , 5 ] , and energy conversion [ 6 ] , where the growing demands for higher efficiency and cost-effectiveness often drive equipment to operate under extreme conditions, thereby increasing the likelihood of catastrophic failures [ 6 , 7 ] . For example, in chemical engineering, the Haber-Bosch process for ammonia synthesis requires temperatures above 400°C and pressures exceeding 200 bar, escalating risks of equipment failure and hazardous leaks [ 8 ] . Similarly, nuclear power plants operate reactors at high pressures and temperatures to maximize output, heightening the threat of catastrophic radiation leakage [ 9 ] . These examples underscore the critical role of process monitoring systems for safeguarding operations in industrial automation [ 10 , 11 ] . Process monitoring operates by firstly estimating next-step values of the quality variable from historical monitoring logs, and subsequently identifying anomalies by detecting significant deviations between predicted and observed values [ 6 , 10 ] . An instrumental process monitoring system must satisfy the dual requirements of high accuracy and computational efficiency for safeguarding industrial equipments. For example, in large-scale chemical plants [ 8 , 12 ] , precise estimation of temperature and pressure is necessary to detect their subtle shifts, enabling operators to intervene before potential reactor instabilities escalate. Meanwhile, real-time efficiency ensures these corrective measures are executed promptly, minimizing the risk of safety incidents or costly downtime. To achieve accurate and efficient process monitoring, a variety of data-driven algorithms have been developed, evolving from early identification methods to advanced deep learning models. Traditional methods such as ARIMA [ 13 ] offer computational simplicity but are limited in handling the nonlinear complexity of industrial data. Statistical methods, including decision trees [ 14 ] , XGBoost [ 15 ] , and regression-based approaches [ 16 ] , provide improved accuracy through richer feature extraction but rely heavily on manual engineering and struggle to scale with large datasets. In contrast, deep learning architectures—spanning convolutional [ 17 ] , recurrent [ 18 , 19 ] , and graph neural networks [ 20 , 21 ] —enable automated feature extraction and GPU acceleration, driving both performance and speed. Notably, Transformers [ 22 ] have emerged as an exemplar solution, characterized by the self-attention layer for capturing temporal patterns. This layer dynamically computes weighted dependencies across all time steps in the monitoring logs, capturing step-wise relationships useful for prediction [ 23 ] . Additionally, the architecture is optimized for GPU acceleration and is well-suited for large-scale monitoring data [ 22 ] processing. These advancements have made Transformers a preferred choice in industrial time-series analytics [ 23 , 24 , 25 ] 1 1 1 The standard Transformer consists of encoder and decoder components. In this paper, ”Transformer” refers to the encoder component commonly used in time-series analytics, rather than the full encoder-decoder architecture. . Despite the promise of Transformers, we contend that the self-attention mechanism exhibits two fundamental limitations that restrict its suitability for the stringent demands of process monitoring. ❶ The reliance on step-wise correlations results in suboptimal accuracy for process monitoring , since it hinders the modeling of long-term discriminative patterns. Specifically, discriminative patterns in monitoring logs—such as gradual drifts, recurring trends, or periodic fluctuations—often emerge at the sub-sequence level and may require modeling long-term histories. It would be difficult to model them by calculating step-wise relationships in a learned space. Moreover, individual sensor observations in industrial monitoring logs carry limited semantic meaning, further hindering the model’s ability to extract discriminative patterns necessary for accurate monitoring. ❷ The reliance on step-wise correlations results in suboptimal efficiency for process monitoring. Each self-attention layer requires pairwise interactions across all time steps, leading to quadratic computational and memory costs. Collectively, canonical Transformers fall short of delivering the required accuracy and efficiency in process monitoring. To overcome these limitations, we propose DeepFilter , a refined Transformer that replaces the self-attention layer with a novel efficient filtering layer specifically tailored for process monitoring. This layer performs adaptive filtering across the entire temporal sequence, effectively modeling long-term discriminative patterns. Theoretical analysis further confirms its capacity to enhance representations of long-term discriminative patterns. Moreover, by eliminating the quadratic complexity inherent in self-attention, the efficient filtering layer significantly reduces computational overhead. Extensive evaluations on real-world datasets demonstrate that DeepFilter consistently delivers superior accuracy and efficiency relative to state-of-the-art models, highlighting its role as an instrumental approach for Transformer-based process monitoring. Organization. Section II reviews related works and highlights technical gaps. Section III introduces the preliminary concepts essential to our methodology. Section IV provides a detailed description of the DeepFilter architecture. Section V presents a case study on process monitoring in real-world nuclear power plants, showcasing DeepFilter’s accuracy and efficiency improvements. Finally, we summarize our conclusions and limitations, and outline directions for future research. II Related Works In the era of Industry 4.0, reliability has emerged as a critical aspect in algorithm design and equipment maintenance [ 26 , 27 , 28 ] . Alongside tasks like soft sensing [ 29 , 30 , 31 ] , fault detection [ 32 , 33 ] , and data recovery [ 34 , 35 , 36 ] , process monitoring serves as a vital safeguarding strategy by detecting early signs of faults or outliers [ 1 , 2 ] . Modern process monitoring approaches are broadly classified into three categories: identification methods, statistical methods, and deep methods. Early identification methods, such as Auto-Regressive (AR), Moving Average (MA), and their integrated forms (ARIMA) [ 13 ] , offer computational simplicity and real-time processing. However, their effectiveness is limited by an inability to capture nonlinear temporal dependencies. In contrast, statistical methods such as decision trees [ 15 ] and generalized linear models [ 16 ] can model nonlinear patterns, but they typically require intensive manual feature engineering and face scalability challenges in large-scale industrial applications. The advent of deep learning has revolutionized process monitoring by enabling automatic feature extraction and enhanced parallel computing. Various architectures, such as Convolutional Neural Networks (CNNs) [ 17 ] , Recurrent Neural Networks (RNNs) [ 18 , 19 ] , and Graph Neural Networks (GNNs) [ 21 , 20 ] , have been developed to