An AM-CNN-BiGRU network with spatiotemporal feature fusion for industrial robot predictive maintenance Authors: Zihao Zang, Yiwen Zhang, Shouxin Ruan, Lei Wang, Xinming Zhang, Linsen Song, Yibo Huang, Shuhong Ye Published: 2025 ABSTRACT Due to aging, high operational loads, and harsh working environments, industrial robots are prone to various failures. Predictive maintenance (PdM) offers a promising solution to this challenge. This study proposes a PdM method for industrial robots based on AM-CNN-BiGRU neural network. First, a convolutional neural network (CNN) and a bidirectional gated recurrent unit (BiGRU) are integrated to leverage the CNN’s ability to extract local spatial features and the BiGRU’s capability for learning global bidirectional temporal dependencies. This combination effectively captures the nonlinear and deep features within the data. Second, vibration, current, and torque data from the robot joints are input into the CNN as distinct channels, enabling multi-source data fusion. Concurrently, an attention mechanism (AM) is incorporated to assign adaptive weights to different data sources, enhancing the identification of critical fault signatures. Finally, ablation studies were conducted, and comparative experiments were performed between AM-CNN-BiGRU and several of the most representative and latest algorithms in the time series forecasting domain. During training, the Lion optimizer was employed for the deep learning models. The results demonstrate that AM-CNN-BiGRU achieved optimal performance across four evaluation metrics: R², MAE, MAPE, and RMSE, thereby validating its effectiveness and reliability. Introduction Industrial robots have been used to automate manufacturing processes due to their intelligence and high degree of integration to improve productivity, production quality, and safety1. However, with aging, high loads, and harsh operating environments, industrial robots are exposed to various failure problems such as accuracy degradation and component damage2. This can seriously affect the robot’s performance and efficiency, and can lead to unplanned downtime of the entire production line, resulting in significant economic losses3. The emergence of predictive maintenance (PdM) provides a new approach to address this problem. PdM is a maintenance methodology that continuously monitors equipment and issues alerts before equipment failure occurs. Its core function is failure prediction so that equipment maintenance can be performed before a failure occurs. Therefore, condition monitoring as well as failure prediction of industrial robots can significantly reduce maintenance costs and improve equipment utilization4,5. In the industrial field, there are three main approaches to predictive maintenance, including reliability-based PdM, model-based PdM, and data-driven PdM6. Reliability-based PdM typically utilizes the statistical characteristics of historical failure data for fault prediction. Consequently, it requires neither the physical structural characteristics of the equipment nor extensive sensor data7. However, reliability-based predictive maintenance is better suited for structurally simple equipment or components and necessitates substantial long-term experimentation and statistical analysis. Model-based PDM typically analyzes the correct behavior of equipment to construct mathematical or physical models reflecting the physical laws underlying performance degradation8. Zhang et al.9,10 proposed a novel expectation maximization-unscented particle filter-Wilcoxon rank sum test (EM-UPF-W) method to adaptively estimate noise variables in degradation models and accurately detect capacity regeneration points under conditions of unlabeled small sample data. Additionally, they introduced an adaptive remaining useful life (RUL) prediction approach based on Kalman filtering and Rauch-Tung-Striebel expectation maximization (KF-EM-RTS) to address the uncertainty in parameter estimation and the lack of historical data for individual batteries. Nevertheless, establishing precise physical models for complex equipment is challenging, which adversely affects the accuracy of predictive maintenance outcomes. Therefore, neither approach is well suited for complex and high-value equipment, such as industrial robots. Data-driven PdM eliminates the requirement for detailed mechanistic knowledge of equipment. Instead, it only needs to start from the data and comprehensively mine the features of the large amount of production data, thus realizing the accurate prediction of faults11,12. Data-driven approaches primarily encompass machine learning and deep learning. Machine learning has been widely applied in PdM. Zhang et al.13 proposed a lifetime extension approach based on a Levenberg-Marquardt back-propagation neural network (LM-BPNN) and power routing in an interleaved DC-DC boost conversion system, aiming to accurately estimate the state of health without identifying a systematic mathematical model and to extend the system’s operational lifetime. Li et al.14 proposed a PdM framework for rotating motors based on machine learning classification and regression modeling and aimed to predict the specific types of faults that may occur in a given motor and determine its RUL. Baptista et al.15 introduced a PdM method combining an autoregressive moving average (ARMA) model with a support vector machine (SVM) regression model. This approach analyzes and extracts health indicators and performance degradation features hidden within equipment data to predict the risk of future failures in specific components or systems. Xu et al.16 proposed a method for detecting bolt loosening in industrial robot joints based on electromechanical modeling and motor current signature analysis (MCSA). They established a dynamic equation coupling motor current with bolt loosening, extracted time-frequency characteristics from the motor current, and subsequently employed an SVM algorithm for bolt loosening detection. Izagirre et al.17 presented a data acquisition and PdM network architecture for industrial robots. They utilized support vector regression (SVR) and extreme learning machine (ELM) algorithms to detect degradation states in robot joints. Validation was performed on a real automotive assembly line by extracting torque signals via a programmable logic controller (PLC) as model inputs. Raouf et al.18 implemented fault prediction for six types of failures in industrial robot drive systems using an SVM model with three-phase current signals as inputs. Pan et al.19 extracted features from vibration signals, which served as inputs to an artificial neural network (ANN) algorithm, achieving prediction of joint backlash faults in industrial robots. Deep learning, as a special machine learning method, has excellent performance in big data processing, deep feature adaptive extraction, and fault prediction by virtue of the design of deep architectures20,21. Many researchers have made significant progress in the field of deep learning, especially in fault prediction. He et al.22 proposed a multiscale mixed convolutional neural network (MSMCNN) based on CNN, which can adaptively extract the depth features of the vibration signals of the robot joints and realize the fault prediction of the harmonic reducer in the robot joints. Long et al.23 proposed a multiscale convolutional capsule network (MCCN), which can effectively predict the fault of industrial robots by collecting attitude data, extracting multi-scale features using CNN, and identifying the spatial relationship in the attitude data using CapsNet. Chen et al.24 proposed a lightweight detection model (Micro-Net) based on a multifunctional local convolutional algorithm and used current signals as inputs to achieve fault prediction in industrial robots. Pan et al.25 proposed a method for predicting joint failures in industrial robots based on a Deep Residual Neural Network (DRNN). Hakami26 aiming to address the challenge of scarce fault data in predictive maintenance, employed Generative Adversarial Networks (GANs) to synthesize data and utilized Long Short-Term Memory (LSTM) networks to extract temporal features. Joseph et al.27 developed a hyperparametrically tuned LSTM model for predictive maintenance of industrial robots. Park et al.28 constructed an edge device integrating data acquisition, processing, storage, and analysis, and implemented fault prediction for industrial robots based on LSTM. Qian et al.29 proposed an improved GRU network, which achieves excellent fault detection capability even with a limited sample size. Dai et al.30 proposed a new reliability assessment scheme based on BiGRU, Wavelet Kernel Network (WKN), and Wiener process model. Building on this foundation, recent research has begun to introduce multi-source data fusion and attention mechanisms to optimize PdM model performance. For instance, Gültekin et al.31 proposed a CNN-based data fusion method that fuses sound and vibration signals while utilizing the short-time Fourier transform to achieve accurate prediction of operational faults in automatically running vehicles. Liu et al.32 proposed a failure prediction method for chain jack hydraulic systems based on multi-source data fusion. The proposed method utilizes pressure, temperature, and flow data under different operating conditions. Huang et al.33 proposed a multi-scale convolutional neural network (CA-MCNN) with an attention mechanism. In CA-MCNN, the learned features are scored, and weights are assigned adaptively through the attention mechanism to improve the learning ability of the model. Tong et al.34 proposed a coordinated attention (CA) model suitable for one-dimensional vibration signals, and established a lightweight coordinated attention convolutional neural network ACNN to realize the fault detection of rolling bearings. In summary, recent studies have shown that deep learning models such as CNN, GRU, and LSTM exhibit significant advantages in solving fault prediction by virtue of their excellent spatial feature extraction capabilities or temporal feature extraction capabilities. However, GRU and LSTM in essence process fault time series information in a unidirectional manner, ignoring the inverse information, which will ultimately affect the accuracy of the fault prediction model. Meanwhile, based on the literature analysis above, there is more limited research on the integration of spatial and temporal feature extraction capabilities in the predictive maintenance domain. In addition, most existing fault prediction efforts for industrial robots essentially use only a single data source, such as vibration, torque, or current, and extracting multidimensional features from multi-sensor data to provide accurate and reliable predictions remains a challenge. To cope with these complex challenges, this paper proposes a PdM model for industrial robots featuring spatio-temporal feature fusion, named AM-CNN-BiGRU. This framework integrates the attention mechanism, the convolutional neural network (CNN), and the bidirectional gated recurrent unit (BiGRU). The main contributions of this study are summarized as follows: A combination of CNN and BiGRU. CNN can automatically extract local spatial features from the processed signals, and BiGRU utilizes its bi-directional time-series modeling capability to capture temporal features during the fault evolution process. The combination of CNN and BiGRU ensures accurate prediction of complex fault features in complex operating environments, while enhancing the reliability of the model.By using CNN channel stacking, the fusion of vibration, current, and torque data of robot joints is achieved, avoiding the drawbacks of a single data source. At the same time, an attention mechanism is constructed to assign different weights to different source data, effectively identifying key fault features.The Lion optimizer was introduced into the training of deep learning models in the field of industrial predictive maintenance and achieved better results compared to the commonly used Adam and SGDM optimizers. A combination of CNN and BiGRU. CNN can automatically extract local spatial features from the processed signals, and BiGRU utilizes its bi-directional time-series modeling capability to capture temporal features during the fault evolution process. The combination of CNN and BiGRU ensures accurate prediction of complex fault features in complex operating environments, while enhancing the reliability of the model. A combination of CNN and BiGRU. CNN can automatically extract local spatial features from the processed signals, and BiGRU utilizes its bi-directional time-series modeling capability to capture temporal features during the fault evolution process. The combination of CNN and BiGRU ensures accurate prediction of complex fault features in complex operating environments, while enhancing the reliability of the model. By using CNN channel stacking, the fusion of vibration, current, and torque data of robot joints is achieved, avoiding the drawbacks of a single data source. At the same time, an attention mechanism is constructed to assign different weights to different source data, effectively identifying key fault features. By using CNN channel stacking, the fusion of vibration, current, and torque data of robot joints is achieved, avoiding the drawbacks of a single data source. At the same time, an attention mechanism is constructed to assign different weights to different source data, effectively identifying key fault features. The Lion optimizer was introduced into the training of deep learning models in the field of industrial predictive maintenance and achieved better results compared to the commonly used Adam and SGDM optimizers. The Lion optimizer was introduced into the training of deep learning models in the field of industrial predictive maintenance and achieved better results compared to the commonly used Adam and SGDM optimizers. The rest of this article is organized as follows: “Proposed methodology” provides a detailed introduction to the overall framework of the predictive maintenance system for industrial robots and the proposed predictive maintenance methods. Section “Experiment” describes the details of the experimental setup and the experimental data collected to evaluate the proposed method in this paper. Section “Results and discussion” presents the experimental results and comprehensive analysis. Finally, “Conclusion” summarizes this study. Proposed methodology Framework of the PdM system In this paper, an overall PdM framework is first proposed for implementing the industrial robot predictive maintenance method, AM-CNN-BiGRU. As illustrated in Fig. 1, the framework comprises four phases. In the data collection phase, vibration signals, current signals, and torque signals are collected from each joint of the industrial robot. Fig. 1The overall framework for the implementation of AM-CNN-BiGRU. The overall framework for the implementation of AM-CNN-BiGRU. In the data preprocessing phase, statistical features are first extracted from the collected raw time series signals, followed by standardization. The resulting feature matrix is then dimensionality-reduced using principal component analysis (PCA) to alleviate the model’s computational burden and accelerate processing. In this phase, the extracted statistical features include the maximum, mean, peak-to-peak, root mean square, kurtosis, waveform factor, and margin factor in the time domain, along with the centroid frequency, mean frequency, root mean square of frequency, and frequency variance in the frequency domain. Collectively, these features effectively characterize the equipment’s fault potential. The corresponding calculation formulas are provided in Table 1. In these formulas, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)}$$\end{document} denotes the time domain signal, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{N}$$\end{document} denotes the length of the signal, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}$$\end{document} denotes the power spectrum, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{k}$$\end{document} denotes the frequency value. In the model training phase, the preprocessed data is randomly divided into three independent data sets according to a predefined ratio, i.e., the training set, validation set, and test set. Samples from the training and validation sets are used to train, evaluate, and iteratively optimize the established AM-CNN-BiGRU model offline. Throughout training, the loss function is computed and model weights are updated via backpropagation. The test set is used to test the performance of the trained AM-CNN-BiGRU model. In the fault prediction phase, the trained AM-CNN-BiGRU model is deployed. Incoming data undergoes processing and statistical feature extraction using the identical methodology established in the data collection and preprocessing phases. These extracted features are then input to the model to generate fault predictions, upon which appropriate maintenance decisions are formulated. Table 1Formulas for calculating statistical features.Time domain statistical featuresFormulaFrequency domain statistical featuresFormulaMaximum \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{1}} = \mathrm{max}\mathrm{\:(}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)}\mathrm{)}$$\end{document} Centroid frequency \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{8}} = \frac{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}{\mathrm{f}}_{\mathrm{k}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}$$\end{document} Mean \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{2}} = \frac{\mathrm{1}}{\mathrm{N}}\sum\:_{\mathrm{i} = \mathrm{1}}^{\mathrm{N}}{\hspace{0.05em}}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)}$$\end{document} Mean frequency \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{9}} = \frac{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}{\mathrm{K}}$$\end{document} Peak-to-peak \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{3}} = {\mathrm{F}}_{\mathrm{1}} - \mathrm{min}\text{}\mathrm{(}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)}\mathrm{)}$$\end{document} Root mean square of frequency \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{10}} = \sqrt{\frac{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}{{\mathrm{f}}_{\mathrm{k}}}^{\mathrm{2}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}}$$\end{document} Root mean square \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{4}} = \sqrt{\frac{\mathrm{1}}{\mathrm{N}}{\sum\:}_{\mathrm{i}=\mathrm{1}}^{\mathrm{N}}{\mathrm{(}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)}\mathrm{)}}^{\mathrm{2}}}$$\end{document} Frequency variance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{11}} = \frac{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}\mathrm{(}{\mathrm{f}}_{\mathrm{k}} - {\mathrm{F}}_{\mathrm{8}}{\mathrm{)}}^{\mathrm{2}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}$$\end{document} Kurtosis \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{5}} = \frac{\sum\:_{\mathrm{i}=\mathrm{1}}^{\mathrm{N}}{\hspace{0.05em}}\mathrm{(}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)-}{\mathrm{F}}_{\mathrm{2}}{\mathrm{)}}^{\mathrm{4}}}{\mathrm{(}\mathrm{N}\mathrm{-1)}{\sqrt{\frac{\sum\:_{\mathrm{i}=\mathrm{1}}^{\mathrm{N}}{\hspace{0.05em}}{\mathrm{(}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)-}{\mathrm{F}}_{\mathrm{2}}\mathrm{)}}^{\mathrm{2}}}{\mathrm{N}-1}}}^{\mathrm{4}}}$$\end{document} Waveform factor \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{6}} = \frac{{\mathrm{F}}_{\mathrm{4}}}{\frac{\mathrm{1}}{\mathrm{N}}\sum\:_{\mathrm{i}=\mathrm{1}}^{\mathrm{N}}{\hspace{0.05em}}\mathrm{|}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)|}}$$\end{document} Margin factor \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{7}} = \frac{\mathrm{max(|}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)|)}}{\mathrm{(}\frac{\mathrm{1}}{\mathrm{N}}\sum\:_{\mathrm{i}=\mathrm{1}}^{\mathrm{N}}{\hspace{0.05em}}\sqrt{\mathrm{|}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)|}}{\mathrm{)}}^{\mathrm{2}}}$$\end{document} Table 1Formulas for calculating statistical features.Time domain statistical featuresFormulaFrequency domain statistical featuresFormulaMaximum \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{1}} = \mathrm{max}\mathrm{\:(}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)}\mathrm{)}$$\end{document} Centroid frequency \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{8}} = \frac{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}{\mathrm{f}}_{\mathrm{k}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}$$\end{document} Mean \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{2}} = \frac{\mathrm{1}}{\mathrm{N}}\sum\:_{\mathrm{i} = \mathrm{1}}^{\mathrm{N}}{\hspace{0.05em}}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)}$$\end{document} Mean frequency \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{9}} = \frac{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}{\mathrm{K}}$$\end{document} Peak-to-peak \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{3}} = {\mathrm{F}}_{\mathrm{1}} - \mathrm{min}\text{}\mathrm{(}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)}\mathrm{)}$$\end{document} Root mean square of frequency \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{10}} = \sqrt{\frac{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}{{\mathrm{f}}_{\mathrm{k}}}^{\mathrm{2}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}}$$\end{document} Root mean square \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{4}} = \sqrt{\frac{\mathrm{1}}{\mathrm{N}}{\sum\:}_{\mathrm{i}=\mathrm{1}}^{\mathrm{N}}{\mathrm{(}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)}\mathrm{)}}^{\mathrm{2}}}$$\end{document} Frequency variance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{11}} = \frac{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}\mathrm{(}{\mathrm{f}}_{\mathrm{k}} - {\mathrm{F}}_{\mathrm{8}}{\mathrm{)}}^{\mathrm{2}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}{\sum\:_{\mathrm{k}=\mathrm{1}}^{\mathrm{K}}{\hspace{0.05em}}\mathrm{s}\mathrm{(}\mathrm{k}\mathrm{)}}$$\end{document} Kurtosis \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{5}} = \frac{\sum\:_{\mathrm{i}=\mathrm{1}}^{\mathrm{N}}{\hspace{0.05em}}\mathrm{(}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)-}{\mathrm{F}}_{\mathrm{2}}{\mathrm{)}}^{\mathrm{4}}}{\mathrm{(}\mathrm{N}\mathrm{-1)}{\sqrt{\frac{\sum\:_{\mathrm{i}=\mathrm{1}}^{\mathrm{N}}{\hspace{0.05em}}{\mathrm{(}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)-}{\mathrm{F}}_{\mathrm{2}}\mathrm{)}}^{\mathrm{2}}}{\mathrm{N}-1}}}^{\mathrm{4}}}$$\end{document} Waveform factor \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{6}} = \frac{{\mathrm{F}}_{\mathrm{4}}}{\frac{\mathrm{1}}{\mathrm{N}}\sum\:_{\mathrm{i}=\mathrm{1}}^{\mathrm{N}}{\hspace{0.05em}}\mathrm{|}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)|}}$$\end{document} Margin factor \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{F}}_{\mathrm{7}} = \frac{\mathrm{max(|}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)|)}}{\mathrm{(}\frac{\mathrm{1}}{\mathrm{N}}\sum\:_{\mathrm{i}=\mathrm{1}}^{\mathrm{N}}{\hspace{0.05em}}\sqrt{\mathrm{|}\mathrm{X}\mathrm{(}\mathrm{i}\mathrm{)|}}{\mathrm{)}}^{\mathrm{2}}}$$\end{document} Formulas for calculating statistical features. Convolutional neural network CNN has maintained its dominant position in the field of image processing. In recent years, it has also been applied to areas such as signal analysis and equipment failure prediction35. CNN performs convolution operations by applying convolution kernels in a sliding-window manner across the input data. Through multi-layer stacking, the shallow convolutional layers capture short-term local features of the data, such as transient impulses or abnormal fluctuations. The deeper convolutional layers then integrate these local features into more complex and global high-level representations. These high-level features exhibit stronger correlations with fault conditions. CNN typically comprises an input layer, convolution layers, pooling layers, a flattening layer, and a fully-connected output layer. Figure 2 illustrates the CNN architecture. Fig. 2The structure of CNN. The structure of CNN. The convolution layer employs multiple convolutional kernels, whose quantity is either manually specified or optimized based on data characteristics and model architecture. These kernels perform convolution operations on local segments of sequential data, enabling simultaneous feature extraction and dimensionality reduction. The mathematical formula is as follows:1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{array}{*{20}{c}} {X_{j}^{l}=b_{j}^{{l - 1}}+\mathop \sum \limits_{{i \in {M_j}}} X_{i}^{{l - 1}}*w_{{ij}}^{{l - 1}}} \end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{X}}_{\mathrm{j}}^{\mathrm{l}}$$\end{document} is the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{j}$$\end{document}th feature output of layer \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{l}$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{M}}_{\mathrm{j}}$$\end{document} is the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{j}$$\end{document}th convolutional region of layer \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{l}$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{w}}_{\mathrm{ij}}^{\mathrm{l}-1}$$\end{document} is the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{j}$$\end{document}th weight value of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{i}$$\end{document}th convolutional kernel of layer \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{l}$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{b}}_{\mathrm{j}}^{\mathrm{l}-1}$$\end{document} is the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{j}$$\end{document}th bias of layer \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{l}-1$$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ast$$\end{document} represents the convolutional operation. Following the convolution layer, a pooling layer is typically added. This layer contains multiple feature maps that correspond one-to-one with those output from the preceding convolutional layer. Using a manually defined window size, the pooling operation downsamples local regions without altering the number of feature maps, while reducing their spatial dimensions according to the window size. Common pooling methods include max pooling and average pooling; this study employs max pooling. The mathematical formula is as follows:2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{P}}_{\mathrm{i}}^{\mathrm{l}+\mathrm{1}}{(}\mathrm{j}\mathrm{)} = \underset{{(}\mathrm{j}\mathrm{-1)}\mathrm{w}\le \mathrm{t}\le \mathrm{jw}}{\mathrm{max}}{{\mathrm{q}}_{\mathrm{i}}^{\mathrm{l}}\mathrm{(}\mathrm{t}\mathrm{)}}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{q}}_{\mathrm{i}}^{\mathrm{l}}\mathrm{(}\mathrm{t}\mathrm{)}$$\end{document} is the value of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{t}$$\end{document}th neuron in the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{i}$$\end{document}th feature of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:l$$\end{document}th layer, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{w}$$\end{document} is the pooling size of the pooled region, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{P}}_{\mathrm{i}}^{\mathrm{l}+\mathrm{1}}\mathrm{(}\mathrm{j}\mathrm{)}$$\end{document} is the value of the neuron in the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{l}+\mathrm{1}$$\end{document}th layer after pooling. Before the fully connected output layer, the features obtained from the previous convolutional and pooling layers need to be flattened using the flatting layer. The fully connected output layer is used for feature re-extraction and summarization. The operations performed in the fully connected output layer are shown below:3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{y}}^{\mathrm{l}} = {\mathrm{f}\mathrm{(}\mathrm{w}}^{\mathrm{l}}\cdot {\mathrm{x}}^{\mathrm{l}-1}+\text{}{\mathrm{b}}^{\mathrm{l}}\mathrm{)}\end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{x}}^{\mathrm{l}-1}$$\end{document} is the output of the flatting layer; \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{w}}^{\mathrm{l}}$$\end{document} is the weight factor of the fully connected layer; \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{b}}^{\mathrm{l}}$$\end{document} is the bias of the fully connected layer. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{y}}^{\mathrm{l}}$$\end{document} is the output of the fully connected layer. The monitoring data of industrial robots is intrinsically high-dimensional multivariate time series. The occurrence and progression of faults are often initially manifested as subtle changes in local spatial patterns. The exceptional capability of CNN in extracting local features enables them to acutely capture these early and weak fault signatures, which poses a significant challenge for many traditional methods to achieve. Bidirectional gated recurrent unit Gated Recurrent Unit (GRU) is a variant structure of the Recurrent Neural Network (RNN) model, which can effectively solve the defects of gradient explosion as well as gradient vanishing that the RNN is prone to when dealing with long sequences. GRU integrates the inputs from the “forget gate” and the LSTM neural network into the “update gate”. Compared with LSTM, the GRU network has one less gating unit, so the model parameter reduction is much more efficient36. The internal calculation process of GRU is as follows:4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{Z}}_{\mathrm{t}} = {\sigma \mathrm{(}\mathrm{W}}_{\mathrm{z}}\mathrm{[}{\mathrm{h}}_{\mathrm{t}-1}\mathrm{,}{\mathrm{x}}_{\mathrm{t}}\mathrm{]}\mathrm{)}\end{array}$$\end{document}5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{R}}_{\mathrm{t}} = {\sigma \mathrm{(}\mathrm{W}}_{\mathrm{r}}\mathrm{[}{\mathrm{h}}_{\mathrm{t}-1}\mathrm{,}{\mathrm{x}}_{\mathrm{t}}\mathrm{]}\mathrm{)}\end{array}$$\end{document}6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}\stackrel{\sim}{{\mathrm{H}}_{\mathrm{t}}} = {\mathrm{tanh}\mathrm{(}\mathrm{W}\mathrm{[}\mathrm{r}}_{\mathrm{t}}{\mathrm{h}}_{\mathrm{t}-1}\mathrm{,}{\mathrm{x}}_{\mathrm{t}}\mathrm{]}\mathrm{)}\end{array}$$\end{document}7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{H}}_{\mathrm{t}} = {\mathrm{(1}-\mathrm{Z}}_{\mathrm{t}}\mathrm{)}{\mathrm{h}}_{\mathrm{t}-1}+\text{}{\mathrm{Z}}_{\mathrm{t}}{\stackrel{{\sim}}{\mathrm{H}}}_{\mathrm{t}}\end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{W}}_{\mathrm{z}}$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{W}}_{\mathrm{r}}$$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{W}$$\end{document} are the weight matrices of the reset gate, update gate, and hidden layer network states, respectively. The Bi-GRU architecture comprises two GRUs operating in opposite directions (forward and backward), which scan the sequence from past to future and from future to past, respectively. The final output integrates information from both directions, ensuring that the state representation at each time point encompasses complete contextual information, thereby enabling enhanced time series prediction. The structure of Bi-GRU is shown in Fig. 3. Fig. 3The structure of BiGRU. The structure of BiGRU. The calculation process of BiGRU is as follows:8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}\overrightarrow{{\mathrm{H}}_{\mathrm{t}}} = {\mathrm{GRU(x}}_{\mathrm{t}}\mathrm{,}\overrightarrow{{\mathrm{H}}_{\mathrm{t+1}}}\mathrm{)}\end{array}$$\end{document}9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}\overleftarrow{{\mathrm{H}}_{\mathrm{t}}} = {\mathrm{GRU(x}}_{\mathrm{t}}\mathrm{,}\overleftarrow{{\mathrm{H}}_{\mathrm{t-1}}}\mathrm{)}\end{array}$$\end{document}10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{H}}_{\mathrm{t}} = {\mathrm{W}}_{\mathrm{t}}\overrightarrow{{\mathrm{H}}_{\mathrm{t}}}+\text{}{\mathrm{V}}_{\mathrm{t}}\overleftarrow{{\mathrm{H}}_{\mathrm{t}}}+\text{}{\mathrm{b}}_{\mathrm{t}}\end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overrightarrow{{\mathrm{H}}_{\mathrm{t}}}$$\end{document} is the forward hidden layer state, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\overleftarrow{{\mathrm{H}}_{\mathrm{t}}}$$\end{document} is the reverse hidden layer state, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{W}}_{\mathrm{t}}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{V}}_{\mathrm{t}}$$\end{document} are the corresponding weights, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{b}}_{\mathrm{t}}$$\end{document} is the bias of the hidden layer bias at time t. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{H}}_{\mathrm{t}}$$\end{document} is the output of BiGRU. The operational state of an industrial robot represents a typical temporal evolution process. Fault occurrence is often gradual and progressive, with its precursors being temporally correlated. The bidirectional gating mechanism of BiGRU enables it to simultaneously learn temporal dependencies from both directions. For instance, a subtle anomaly signal might only lead to significant performance degradation after an extended period. While the forward GRU may struggle to retain such long-term dependencies, the backward GRU can retrospectively trace the sequence, thereby more readily identifying early weak anomaly signals. This capability to capture bidirectional temporal dependencies is crucial for accurately determining the progression trend of faults, which remains unachievable by unidirectional network architectures. Attention mechanism Attention mechanism is a method proposed according to the human visual system and cognitive mechanism, which can improve the expressiveness of the deep learning model and increase the sensitivity of the model to key features37. Using the attention mechanism in the input data of CNN can link the different channels of the input data to calculate the degree of importance of each channel, assign weights to different channel features, and improve the network’s expressive ability in feature representation, which in turn improves the performance of the model38. The computational procedure of the attention mechanism constructed in this paper is as follows: The input sample is denoted as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{S} = \mathrm{[}{\mathrm{S}}_{\mathrm{1}}\mathrm{,\:}{\mathrm{S}}_{\mathrm{2}}\mathrm{,\:}\cdots \mathrm{,\:}{\mathrm{S}}_{\mathrm{i}}\mathrm{,\:}\cdots \mathrm{,\:}{\mathrm{S}}_{\mathrm{h}}\mathrm{]}$$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{h}$$\end{document} is the number of input sample channels. 1) Global average pooling of feature data for each channel:11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{m}}_{\mathrm{i}} = {\mathrm{Avgpool(}\mathrm{S}}_{\mathrm{i}}\mathrm{)} = \frac{\mathrm{1}}{\mathrm{l}}\sum\:_{\mathrm{j}=\mathrm{1}}^{\mathrm{l}}{\hspace{0.05em}}{\mathrm{S}}_{\mathrm{i}}^{\mathrm{j}}\end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{S}}_{\mathrm{i}}$$\end{document} denotes the feature sequence of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{i}$$\end{document}th channel, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{l}$$\end{document} denotes the length of the feature sequence. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{S}}_{\mathrm{i}}^{\mathrm{j}}$$\end{document} denotes the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{j}$$\end{document}th data of the feature sequence of the ith channel. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{m}}_{\mathrm{i}}$$\end{document} is the result of the global average pooling of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{i}$$\end{document}th channel. This step aims to compress the global information of each channel and preliminarily estimate the overall activation level of that channel. 2) Score the feature of the ith channel:12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}\mathrm{F} = \sigma \mathrm{(}\mathrm{W}\cdot \mathrm{M}+\text{}\mathrm{b}\mathrm{)}\end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma$$\end{document} is the scoring function, often using the sigmoid function or ReLU function. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{M}={{\mathrm{M}}_{\mathrm{1}},{\mathrm{M}}_{\mathrm{2}},\ldots ,{\mathrm{M}}_{\mathrm{i}},\ldots ,{\mathrm{M}}_{\mathrm{h}}}$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{F}={{\mathrm{f}}_{\mathrm{1}},{\mathrm{f}}_{\mathrm{2}},\cdots , {\mathrm{f}}_{\mathrm{i}}, \cdots ,{\mathrm{f}}_{\mathrm{h}}}$$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{f}}_{\mathrm{i}}$$\end{document} is the score of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{i}$$\end{document}th channel feature. This directly corresponds to the practical problem of uneven contribution from different data sources. 3) Normalize the scores:13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\alpha }_{\mathrm{i}} = {\mathrm{softmax}\mathrm{(}\mathrm{f}}_{\mathrm{i}}\mathrm{)} = \frac{\mathrm{exp}\mathrm{(}{\mathrm{f}}_{\mathrm{i}}\mathrm{)}}{{\sum\:}_{\mathrm{i}}\mathrm{f}}\end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{{\alpha}}_{\mathrm{i}}$$\end{document} is the weight assigned to the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:i$$\end{document}th channel data \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{S}}_{\mathrm{i}}$$\end{document} by the attention mechanism. This enables the quantification of the contribution ratio of each channel in the model. 4) Obtain the final output \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{M}$$\end{document} of the attention mechanism:14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}\mathrm{M}={ \mathrm{S}\otimes \mathrm{A}=\text{{}\mathrm{s}}_{\mathrm{1}}{\alpha }_{\mathrm{1}},{\mathrm{s}}_{\mathrm{2}}{\alpha }_{\mathrm{2}},\ldots ,{\mathrm{s}}_{\mathrm{h}}{\alpha }_{\mathrm{h}}\text{}}\end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{A}={{\alpha }_{\mathrm{1}},{\alpha }_{\mathrm{2}},\cdots ,{\alpha }_{\mathrm{i}},\ldots ,{\alpha }_{\mathrm{h}}}$$\end{document}. ⊗ is the multiplication of the corresponding elements of two sets. Failures of industrial robots are usually caused by multi-source factors, and different sensor data have significantly different sensitivity to different fault factors. Therefore, the prediction model is required to have adaptive focusing ability and be able to dynamically evaluate the importance of different data sources. The introduction of attention mechanism can effectively achieve this goal. Proposed AM-CNN-BiGRU structure This section details the proposed methodology. Figure 4 shows the model structure of the proposed AM-CNN-BiGRU. The proposed method uses an architecture consisting of a deep stack of CNNs and BiGRUs along with an attention mechanism for feature extraction and time series prediction. CNN has significant advantages over traditional neural networks in terms of sensing the field-of-view and weight sharing between hidden layers. For the problem of fault prediction in industrial robots, where the time series is significantly nonlinear and stochastic, CNN can reduce the network complexity and the number of weights through the mechanism of weight sharing between layers and determine the local input space pattern that produces the strongest response by computing the same convolutional kernel. Before the CNN input data, it first assigns different weights to different channel feature data through the attention mechanism to determine the importance of different source data. BiGRU can further process the spatial features extracted by the CNN and continue to extract the deep temporal features to improve the accuracy of the model in predicting the time series. Fig. 4The structure of AM-CNN-BiGRU. The structure of AM-CNN-BiGRU. 1) AM layer. Before feature extraction of input data using CNN, the weights of different channel data are determined through the AM layer. The structure of the AM layer is shown in the Attention layer in Fig. 4. (1) The global average pooling operation is first performed on the three-channel data \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{X}=\mathrm{[}{\mathrm{X}}_{\mathrm{1}}\mathrm{,\:}{\mathrm{X}}_{\mathrm{2}}\mathrm{,\:}{\mathrm{X}}_{\mathrm{3}}\mathrm{]}$$\end{document} in the Attention layer. The formula is as follows:15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{X}}_{\mathrm{Avg,i}} = \mathrm{A}\mathrm{v}\mathrm{g}\mathrm{p}\mathrm{o}\mathrm{o}\mathrm{l}\mathrm{(}{\mathrm{X}}_{\mathrm{i}}\mathrm{)}\text{} = \text{}\frac{\mathrm{1}}{\mathrm{L}}{\sum\:}_{\mathrm{j}=\mathrm{1}}^{\mathrm{L}}{\mathrm{X}}_{\mathrm{i,j}}\mathrm{,}\text{}\mathrm{i} = \mathrm{1}\mathrm{,}\text{}\mathrm{2}\mathrm{,}\text{}\mathrm{3}\end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{X}}_{\mathrm{i,j}}$$\end{document} is the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{j}$$\end{document}th data of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{i}$$\end{document}th channel of data \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{X}$$\end{document}; \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{X}}_{\mathrm{Avg,i}}$$\end{document} is the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{i}$$\end{document}th data of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{X}}_{\mathrm{Avg}}$$\end{document}. (2) Score each channel. The formula is as follows:16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}\mathrm{F} = \sigma \mathrm{(}\mathrm{W}\cdot {\mathrm{X}}_{\mathrm{Avg}}+\text{}\mathrm{b}\mathrm{)}\end{array}$$\end{document}where, the scoring function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\sigma$$\end{document} uses the Sigmoid function; \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{F}={{\mathrm{f}}_{\mathrm{1}},{\mathrm{f}}_{\mathrm{2}},{\mathrm{f}}_{\mathrm{3}}}$$\end{document}. (3) Normalize the score obtained for the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:i$$\end{document}th channel. The formula is as follows:17\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\alpha }_{\mathrm{i}} = \mathrm{softmax}\mathrm{(}{\mathrm{f}}_{\mathrm{i}}\mathrm{)} = \frac{\mathrm{exp}\mathrm{(}{\mathrm{f}}_{\mathrm{i}}\mathrm{)}}{{\sum\:}_{\mathrm{i}}\mathrm{f}}\mathrm{,}\text{}\mathrm{i} = \mathrm{1}\mathrm{,}\text{}\mathrm{2}\mathrm{,}\text{}\mathrm{3}\end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\alpha }_{\mathrm{i}}$$\end{document} is the weight assigned to the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{i}$$\end{document}th channel. (4) The final output of the AM layer is obtained by multiplying each of the obtained channel weights by the corresponding channel data of the initial signal \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:X$$\end{document}, respectively. The formula is as follows:18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}\mathrm{O} = \mathrm{X}\otimes \mathrm{A} = {{\mathrm{X}}_{\mathrm{1}}{\alpha }_{\mathrm{1}},{\mathrm{X}}_{\mathrm{2}}{\alpha }_{\mathrm{2}},{\mathrm{X}}_{\mathrm{3}}{\alpha }_{\mathrm{3}}}\end{array}$$\end{document}where, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{A}={{\alpha }_{\mathrm{1}},{\alpha }_{\mathrm{2}},{\alpha }_{\mathrm{3}}}$$\end{document}. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{O}$$\end{document} is the feature data after the Attention layer assigns channel weights. 2) CNN layer. After the AM layer assigns weights to different input data, CNN is utilized to extract deep features. As shown in the CNN layer in Fig. 4, the CNN layer is mainly composed of two convolutional modules. Each convolutional module is, in turn, made up of a convolutional layer and a pooling layer stacked on top of each other. Among them, the convolutional layer uses the ReLU activation function, and the pooling layer chooses the maximum pooling. The settings of the hyperparameters of the CNN layer in the framework of this method are specifically shown in Table 2. Table 2The hyperparameters of the CNN layer.LayerParametersConvolution layerFilters = 8; Kernel size = 4; Padding=“same”; Stride = 1; Activation function = ReLUMax pooling layerkernel size = 2Convolution layerFilters = 16; Kernel size = 4; Padding=“same”; Stride = 1; Activation function = ReLUMax pooling layerkernel size = 2 Table 2The hyperparameters of the CNN layer.LayerParametersConvolution layerFilters = 8; Kernel size = 4; Padding=“same”; Stride = 1; Activation function = ReLUMax pooling layerkernel size = 2Convolution layerFilters = 16; Kernel size = 4; Padding=“same”; Stride = 1; Activation function = ReLUMax pooling layerkernel size = 2 The hyperparameters of the CNN layer. 3) BiGRU layer. After feature extraction using the CNN layer, the extracted feature vector is passed as input to the BiGRU layer to continue extracting temporal features. The hidden layer in the BiGRU network structure is divided into forward and backward parts, which are used to read the sequence information of the current time series and the reverse data information of the same time series. The state of the hidden layer is obtained by weighting the states of the forward and reverse hidden layers. The calculation process is shown below:19\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{H}}_{\mathrm{t}}^{\mathrm{f}} = \mathrm{G}\mathrm{R}\mathrm{U}\mathrm{(}{\mathrm{x}}_{\mathrm{t}}\mathrm{,}\text{}{\mathrm{H}}_{\mathrm{t}+\mathrm{1}}^{\mathrm{f}}\mathrm{)}\end{array}$$\end{document}20\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{H}}_{\mathrm{t}}^{\mathrm{b}} = \mathrm{G}\mathrm{R}\mathrm{U}\mathrm{(}{\mathrm{x}}_{\mathrm{t}}\mathrm{,}\text{}{\mathrm{H}}_{\mathrm{t}-1}^{\mathrm{b}}\mathrm{)}\end{array}$$\end{document}21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{H}}_{\mathrm{t}} = \left[{\mathrm{H}}_{\mathrm{t}}^{\mathrm{f}}\mathrm{,\:}{\mathrm{H}}_{\mathrm{t}}^{\mathrm{b}}\right]\end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{x}}_{\mathrm{t}}$$\end{document} is the data at moment \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{t}$$\end{document} in the BiGRU input, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{H}}_{\mathrm{t}}^{\mathrm{f}}$$\end{document} is the forward hidden state, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{H}}_{\mathrm{t}}^{\mathrm{b}}$$\end{document} is the reverse hidden state, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{H}}_{\mathrm{t}}$$\end{document} is the fused features output from the BiGRU layer. After extracting the features using CNN and BiGRU, the faults of industrial robots are predicted by a fully connected layer. The settings of the hyperparameters of the BiGRU layer and the fully connected layer in the structure of the proposed method are specified in Table 3. Table 3The hyperparameters of the BiGRU layer and the FC layer.LayerParametersBiGRU LayerHidden layer 1Number of hidden neurons = 32Hidden layer 2Number of hidden neurons = 64FC LayerFully connected layerActivation function = Sigmoid Table 3The hyperparameters of the BiGRU layer and the FC layer.LayerParametersBiGRU LayerHidden layer 1Number of hidden neurons = 32Hidden layer 2Number of hidden neurons = 64FC LayerFully connected layerActivation function = Sigmoid The hyperparameters of the BiGRU layer and the FC layer. Optimizer Optimizers are algorithms that govern parameter updates during deep learning model training. During backpropagation, optimizers direct each parameter’s adjustment magnitude and direction, progressively minimizing the objective function toward a global minimum. Optimizer selection critically impacts both training efficiency and final model performance. Google Research (2023) proposed a method for discovering optimized algorithms for deep neural network training through program search. After thousands of TPU hours of arithmetic search, combined with human intervention, they obtained the Lion optimizer39. Use the Lion optimizer to update the parameter expression as follows:22\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{c}}_{\mathrm{t}+\mathrm{1}} = \beta _{\mathrm{1}}{\mathrm{m}}_{\mathrm{t}}+\text{}\mathrm{(}\mathrm{1} - \beta _{\mathrm{1}}\mathrm{)}{\mathrm{g}}_{\mathrm{t}+\mathrm{1}}\end{array}$$\end{document}23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{W}}_{\mathrm{t}+\mathrm{1}} = {\mathrm{W}}_{\mathrm{t}} - \alpha \mathrm{(}\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\mathrm{(}{\mathrm{c}}_{\mathrm{t}+\mathrm{1}}\mathrm{)}+\text{}\lambda{\mathrm{W}}_{\mathrm{t}}\mathrm{)}\end{array}$$\end{document}24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}{\mathrm{m}}_{\mathrm{t}+\mathrm{1}} = \beta _{\mathrm{2}}{\mathrm{m}}_{\mathrm{t}}+\text{}\mathrm{(}\mathrm{1} - \beta _{\mathrm{2}}\mathrm{)}{\mathrm{g}}_{\mathrm{t}+\mathrm{1}}\end{array}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{W}}_{\mathrm{t}}$$\end{document} is the model parameter matrix for the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{t}$$\end{document}th iteration, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\alpha$$\end{document} is the learning rate, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{g}}_{\mathrm{t}}$$\end{document} is the gradient of the loss function, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{sign}$$\end{document} is the sign function, i.e., 1 for positive numbers and − 1 for negative numbers. In contrast to Adam and various other adaptive optimizers that require the preservation of both first-order moments and second-order moments, the Lion optimizer requires only momentum and utilizes symbolic manipulation to compute the parameter updates, and the same magnitude is used for each parameter update. Its performance gain increases as the size of the training batch increases. It also requires a smaller learning rate due to the larger paradigm of updates produced by the sign function. Another key feature of the Lion optimizer is that it performs momentum updates after variable parameter updates, while eliminating the need to compute second-order moments. This significantly reduces memory consumption and enhances computational efficiency when training large deep learning models39. The sign function can be regarded as a noise injection or regularization mechanism. Sensor time series data from industrial robots typically contain noise, abrupt changes, and complex nonlinear patterns. This parameter update approach in the Lion optimizer prevents predictive maintenance models from overfitting to noise in training data while theoretically aiding convergence to flatter minima regions in the loss function, thereby enhancing the robustness of predictive maintenance models. The Lion optimizer has demonstrated superior performance and effectiveness compared to existing classical optimizers across various model training tasks, including language models, image classification models, diffusion models, and image-text matching models. Based on the various advantages of the Lion Optimizer for training predictive maintenance models, this paper pioneers the novel application of Lion to train industrial robot failure prediction models. To the best of our knowledge, this represents the first work to empirically demonstrate the effectiveness of the Lion optimizer for temporal data prediction tasks within the domain of predictive maintenance. Experiment To validate the effectiveness and generalization capability of the proposed AM-CNN-BiGRU model, our research team conducted two sets of experiments. One involved constructing an industrial robot test bench for data collection, building a dataset, and performing experiments. The other utilized the NASA turbofan engine dataset, the most commonly used dataset in the field of predictive maintenance. Industrial robot dataset 1) Experimental platform construction. We took the axis end accuracy faults of a six-axis industrial robot as the research object, and established an experimental platform in a high-end laboratory of an automobile factory. As shown in Fig. 5, the experimental bench consists of a six-axis tandem industrial welding robot (KR270 R2700 ultra) equipped with a C-type spot welder, a robot control cabinet (KR C4), a laptop computer (MECHREVO), a data collector (IN-SDG), six vibration acceleration sensors (310 A-20), and a laser accuracy tracker (Leica AT960-MR). Fig. 5Robot experiment platform. Robot experiment platform. 2) Data collection and preprocessing. The vibration signal, current signal, and torque signal of each joint of the robot and the repeated positioning accuracy data of the end of the robot shaft are collected during the working process. The vibration signal is collected by the vibration sensor, and the current signal and torque signal are obtained by the robot controller. The sampling frequency is set to 1024 Hz, and each data segment collected contains 8200 data points. The vibration data (unit: m/s2) of the six joints collected are shown in Fig. 6. In order to collect data, we conducted a two-year tracking experiment. Simultaneously, in order to augment the training sample size, the sliding window technique is applied to segment the time-series data along the temporal dimension38. The window length is set to 1024, and the step size is set to 200. 36 samples are obtained for each data segment, and each sample contains 1024 data points. Figure 7 shows the final obtained vibration data (unit: m/s2), moment data (unit: N·m), and current data (unit: A) for the third joint of the robot. Repeat positioning accuracy data of the robot’s axis ends in each operation cycle were measured by a laser accuracy tracker. The final accuracy data obtained mainly includes the repeat positioning accuracy errors in the X, Y, and Z directions, as well as the total spatial repeat positioning accuracy error. Five sets of repetitive positioning accuracy data (unit: mm) collected in the experiment are listed in Table 4. Table 4Repetitive positioning accuracy data.IDX-directionY-directionZ-directionDifference in X-directionDifference in Y-directionDifference in Z-directionTotal difference1208.7792947.657368.96500002208.7992947.645368.9680.02-0.0120.0030.02353208.7822947.676369.000-0.0170.0310.0320.04774208.8232947.667368.9750.041-0.009-0.0250.04895208.8382947.690369.0030.0150.0230.0280.0392 Table 4Repetitive positioning accuracy data.IDX-directionY-directionZ-directionDifference in X-directionDifference in Y-directionDifference in Z-directionTotal difference1208.7792947.657368.96500002208.7992947.645368.9680.02-0.0120.0030.02353208.7822947.676369.000-0.0170.0310.0320.04774208.8232947.667368.9750.041-0.009-0.0250.04895208.8382947.690369.0030.0150.0230.0280.0392 Repetitive positioning accuracy data. Fig. 6Vibration data of each joint. Vibration data of each joint. Fig. 7Vibration data, torque data, and current data of the third joint. Vibration data, torque data, and current data of the third joint. The training of a fault prediction model requires the simultaneous utilization of multiple operating cycles of continuous time-stamped data as input to the model. If the samples of original vibration signals, current signals, and torque signals of six joints from multiple operating cycles are directly inputted into the model for training at the same time, the dimension of the input data will be too large, which not only overly occupies the computer memory, but also affects the training of the model, and is prone to the problem of underfitting or overfitting. Therefore, the statistical features of the samples originally containing 1024 data points are extracted as inputs to the model. As mentioned in the previous section, the extracted statistical features mainly include a total of 11 features in the time and frequency domains. Afterwards, the same statistical feature values corresponding to each joint of the robot are normalized, and their average values are calculated, and a set of 11-dimensional feature matrices is obtained for each signal. Finally, principal component analysis (PCA) is used to downscale the obtained feature matrix. Figure 8 shows the contribution rate of principal components during the dimensionality reduction process of feature matrices corresponding to vibration signals, current signals, and torque signals using PCA. The top 6 principal components of each type of signal whose sum of information contribution is greater than 80% are selected as the input features of the AM-CNN-BiGRU model. As can be seen from the figure, the sum of the first 5 principal components of the current signal has reached 82%, but the first 6 principal components of the current signal are still chosen as input features in order to make the data dimension consistent for each type of signal. We conducted a preliminary experiment and compared the results between using the first six and the first five principal components (for the five-component case, the sixth dimension was padded with a zero). The results were essentially consistent, indicating that using six components does not introduce redundancy and is feasible. After the statistical feature dimensionality reduction is completed, the input feature data of the model, as well as the repetitive localization accuracy data, need to be packaged and processed in timestamp order. In this paper, the six features values of vibration data, torque data, and current data \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{{\mathrm{X}}_{\mathrm{t}\mathrm{-29}},{\mathrm{X}}_{\mathrm{t}-\mathrm{28}},\ldots ,{\mathrm{X}}_{\mathrm{t}}}$$\end{document} for 30 cycles before moment \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{t}$$\end{document} and the repetitive localization accuracy labels \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{{\mathrm{Y}}_{\mathrm{t}+\mathrm{1}},{\mathrm{Y}}_{\mathrm{t}+\mathrm{2}},\ldots ,{\mathrm{Y}}_{\mathrm{t}+\mathrm{30}}}$$\end{document} for 10 cycles after moment \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{t}$$\end{document} are packaged and processed, i.e., the robot operation data of the previous 30 consecutive operation cycles are utilized for the prediction of the repetitive localization accuracy of the following 10 consecutive operation cycles. The size of this window is designed to match the architecture of the AM-CNN-BiGRU model, allowing it to fully leverage the advantages of each hierarchical component. Predicting the subsequent 10 operation cycles not only avoids the error accumulation associated with excessively long prediction horizons but also provides sufficient warning time. The final dimension of each input sample obtained is (3, 30, 6). Finally, the obtained data is divided into three parts: 70% of the data as a training set, 20% as a validation set, and 10% as a test set. Fig. 8PCA principal component contribution rate for each signal of the robot. PCA principal component contribution rate for each signal of the robot. NASA turbofan engine dataset 1) Dataset description. The NASA Turbofan Engine Dataset is generated by the C-MAPSS (Commercial Modular Aero-Propulsion System Simulation) software to simulate the degradation process of turbofan engines40. This dataset captures the degradation progression of turbofan engines, recording sensor data throughout the entire process, and is widely used for the development and validation of algorithms in the field of predictive maintenance. Twenty-one sensors are deployed at various locations to monitor the engine’s condition. The dataset comprises four subsets, denoted as FD001, FD002, FD003, and FD004. Each subset differs in operational conditions and failure modes. Each subset consists of three parts: a training set, a test set, and RUL labels. The training set contains complete run-to-failure sensor data for each engine, from healthy operation to failure, while the test set contains engines with unknown health states, providing only a portion of the full lifecycle data. In this paper, the FD001 subset, with fixed operational modes and failure modes, is used to validate the fault prediction algorithm. The dataset includes a total of 21 sensor data types, encompassing temperature, pressure, flow rate, rotational speed, among others. 2) Data preprocessing. Throughout the entire operational cycle, some sensor data exhibit little to no variation, indicating no correlation with engine degradation. In this study, data variance is used to represent the degree of change-smaller variance values indicate less variability41. The results of the variance analysis are shown in Fig. 9. It can be observed that the variance of sensor data s9, s14, s4, s3, and s17 demonstrates the most significant changes, hence these sensors are retained in the dataset. The data from all other sensors are discarded to reduce dimensionality and improve computational efficiency. Fig. 9Variance of sensor data. Variance of sensor data. Since the sequence length of each engine varies and to increase the number of samples, the sliding window method was similarly employed to segment the time series data. The window length was set to 32 with a stride of 2. A total of 8800 samples were obtained. Subsequently, 11 statistical features were extracted from each sample to serve as model inputs, followed by data normalization. The mean value was also computed, resulting in an 11-dimensional feature matrix for each type of signal. Finally, PCA was applied to reduce the dimensionality of these feature matrices. Figure 10 shows the contribution rate plot of the principal components during the PCA dimensionality reduction process applied to the feature matrices corresponding to sensor data s9, s14, s4, s3, and s17. The top five principal components for each type of signal, which collectively account for over 80% of the cumulative information contribution rate, were selected as the input features for the AM-CNN-BiGRU model. After completing the statistical feature dimensionality reduction, the RUL labels of the data were packaged using the same method as applied to the robot dataset. The final dimension of each input sample obtained is (5, 30, 5). Finally, the resulting dataset was divided into training, validation, and test sets following the same ratio as used for the robot dataset. Fig. 10PCA principal component contribution rate for key sensor data of the C-MAPSS. PCA principal component contribution rate for key sensor data of the C-MAPSS. Design of experiments The models used in this experiment were written based on Python 3.8 using the Tensorflow framework as well as the Scikit-learn machine learning library. And the model was trained on a laptop with an NVIDIA RTX 4070 GPU. The designed experiment consists of three parts. In the first part, the AM-CNN-BiGRU model is trained using the Lion optimizer, Adam optimizer, and SGDM optimizer, respectively, to verify the effectiveness of the Lion optimizer for deep learning model training for industrial datasets. The settings of the main parameters of each optimizer in the experiment are shown in Table 5. In the second part, a set of ablation studies are conducted to compare the performance differences between the model with the removal of the attention mechanism AM (CNN-BiGRU), the model with the removal of the CNN (AM-BiGRU), and the model with the removal of the BiGRU (AM-CNN) and the AM-CNN-BiGRU, respectively, to validate the contribution of each module and the rationality of the architectural design. In the third part, comparative experiments were conducted on two datasets, comparing the AM-CNN-BiGRU model with several widely-used algorithms in time series prediction. These included two shallow learning algorithms (SVR and RF), three deep learning algorithms (TCN, LSTM and Transformer), and two recently proposed fault prediction algorithms (CNN-MMHA42 and BiLSTM-Trans43, further validating the superior performance of AM-CNN-BiGRU in predictive maintenance applications. The main hyperparameters for each algorithm, obtained through the grid search algorithm, are shown in Table 6. Table 5Main parameters of the optimizers.OptimizerParametersSGDMlearn_rate = 1e-2; beta = 0.9Adamlearning_rate = 1e-3; beta_1 = 0.9; beta_2 = 0.999; epsilon = 1e-7; weight_decay = NoneLionlearning_rate = 1e-4; beta1 = 0.9; beta2 = 0.99; weight_decay = None Table 5Main parameters of the optimizers.OptimizerParametersSGDMlearn_rate = 1e-2; beta = 0.9Adamlearning_rate = 1e-3; beta_1 = 0.9; beta_2 = 0.999; epsilon = 1e-7; weight_decay = NoneLionlearning_rate = 1e-4; beta1 = 0.9; beta2 = 0.99; weight_decay = None Main parameters of the optimizers. Table 6Algorithm hyperparameters.AlgorithmHyperparametersSVRKernel=“rbf”; C = 10; gamma=’scale’; epsilon = 0.05RFn_estimators = 200; max_depth = 10; min_samples_split = 2TCNConvolution layers = 4; kernel size = 3; filters = 64; dilations=[1, 2, 4, 8]; Dropout = 0.3LSTMunits = 64; num_layers = 2; Dropout = 0.2; return_sequences = TrueTransformernum_heads = 4; key_dim = 32; d_model = 128; dff = 512; num_layers = 2CNN-MMHAThe shape of WConversion = 13 × 49; convolutional layers = 3; fully connected layers = 2BiLSTM-TransAttention head size = 3; Gradient clip value = 0.03; Hidden continuous size = 73; Hidden size = 162; BiLSTM layers = 4 Table 6Algorithm hyperparameters.AlgorithmHyperparametersSVRKernel=“rbf”; C = 10; gamma=’scale’; epsilon = 0.05RFn_estimators = 200; max_depth = 10; min_samples_split = 2TCNConvolution layers = 4; kernel size = 3; filters = 64; dilations=[1, 2, 4, 8]; Dropout = 0.3LSTMunits = 64; num_layers = 2; Dropout = 0.2; return_sequences = TrueTransformernum_heads = 4; key_dim = 32; d_model = 128; dff = 512; num_layers = 2CNN-MMHAThe shape of WConversion = 13 × 49; convolutional layers = 3; fully connected layers = 2BiLSTM-TransAttention head size = 3; Gradient clip value = 0.03; Hidden continuous size = 73; Hidden size = 162; BiLSTM layers = 4 Algorithm hyperparameters. Evaluation metrics Four model evaluation metrics, namely, the coefficient of determination R2, the mean absolute error MAE, the mean absolute percentage error MAPE, and the root mean square error RMSE, are used to evaluate the prediction results of different algorithmic models. R² is a statistic that measures the goodness of fit of a regression model, which indicates how well the regression model fits the observations and represents the percentage of the dependent variable in the model that can be explained by the independent variable. The larger the value of R2, the better the prediction of the regression model, which is calculated by the following formula.25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{R^2}~ = ~1 - \frac{{\mathop \sum \nolimits_{i = 1}^n {{\left( {{y_i} - {{\hat y}_i}} \right)}^2}}}{{\mathop \sum \nolimits_{i = 1}^n {{\left( {{y_i} - {{\bar y}_i}} \right)}^2}}}}$$\end{document} MAE measures the average difference between the model’s predicted value and the true value in absolute terms. The closer the MAE value is to 0, the better the prediction is and the higher the model’s prediction accuracy is. The calculation formula is as follows.26\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathrm{MAE}}~ = ~\frac{{\mathrm{1}}}{{\mathrm{n}}}\mathop \sum \limits_{{\mathrm{i}} = 1}^{\mathrm{n}} \left| {{{\mathrm{y}}_{\mathrm{i}}} - {{{{\hat y}}}_{\mathrm{i}}}} \right|}$$\end{document} MAPE is a variant of MAE. This evaluation metric is in the form of a percentage and is not affected by outliers. The formula for calculating MAPE is as follows.27\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathrm{MAPE}}~ = ~\frac{1}{n}\mathop \sum \limits_{i = 1}^n \left| {\frac{{{y_i} - {{\hat y}_i}}}{{{y_i}}}} \right| \times 100\% }$$\end{document} RMSE is the arithmetic square root of the mean square error MSE, also the most commonly used model evaluation metric in regression prediction models, and is mainly used to measure the standard deviation of the prediction error. The formula is as follows.28\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathrm{RMSE~}} = ~\sqrt {\frac{1}{n}\mathop \sum \limits_{i = 1}^n {{({y_i} - {{\hat y}_i})}^2}} }$$\end{document} Results and discussion On the industrial robot dataset, the AM-CNN-BiGRU model was trained using three optimizers. Figure 11 shows the change in the loss function value of the model during the training process. From the comparison of the convergence curves of the three optimizers in the figure, it can be seen that it takes about 50 iterations to reach the convergence state using the Adam optimizer, and about 40 iterations to reach the convergence state using the SGDM optimizer. Moreover, the two optimizers have poorer smoothness in the training process. However, the training using the Lion optimizer has converged after 25 iterations, and the trend is smooth during the whole training period. And it is more stable after convergence, and the final convergence effect is better than Adam and SGDM optimizers. Fig. 11Training loss rate of the AM-CNN-BiGRU. Training loss rate of the AM-CNN-BiGRU. Table 7 shows the prediction performance results of the models trained using Adam optimizer, SGDM optimizer, and Lion optimizer, along with the average of the model training time. The comparison shows that the performance of AM-CNN-BiGRU trained using the Lion optimizer is better than that trained using the Adam optimizer and the SGDM optimizer under the four evaluation metrics. R2 is 0.968, MAE is 2.15 × 103, MAPE is 7.79% and RMSE is 2.71 × 103. The R2, MAE, MAPE, and RMSE of the Lion optimizer are improved by 3.53%, 28.09%, 20.67%, and 27.35%, respectively, compared to the Adam optimizer, while they are improved by 5.10%, 34.05%, 21.79%, and 29.61%, respectively, compared to the SGDM optimizer. In addition, it can be seen from the training time of the model that the Lion optimizer has shortened the average training time by 26.77% and 21.68% compared to the Adam optimizer and SGDM optimizer, respectively, and has faster operation speed and higher operation efficiency. Table 7Performance metrics of the AM-CNN-BiGRU.Algorithm R 2 MAE (×103)MAPE (%)RMSE (×103)Time (min)Lion 0.968 2.15 7.79 2.71 31.21 Adam0.9352.999.823.7342.62SGDM0.9213.269.963.8539.85 Table 7Performance metrics of the AM-CNN-BiGRU.Algorithm R 2 MAE (×103)MAPE (%)RMSE (×103)Time (min)Lion 0.968 2.15 7.79 2.71 31.21 Adam0.9352.999.823.7342.62SGDM0.9213.269.963.8539.85 Performance metrics of the AM-CNN-BiGRU. The above results demonstrate that the performance improvement of the AM-CNN-BiGRU model trained with the Lion optimizer in this experiment can be attributed to multiple factors. Firstly, its update mechanism based on the sign function introduces an intrinsic regularization effect, significantly enhancing the model’s generalization capability on noisy data, thereby improving training stability and overall performance while avoiding overfitting. Secondly, its algorithm eliminates the need to compute the second moment, resulting in smaller memory footprint and higher operational efficiency, which accelerates model convergence. Leveraging these advantages, the Lion optimizer provides novel insights and a powerful tool for addressing challenges in industrial predictive maintenance, such as poor data quality, the need for rapid model iteration, and edge deployment constraints. In subsequent experiments, the Lion optimizer was utilized for training all deep learning models. The results of ablation studies on both datasets are presented in Table 8. The experimental findings demonstrate that the comprehensive performance of the AM-CNN-BiGRU model significantly surpasses that of variant models with individual modules removed. Specifically, on the robot dataset, removing the AM module led to a 3.31% decrease in R2, while MAE, MAPE, and RMSE increased by 44.65%, 52.37%, and 40.96% respectively. On the C-MAPSS dataset, removing the AM module resulted in a 3.36% reduction in R2, with MAE, MAPE, and RMSE increasing by 15.84%, 48.02%, and 9.99% respectively. This indicates that the AM module enables the model to effectively focus on critical temporal nodes. On the robot dataset, removing the CNN module caused a 5.27% decrease in R2, while MAE, MAPE, and RMSE increased by 59.07%, 56.87%, and 60.89% respectively. On the C-MAPSS dataset, removing the CNN module led to a 4.93% reduction in R2, with MAE, MAPE, and RMSE increasing by 21.92%, 69.82%, and 15.35% respectively. These results highlight the crucial role of CNN in spatial feature extraction. On the robot dataset, removing the BiGRU module resulted in a 4.34% decrease in R2, while MAE, MAPE, and RMSE increased by 55.35%, 59.05%, and 51.66% respectively. On the C-MAPSS dataset, removing the BiGRU module caused a 3.78% reduction in R2, with MAE, MAPE, and RMSE increasing by 20.35%, 65.15%, and 12.67% respectively. This demonstrates BiGRU’s essential role in temporal feature extraction and its critical function in capturing long-term bidirectional dependencies in time series. Figures 12 and 13 illustrate the changes in each metric when different modules are removed across both datasets. The results show that the absence of the CNN module has the most substantial impact on model performance. Table 8Results of ablation studies.Data setAlgorithmR2MAEMAPE (%)RMSERobotAM-CNN-BiGRU 0.968 2.15 × 10 − 3 7.79 2.71 × 10 − 3 CNN-BiGRU0.9363.11 × 10− 311.873.82 × 10− 3AM-BiGRU0.9173.42 × 10− 312.224.36 × 10− 3AM-CNN0.9263.34 × 10− 312.394.11 × 10− 3C-MAPSSAM-CNN-BiGRU 0.953 10.86 8.35 13.42 CNN-BiGRU0.92112.5812.3614.76AM-BiGRU0.90613.2414.1815.48AM-CNN0.91713.0713.7915.12 Table 8Results of ablation studies.Data setAlgorithmR2MAEMAPE (%)RMSERobotAM-CNN-BiGRU 0.968 2.15 × 10 − 3 7.79 2.71 × 10 − 3 CNN-BiGRU0.9363.11 × 10− 311.873.82 × 10− 3AM-BiGRU0.9173.42 × 10− 312.224.36 × 10− 3AM-CNN0.9263.34 × 10− 312.394.11 × 10− 3C-MAPSSAM-CNN-BiGRU 0.953 10.86 8.35 13.42 CNN-BiGRU0.92112.5812.3614.76AM-BiGRU0.90613.2414.1815.48AM-CNN0.91713.0713.7915.12 Results of ablation studies. Fig. 12Results of ablation studies on robot dataset. Results of ablation studies on robot dataset. Fig. 13Results of ablation studies on C-MAPSS dataset. Results of ablation studies on C-MAPSS dataset. The comparative experimental results of the fault prediction performance between the proposed AM-CNN-BiGRU and several other algorithms on both datasets are presented in Table 9. Meanwhile, Fig. 14 displays the visualization results of various deep learning algorithms on the two datasets. It can be observed that the AM-CNN-BiGRU model outperforms other models across all four evaluation metrics. On the robotic dataset, for R2, AM-CNN-BiGRU achieved a 1.26% improvement compared to the best-performing baseline algorithm BiLSTM-Trans, while for MAE, MAPE, and RMSE, it demonstrated improvements of 15.35%, 18.26%, and 14.78% respectively. On the C-MAPSS dataset, for R2, AM-CNN-BiGRU showed a 0.95% improvement over BiLSTM-Trans, with improvements of 1.63%, 37.03%, and 1.40% for MAE, MAPE, and RMSE respectively. Beyond the comparison with BiLSTM-Trans, further analysis incorporating the performance of CNN-MMHA, Transformer, LSTM, and TCN reveals that the AM-CNN-BiGRU model effectively utilizes CNN’s exceptional local feature extraction capability to accurately detect early and subtle fault indicators. The bidirectional gating mechanism of BiGRU enables the model to simultaneously capture temporal dependencies from both forward and backward directions, thereby precisely identifying fault progression patterns. Furthermore, the Attention Mechanism (AM) provides adaptive focusing capability, dynamically evaluating the significance of different data sources. These advanced capabilities of the AM-CNN-BiGRU model are particularly challenging to achieve using conventional single-model architectures. A comparative analysis of the convergence speed among several deep learning models during the training process was also conducted, with the number of epochs required for the model loss value to converge below 0.01 serving as the evaluation metric. The results demonstrate that the proposed AM-CNN-BiGRU model significantly outperforms all other compared models in terms of convergence speed. On the two datasets, it required only 25 epochs and 22 epochs, respectively, to reach the target loss value. The efficient integration of different modules in our proposed model contributes to a more efficient and stable optimization process. Table 9Results of comparative experiments.Data setAlgorithmR2MAEMAPE (%)RMSEEpochs requiredRobotSVR0.7645.69 × 10− 320.217.36 × 10− 3–RF0.7725.75 × 10− 321.757.22 × 10− 3–TCN0.8824.21 × 10− 315.525.19 × 10− 358LSTM0.9093.60 × 10− 312.314.55 × 10− 341Transformer0.9373.11 × 10− 311.393.79 × 10− 371CNN-MMHA0.9482.78 × 10− 310.923.41 × 10− 335BiLSTM-Trans0.9562.54 × 10− 39.533.18 × 10− 349AM-CNN-BiGRU 0.968 2.15 × 10 − 3 7.79 2.71 × 10 − 3 25 C-MAPSSSVR0.80819.9139.9825.26–RF0.85617.5130.4721.94–TCN0.91613.5826.3916.7235LSTM0.92213.2729.6716.1949Transformer0.93111.8119.3115.2164CNN-MMHA0.93611.4519.0614.5631BiLSTM-Trans0.94411.0413.2613.6145AM-CNN-BiGRU 0.953 10.86 8.35 13.42 22 Table 9Results of comparative experiments.Data setAlgorithmR2MAEMAPE (%)RMSEEpochs requiredRobotSVR0.7645.69 × 10− 320.217.36 × 10− 3–RF0.7725.75 × 10− 321.757.22 × 10− 3–TCN0.8824.21 × 10− 315.525.19 × 10− 358LSTM0.9093.60 × 10− 312.314.55 × 10− 341Transformer0.9373.11 × 10− 311.393.79 × 10− 371CNN-MMHA0.9482.78 × 10− 310.923.41 × 10− 335BiLSTM-Trans0.9562.54 × 10− 39.533.18 × 10− 349AM-CNN-BiGRU 0.968 2.15 × 10 − 3 7.79 2.71 × 10 − 3 25 C-MAPSSSVR0.80819.9139.9825.26–RF0.85617.5130.4721.94–TCN0.91613.5826.3916.7235LSTM0.92213.2729.6716.1949Transformer0.93111.8119.3115.2164CNN-MMHA0.93611.4519.0614.5631BiLSTM-Trans0.94411.0413.2613.6145AM-CNN-BiGRU 0.953 10.86 8.35 13.42 22 Results of comparative experiments. Fig. 14Results of comparative experiments. (a) Industrial robot dataset, (b) C-MAPSS dataset. Results of comparative experiments. (a) Industrial robot dataset, (b) C-MAPSS dataset. Among the test samples, five temporal sample points (A, B, C, D, E) exhibited repeat positioning accuracy that exceeded the threshold, resulting in faults. As shown in Fig. 15(a), all five fault points were accurately predicted using the AM-CNN-BiGRU model. Figure 15(b) demonstrates that while all five fault points were correctly predicted by the BiLSTM-Trans model, the non-fault point F was erroneously predicted as faulty. According to Fig. 15(c), the CNN-MMHA model accurately predicted four fault points (B, C, D, E) but failed to correctly predict point A. Figure 15(d) shows that the Transformer model similarly accurately predicted four fault points (B, C, D, E), while missing point A and falsely identifying non-fault point G as faulty. As illustrated in Fig. 15e, the LSTM model not only failed to predict point A accurately but also mistakenly identified non-fault points G and H as faulty. Finally, Fig. 15f shows that the TCN model failed to correctly predict points A and C, while generating false predictions for non-fault points I and F. Fig. 15Visualization of prediction results. Visualization of prediction results. Conclusion This paper proposes a predictive maintenance method for industrial robots using an AM-CNN-BiGRU model with spatiotemporal feature fusion and multi-source data fusion. The model uses vibration, current, and torque data from each joint of the robot as input, avoiding the limitations of single-source data. First, an attention mechanism is proposed to assign different weights to multi-source data, determining the importance of different data sources. The weighted multi-source data sequences are then input into the CNN as different channels. The CNN extracts complex high-dimensional spatial features from the input sequences. Considering the variable periodicity of time series, the BiGRU is further employed to learn inherent temporal dependencies and extract high-dimensional temporal features. Additionally, the Lion optimizer is innovatively introduced into the training process of the predictive maintenance model proposed in this paper. To validate the effectiveness and generalization capability of the proposed AM-CNN-BiGRU model, two sets of experiments were conducted. One involved building an industrial robot experimental platform for data acquisition to construct a dataset and perform experiments, while the other utilized the C-MAPSS dataset for validation. First, on the robot dataset, the AM-CNN-BiGRU model was trained using the Lion optimizer, Adam optimizer, and SGDM optimizer, respectively. The results confirmed the effectiveness of the Lion optimizer for training deep learning models on industrial datasets. Subsequently, ablation studies were performed on both datasets to compare the performance differences between the proposed AM-CNN-BiGRU and its variants: the model with the Attention Mechanism (AM) removed (CNN-BiGRU), the model with the CNN module removed (AM-BiGRU), and the model with the BiGRU module removed (AM-CNN). These results validated the rationality of the model’s architectural design. Finally, comparative experiments were carried out on both datasets, pitting AM-CNN-BiGRU against several popular algorithms in the field of time series prediction. These included two shallow learning algorithms (SVR and RF), three deep learning algorithms (TCN, LSTM, and Transformer), and two state-of-the-art fault prediction models (CNN-MMHA and BiLSTM-Trans). The results demonstrated that the proposed AM-CNN-BiGRU model achieved superior prediction performance on both datasets. This study reveals the significant potential of integrated models in the industrial AI field, providing a powerful tool for transforming industrial robot maintenance strategies from “scheduled prevention” to “precision prediction”. More importantly, this research offers a universal spatiotemporal feature fusion framework that can be transferred to other industrial equipment fault diagnosis scenarios, facilitating the deep evolution of industrial operations towards intelligence. However, due to time constraints, data were collected only under a single operational condition of the robot. Research on model performance across different working conditions (including studies on the differential contributions of multi-source data to fault prediction under varying conditions) and lightweight deployment of the model will be our future research directions.