A Temporal Network Based on Characterizing and Extracting Time Series in Copper Smelting for Predicting Matte Grade Authors: Junjia Zhang, Zhuorui Li, Enzhi Wang, Bin Yu, Jiangping Li, Jun Ma, Jongmyon Kim Published: 2024 ABSTRACT AbstractAddressing the issues of low prediction accuracy and poor interpretability in traditional matte grade prediction models, which rely on pre-smelting input and assay data for regression, we incorporate process sensors’ data and propose a temporal network based on Time to Vector (Time2Vec) and temporal convolutional network combined with temporal multi-head attention (TCN-TMHA) to tackle the weak temporal characteristics and uncertain periodic information in the copper smelting process. Firstly, we employed the maximum information coefficient (MIC) criterion to select temporal process sensors’ data strongly correlated with matte grade. Secondly, we used a Time2Vec module to extract periodic information from the copper smelting process variables, incorporates time series processing directly into the prediction model. Finally, we implemented the TCN-TMHA module and used specific weighting mechanisms to assign weights to the input features and prioritize relevant key time step features. Experimental results indicate that the proposed model yields more accurate predictions of copper content, and the coefficient of determination (R2) is improved by 2.13% to 11.95% and reduced compared to the existing matte grade prediction models.Keywords: copper smelting process sensors’ data, temporal representation and feature extraction, temporal convolutional network, matte grade prediction 1. Introduction In the copper smelting process, the copper matte grade is an important production indicator that reflects the quality of the smelting process. A suitable matte grade can reduce the material consumption and blowing time in the subsequent converter smelting, while an unsuitable grade will increase consumption [1]. The stability of the matte grade is essential for ensuring the efficient and safe operation of subsequent production [2]. However, the manual sampling, cooling, and analysis of matte are conducted during the smelting process, resulting in a time delay of over one hour. Due to this detection delay and human factors, the measured matte grade indicators are difficult to feedback in a timely manner to the furnace system, which hinders the tuning of the control system [3]. Therefore, how to accurately predict the matte grade to optimize the operating parameters and improve the efficiency of copper smelting process has been a challenge for managers and researchers [4]. Currently, data-driven based copper grade prediction methods include regression prediction and recursive prediction. Regression prediction methods construct linear mathematical models by analyzing the relationships between variables, characterized by high accuracy and ease of implementation. Since 2007, these methods have been widely used in copper grade prediction [2]. Weihua Gui et al. [2] developed an early copper grade prediction model using multi-phase fuzzy neural network with constrained gradient descent for parameter updating. Shouyi Yu et al. [5] and Luo Zhao et al. [6] proposed a method based on back propagation neural networks (BP) to mine the parameter information among copper grade, copper temperature, and the silicon–iron ratio in slag to achieve copper grade prediction. Peng Deng et al. [7] combined long short-term memory networks (LSTMs) with mechanistic models to predict four parameters (oxygen enrichment, abrasive grade, control parameters, and temperature). Xincai Li et al. [8] optimized support vector machines (SVMs) using sparrow search algorithm to establish a nonlinear relationship model between copper grade and the amounts of copper ore, fuel, and ore composition for prediction. Peng Xiaobo et al. [9] considered the difficulty of statistically and mechanistically determining the parameters. They proposed a dynamic T-S recursive fuzzy neural network to derive the influence weights of various mineral quantities and constructed a soft measurement model for copper grade. Although these regression-based prediction methods achieved satisfying results, they rely on real-time process variable inputs for output results, and this leads to less interpretability for intermediate processes and low accuracy of prediction results. This still fails to address the issues of guidance lag in production and the timely feedback needed for furnace parameter adjustments. In recent years, with the growing demand and development of industrial prediction, recursive prediction based on deep learning has gradually become a trend, addressing the aforementioned issues [10]. This method predicts the copper grade of the next furnace by learning patterns from historical sensor process data to give on-site personnel enough time and strategies to adjust and guide production. Current methods include artificial neural networks (ANNs) [11], temporal convolutional networks (TCNs) [12], bidirectional long short-term memory networks (BiLSTMs) [13], BiGRU [14], and the Transformer network proposed by the Google team [15] and its variant, Informer [16]. Among these, the TCN model uses residual connections with convolution to control the feature extraction process, making it more effective than convolutional neural networks (CNNs) in handling sequential data with time characteristics, and allowing it to recursively capture temporal patterns, thus becoming a popular model for time series prediction. Mao et al. [10] proposed a recursive model that takes into account the temporal analysis in the field of matte grade prediction and demonstrated that, by considering temporal correlations, the model yields more accurate predictions. Lin et al. [17] applied the TCN to perform multi-layer convolution operations on the input time feature sequences, enabling the learning of global temporal feature information. Liang et al. [18] utilized stacked TCNs with attention mechanisms for spatiotemporal prediction. It is shown that leveraging the model can effectively enhance prediction accuracy. Pan et al. [19] performed sequential prediction through the extraction of multi-period features under different time windows. The TCN model with its powerful residual structure and dilated causal convolution pattern, effectively captures temporal information within large-scale data, making it a popular choice for time series prediction [20]. However, as noted in the aforementioned [17,18,19], although TCN can capture dependencies within a certain time range, its hierarchical structure limits the growth of the receptive field as the depth increases. This limitation makes it difficult for TCN to cover the global information of long time series, preventing it from fully capturing critical temporal information. Moreover, with the advent of the Transformer module and the attention mechanism [15], there has been a significant advantage in capturing long-term dependencies and global information, which enables better extraction of critical temporal features. Consequently, attention mechanisms are commonly incorporated into the current time series prediction to achieve higher prediction accuracy. Guangxun E et al. [21] introduced attention mechanisms in 1D-CNN to explore the temporal and spatial correlations of multi-dimensional feature sequences for high-precision predictions. Li et al. [22] introduced a dual attention mechanism for feature space and temporal aspects based on TCN, dynamically obtaining important metrics from both feature and time dimensions to improve prediction accuracy. Yang et al. [23] integrated a multi-head attention into the traditional generative adversarial network (GAN) method to better learn the latent structure of chillers’ fault data. Li et al. [24] utilized the improved layer-wise relevance propagation (ImLRP) method, employing attention relevance scores to assess the contribution of each neuron to the output. Zhao et al. [25] proposed a convolutional adversarial multi-wavelet neural network (CAMCNN) to address the issue of insufficient utilization of target data for feature extraction. The prediction accuracy of TCN, LSTM, and GRU models was improved by incorporating an attention mechanism. It is evident that the attention mechanism can effectively help the model focus on the most important parts of the input data, thereby enhancing its ability to capture critical information. This mechanism allows the model to dynamically adjust its attention to different time steps in the input sequence based on the current task, making it more flexible and efficient when processing complex time series data. However, considering the inconsistent production periods in copper smelting process, this leads to the following issues: (1)Uncertain periodic information of input variables: Due to the influence of actual industrial production on-site, there may be irregular production intervals caused by human factors. With fluctuating amounts added, the periodic characteristics of the time series information are unclear. Uncertain periodic information of input variables: Due to the influence of actual industrial production on-site, there may be irregular production intervals caused by human factors. With fluctuating amounts added, the periodic characteristics of the time series information are unclear. (2)Difficult to extract periodic features: In the copper smelting process, the influence of temporal characteristics for different input variables on the output results is different at each time step. It is essential to enhance the weights of input variables at key time steps, which can extract the key periodic features to ensure the accuracy of matte grade prediction. Difficult to extract periodic features: In the copper smelting process, the influence of temporal characteristics for different input variables on the output results is different at each time step. It is essential to enhance the weights of input variables at key time steps, which can extract the key periodic features to ensure the accuracy of matte grade prediction. In response to the abovementioned issues, many scholars in the industrial field have adopted the following solutions: Fargalla Mandella et al. [26] proposed a combination model called Time2Vec attention-based CNN-BiGRU neural network, Zhou et al. [27] introduced a model called an adaptive timing encoding mechanism based Transformer with Time2Vec, Li et al. [28] developed the PSO-TCN-attention neural network, and Cai et al. [29] presented a malicious network traffic detection model based on a bidirectional temporal convolutional network with a multi-head self-attention mechanism. All these models have effectively addressed various time-related issues in their respective fields, achieving excellent results. Therefore, we build upon their research and propose improvements, proposing a novel matte grade prediction method based on Time2Vec and TCN-TMHA, and focusing on enhancing the temporal periodicity of the input variables and extracting key time step features within the production cycle. The main contributions and innovations of this paper are as follows: (a)Utilization of Time to Vector (Time2Vec) for temporal embedding: The Time2Vec temporal embedding layer captures periodic relationships among different copper smelting monitoring data variables, thereby constructing periodic characteristics. This provides a supportive approach to build periodic features that are fed into the predictive model, offering robust data support for subsequent matte grade prediction. Utilization of Time to Vector (Time2Vec) for temporal embedding: The Time2Vec temporal embedding layer captures periodic relationships among different copper smelting monitoring data variables, thereby constructing periodic characteristics. This provides a supportive approach to build periodic features that are fed into the predictive model, offering robust data support for subsequent matte grade prediction. (b)Combination Model of TCN-Temporal Multi-Head Attention (TMHA): A combination model of TCN-TMHA is utilized. This model leverages dilated causal convolutions and residual connections to enhance the modeling of complex temporal dependencies in the copper smelting process. It also employs TMHA mechanisms to explore the coupling relationships between process variables and copper grade, while using specific weighting mechanisms to enhance the representation of time steps and input features, thereby strengthening the information expression of important periodic features. Combination Model of TCN-Temporal Multi-Head Attention (TMHA): A combination model of TCN-TMHA is utilized. This model leverages dilated causal convolutions and residual connections to enhance the modeling of complex temporal dependencies in the copper smelting process. It also employs TMHA mechanisms to explore the coupling relationships between process variables and copper grade, while using specific weighting mechanisms to enhance the representation of time steps and input features, thereby strengthening the information expression of important periodic features. (c)Addressing Dependency on Real-Time Process Variables: In the field of copper grade prediction, this method resolves the issue of regression prediction relying on real-time process variable inputs. By using recursive models for time series prediction that do not depend on real-time data, it leverages historical data to anticipate future copper grade levels, enabling timely on-site parameter adjustments and improving prediction accuracy. Addressing Dependency on Real-Time Process Variables: In the field of copper grade prediction, this method resolves the issue of regression prediction relying on real-time process variable inputs. By using recursive models for time series prediction that do not depend on real-time data, it leverages historical data to anticipate future copper grade levels, enabling timely on-site parameter adjustments and improving prediction accuracy. 2. Theory and Methodology 2.1. Problem Description Our datasets are based on production data from a copper smelting enterprise in the southwestern region of China. As shown in Figure 1, the copper smelting process aims to obtain blister copper. First, ore is obtained from the raw material storage, and then, materials coal, quartz, sulfur ore, copper concentrate and cold material are added and mixed before being fed into the copper smelting furnace. To achieve the desired grade of blister copper, it is necessary to adjust the opening of the burner, fuel, and flux using sensors and control valves to regulate the production process. However, the grade of blister copper cannot be measured directly by sensors, so data from other sensors are used for prediction. The process variables collected by sensors for matte grade measurement. Traditional regression prediction methods require the construction of a soft measurement regression model, which cannot provide timely guidance for production. Therefore, this paper employs a time series recursive model to explore the temporal patterns of data changes in the copper smelting process, thus enabling the prediction of blister copper grade based on historical time steps. We establish a prediction model for the grade of blister copper and initially select 19 process data inputs obtained through sensor collection and manual analysis as time series inputs. These inputs include the contents of Cu, S, Fe, Si, and Ca in the copper concentrate, the amount of flux added, the iron-silicon ratio in the slag, oxygen concentration, oxygen flow rate, molten pool temperature, and burner height, represented as time series inputs x1~x19, the target label for predicting copper grade is denoted as Y. In the copper smelting process, where the time cycle is not fixed (ranging from 55 to 70 min), various input time series data exhibit inter-coupling, weak periodic expressions, and a lack of distinct time step characteristics. These issues bring challenges for sequential prediction of matte grade. Therefore, the focus of this study is to enhance the periodicity of the input data and increase the weight of the input features at key time steps in the sequence, thereby achieving a relatively high accuracy in the sequential prediction of matte grade. 2.1.1. Characterization of the Periodicity of Time Series Signals As an example, the original time series characteristics of the copper concentrate input are shown in Figure 2 (upper). Due to the influence of actual industrial production on-site, the production cycle is irregular, and anomalies caused by human factors result in the periodicity of the time series data being unclear. How to represent the periodic and non-periodic characteristics of the original data, thereby showcasing the time series characteristics of the production data, is one of the challenges faced in predicting the grade of blister copper. Periodic representation of time series signals based on Time2Vec. To address this, we utilize Time2Vec to encode the input time series data, providing a representation of temporal information across multiple time series [23]. The structure of Time2Vec is illustrated in Figure 2. Convert time data into representations for both periodic and non-periodic features, which allows for the extraction of high-order feature representations as follows: where k is the variable that defines the periodic regions within the high-order features. For the i component, when i = 0, it represents a linear component, while for 0