Robust Epileptic Seizure Detection Using Long Short-Term Memory and Feature Fusion of Compressed Time–Frequency EEG Images Authors: Shafi Ullah Khan, Sana Ullah Jan, Insoo Koo, Yvonne Tran Published: 2023 ABSTRACT AbstractEpilepsy is a prevalent neurological disorder with considerable risks, including physical impairment and irreversible brain damage from seizures. Given these challenges, the urgency for prompt and accurate seizure detection cannot be overstated. Traditionally, experts have relied on manual EEG signal analyses for seizure detection, which is labor-intensive and prone to human error. Recognizing this limitation, the rise in deep learning methods has been heralded as a promising avenue, offering more refined diagnostic precision. On the other hand, the prevailing challenge in many models is their constrained emphasis on specific domains, potentially diminishing their robustness and precision in complex real-world environments. This paper presents a novel model that seamlessly integrates the salient features from the time–frequency domain along with pivotal statistical attributes derived from EEG signals. This fusion process involves the integration of essential statistics, including the mean, median, and variance, combined with the rich data from compressed time–frequency (CWT) images processed using autoencoders. This multidimensional feature set provides a robust foundation for subsequent analytic steps. A long short-term memory (LSTM) network, meticulously optimized for the renowned Bonn Epilepsy dataset, was used to enhance the capability of the proposed model. Preliminary evaluations underscore the prowess of the proposed model: a remarkable 100% accuracy in most of the binary classifications, exceeding 95% accuracy in three-class and four-class challenges, and a commendable rate, exceeding 93.5% for the five-class classification.Keywords: artificial intelligence, EEG, seizure detection, continues wavelet transform, hybrid features 1. Introduction Approximately 1% of the global population is affected by epilepsy [1]. This condition poses significant challenges that can even be life-threatening for those affected. Among these patients, one-third do not respond to medications and need physical interventions [2,3]. Epileptic seizures are characterized by swift and abnormal fluctuations in the electrical patterns of the brain [4]. In severe cases, they can cause the entire body to become unresponsive [5]. Electroencephalogram (EEG) signals have been the fundamental reference for detecting epileptic seizures, helping to identify the seizure origin, and facilitating the treatment of the affected brain tissues through medication and surgical procedures [6]. EEG signals contain significant features that detail both regular and irregular brain activities, particularly epileptic seizures. In addition, high-temporal-resolution EEG data from the scalp, spanning multiple input channels, can be acquired through distributed continuous sensing techniques [7]. Traditionally, diagnosing epilepsy through visual analysis of EEG recordings, both clinically and conventionally, is labor-intensive and prone to error, with varying consistency among experts, because of its heavy reliance on human expertise and skill [8,9]. Many EEG automatic seizure detection systems struggle with real-time specificity and sensitivity, making them less suitable for clinical applications. There is a pressing need for an advanced computer-aided system that can efficiently assist neurologists in detecting epileptic seizures, ultimately reducing the time spent analyzing extensive EEG recordings [10]. In areas with a scarcity of neurologists, the excessive dependence on human expertise can increase the costs and cause delays in treating epilepsy. Tackling these issues is essential to guarantee affordable epilepsy care in low-to-middle-income regions, particularly in isolated locations with restricted access to skilled professionals and advanced facilities. Improving access to automated seizure detection using EEG signals has been studied extensively to mitigate this issue [11]. Machine learning is used widely to detect diseases automatically from biomedical signals, such as ECG and EEG. For example, a previous study [12] used two distinct features to detect epileptic seizures: fractal-based nonlinear features and entropy-based features. These features were inputted into two machine learning classifiers: Support Vector Machine (SVM) and K-Nearest Neighbor (KNN). The classifiers were trained and tested on the Bonn Epilepsy database. This database comprises five distinct classes: Set S, Set F, Set N, Set O, and Set Z. Set S represents seizure activity typically observed in epileptic patients. Both Set F and Set N denote seizure-free states in the epileptic class, Set O is associated with a normal, non-epileptic state where the subject’s eyes are closed, while Set Z corresponds to the normal state with the subject’s eyes open. In their evaluation, they considered binary (e.g., Z–S, O–S, and N–S) and the three-class detection problem (ZO–NF–S). In addition, another study [13] introduced a framework that integrates fuzzy-based methods and conventional machine-learning techniques to identify epileptic EEG samples in binary classification problems. A limited set of features and linear (using the Naïve Bayes classifier) and nonlinear (using the K-Nearest Neighbor classifier) approaches were applied to classify the EEG samples [14]. Binary classification tasks were involved in classifying various classes, i.e., Z–S, O–S, N–S, F–S, ZO–S, and ZN–E. Similarly, another study [15] used the statistical features and classified with SVM (AdaBoost Least-Square SVM). The resulting accuracy for the binary FNOZ-S classification problem in the Bonn dataset was 99%. In particular, none of these authors extended the evaluation of their proposed methods to include multi-class classifications. Beyond traditional machine learning techniques, various deep learning architectures have been introduced to detect epileptic seizures in the EEG data. A previous study [16] utilized deep learning approaches to extract the important features from EEG data. In particular, a Convolutional Neural Network (CNN) was implemented for the differentiation tasks among normal, preictal, and seizure classes. The author of [17] introduced an experimental and methodological approach that mapped microscale local network dynamics with high spatiotemporal resolution and employed a quantitative analysis framework to elucidate the dynamics of seizure initiation and progression in vivo. In addition, the discrete wavelet transform (DWT) was used for feature extraction from the EEG data [18]. A combination of genetic algorithm and artificial neural network (ANN) and the Support Vector Machine (SVM) classifiers were used to address binary and three-class classification challenges in the Bonn Epilepsy database. Many seizure detection methods concentrate on specific domains, such as utilizing time–frequency domain methods, i.e., continuous wavelet transform (CWT), time domain, frequency domain, and statistical attributes [19,20,21,22]. Unlike the other methods, the proposed epileptic detection model innovatively combines the best of these attributes. A comprehensive set of important features is obtained by leveraging the complex insights from the statistical domain that is characterized by rich features, such as the mean, median, variance, skewness, and kurtosis, with the compressed time–frequency domain images (CWT Images) processed through an autoencoder. This hybrid integration of Convolutional Autoencoder (CAE) latent space and statistical features ensures model robustness, making it adept at capturing the most vital information for classification. A long short-term memory network was used to optimize the approach, allowing precise classifications ranging from binary to five-class classification challenges, particularly fine-tuned for the Bonn Epilepsy dataset. Contribution The main contributions of this work are as follows: This study introduces a significant advancement in epileptic seizure detection. The proposed novel deep learning method seamlessly merges the compressed latent space features from the time–frequency domain with statistical attributes of the EEG signal. This integrated feature pool captures time–frequency and statistical information, making this approach different in robustness and accuracy. This study introduces a significant advancement in epileptic seizure detection. The proposed novel deep learning method seamlessly merges the compressed latent space features from the time–frequency domain with statistical attributes of the EEG signal. This integrated feature pool captures time–frequency and statistical information, making this approach different in robustness and accuracy. The proposed hybrid model uses an optimal window size for EEG segmentation, ensuring minimal data loss and a set overlap ratio. After rigorous evaluation, this method selects the best window size for maximal data coverage, which is crucial for precise EEG classification. This strategy upholds data integrity, boosting the classification reliability of the model. The proposed hybrid model uses an optimal window size for EEG segmentation, ensuring minimal data loss and a set overlap ratio. After rigorous evaluation, this method selects the best window size for maximal data coverage, which is crucial for precise EEG classification. This strategy upholds data integrity, boosting the classification reliability of the model. A CAE was used for feature extraction from CWT images. CAEs excel at handling image data like EEG-based CWT by preserving spatial structures. The CAE retained the most important features and eliminated noise by compressing and reconstructing the image. This method reduced data dimensionality and identified the most vital EEG patterns, enhancing precision and accuracy in subsequent analysis. A CAE was used for feature extraction from CWT images. CAEs excel at handling image data like EEG-based CWT by preserving spatial structures. The CAE retained the most important features and eliminated noise by compressing and reconstructing the image. This method reduced data dimensionality and identified the most vital EEG patterns, enhancing precision and accuracy in subsequent analysis. The CAE latent space features still contain some less important features. Principal Component Analysis (PCA) was applied to extract the most relevant features from the latent space, enhancing the classification accuracy. The CAE latent space features still contain some less important features. Principal Component Analysis (PCA) was applied to extract the most relevant features from the latent space, enhancing the classification accuracy. LSTM networks were used for classification, capitalizing on their proficiency with time-series data. Given the sequential nature of the EEG signals, LSTMs, with their ability to capture long-term dependencies, provided enhanced accuracy in detecting intricate seizure patterns. LSTM networks were used for classification, capitalizing on their proficiency with time-series data. Given the sequential nature of the EEG signals, LSTMs, with their ability to capture long-term dependencies, provided enhanced accuracy in detecting intricate seizure patterns. While many studies evaluate the Bonn dataset for binary classification, some extend to three or four classes, with few tackling a five-class problem. This study encompassed classifications from binary to five class, achieving unprecedented accuracy, i.e., 100% for binary, >95% for three and four classes, and above 93% for the five-class categorization, marking the highest recorded performance in terms of accuracy. While many studies evaluate the Bonn dataset for binary classification, some extend to three or four classes, with few tackling a five-class problem. This study encompassed classifications from binary to five class, achieving unprecedented accuracy, i.e., 100% for binary, >95% for three and four classes, and above 93% for the five-class categorization, marking the highest recorded performance in terms of accuracy. The remainder of this article is organized as follows. Section 2 provides an in-depth explanation of the model design and components. Section 3 reports the dataset description and the model performance on the benchmark dataset. Finally, Section 4 provides the concluding remarks on the article. 2. Proposed Method This section provides an overview of the proposed methodology for epilepsy detection, leveraging a hybrid model that combines an autoencoder and a Recurrent Neural Network (RNN), specifically the long short-term memory (LSTM) variant. The procedure starts with a windowing technique, segmenting the continuous signal into smaller, manageable packets. This approach ensures that every datum is captured accurately. Once segmented, critical statistical features for each windowed segment are calculated, capturing the primary characteristics of the data. Subsequently, the continuous wavelet transform is applied to the segmented data. This transformation extracts time–frequency information from each segment, providing a more detailed representation of the signal dynamics. The resulting time–frequency images serve as input to the Convolutional Autoencoder, which distills the data into a latent feature space. Owing to the potential high dimensionality of this latent space, PCA was implemented to streamline the feature set, retaining only those components that contribute significantly to the variance and, by extension, the classifiability of the data. These condensed features are merged with the previously computed statistical features, producing a hybrid feature pool. This comprehensive feature set captures both the inherent characteristics of the signal and its nuanced, transformed representations. Finally, this paper introduces the LSTM model, which takes this hybrid feature set as input and determines the epilepsy state of the signal. The inherent capacity of the LSTM to process sequential data makes it particularly suited for this task, ensuring accurate classifications across various detection scenarios. Figure 1 presents a visual representation of the entire process. Overview of the proposed epileptic seizure detection model. 2.1. Windowing The Bonn University Epilepsy dataset comprises five distinct subsets, Set Z, Set O, Set N, Set F, and Set S, and the details of which are described earlier in the introduction section. Each subset contains 100 samples, resulting in 500 samples across the entire dataset. In the present study, all 100 samples were chained, and a windowing technique was applied to create small segments of the EEG signal. In signal processing analysis, windowing plays a pivotal role, primarily in combating the challenges of spectral leakage. Spectral leakage is a key concern in signal processing, particularly relevant when analyzing EEG signals. It occurs when energy from the signal’s true frequency leaks into other frequencies, often due to the finite length of the signal window. This can distort the true frequency content of EEG data, potentially affecting the accuracy of seizure detection. Moreover, windowing enhances temporal localization, ensuring that specific spectral events are precisely mapped within distinct time frames. The technique also fine-tunes the frequency resolution, delineating closely packed frequency components with clarity [23]. Given the advantage, the sliding window technique was employed to partition each sample into multiple smaller signal segments. An overlapping sliding window method, implementing a 1458 data-point window with a 486 data-point overlap, was used to ensure no data-points were omitted. This window, shown in Figure 2, successively slides across the data, producing smaller signal segments, the combination of which represents the complete signal of the subject. The mathematical formulation of the sliding window technique with overlap, for a given signal S of length L, the starting and ending points of the ith windowed segment Si, is expressed below. Equation (1) indicates the starting point of each window, and Equation (2) expresses the ending point of the window. Windowing process. For the ith window, where; ω is the window length, and in this case, ω=1458. ω is the window length, and in this case, ω=1458. ϑ is the overlap length, and here, ϑ=486. ϑ is the overlap length, and here, ϑ=486. i is the window number (e.g., i=1 for the first window, i=2 for the second, so on). i is the window number (e.g., i=1 for the first window, i=2 for the second, so on). It should be always ensured that ω>0 for the above formulation to be valid. It should be always ensured that ω>0 for the above formulation to be valid. 2.2. Continuous Wavelet Transformation (CWT) Electroencephalography (EEG) records the electrical activity of the brain, producing inherently non-stationary signals. Traditional Fourier methods, which analyze the signals in terms of sinusoids with infinite duration, may not effectively capture the transient or time-varying phenomena of the EEG data [24]. On the other hand, wavelet transform is a computational method designed to analyze non-stationary signals by decomposing them into various frequency components while maintaining temporal resolution. The wavelet transform employs basic functions called “wavelets”, allowing simultaneous frequency and time domain analysis [25,26]. Equation (3) is a mathematical expression for the wavelet transform. where f(τ) is the input signal; ψ*(⋅) represents the complex conjugate of the wavelet function; s is the scale factor (which is inversely related to frequency); and t is the translation factor (related to time). Extending this concept, the CWT is a specialized form of wavelet transform wherein the wavelet undergoes continuous scaling and translation, allowing temporal and spectral analysis [27]. CWT’s multi-resolution characteristic is particularly advantageous for interpreting EEG signals, given that different physiological phenomena might present themselves at diverse scales. The expression for CWT of a function f(t) relative to a wavelet ψ(t) is as follows: with the modified wavelet given by the following: ψ is called the mother wavelet, which is a short wave-like oscillation. ψ is called the mother wavelet, which is a short wave-like oscillation. s is the scaling factor. The function is stretched if s>1 or compressed if 01 or compressed if 0