Identification of Epileptic Spasms (ESES) Phases Using EEG Signals: A Vision Transformer Approach Conference: Undergraduate Consortium; Dec, 2024; Zhuhai, Guangdong Price: 15.00 ISBN: 978-1-4503-XXXX-X/23/08 CCS: General and reference Empirical studies CCS: Computing methodologies Neural networks CC
Wei Gong, Yaru Li · 2024
Read Identification of Epileptic Spasms (ESES) Phases Using EEG Signals: A Vision Transformer Approach Conference: Undergraduate Consortium; Dec, 2024; Zhuhai, Guangdong Price: 15.00 ISBN: 978-1-4503-XXXX-X/23/08 CCS: General and reference Empirical studies CCS: Computing methodologies Neural networks CC by Wei Gong, Yaru Li (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-13028.txt.
Introduction
Wei Gong and Yaru Li detect a sleep-related epilepsy pattern in brainwave recordings using a vision transformer, in a 2024 paper. The pattern, tied to epileptic spasms, was traditionally spotted by hand or with shallow algorithms on single channels, which generalized poorly and used small samples.
Their model treats spectrograms of multi-channel EEG as images and lets attention pick out the telling features across channels. Working in the frequency domain exposes long-range structure that raw traces hide. The system reported ninety-seven percent accuracy without heavy preprocessing. That lightness matters for real-time, large-scale clinical use. The authors present the approach as a significant step for detecting this disorder and kindred neurological conditions. Attention over spectrograms, they argue, suits EEG's mix of rhythm and burst better than the older pipelines.
Catalog record
| CREATOR | Wei Gong, Yaru Li |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-13028
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Identification of Epileptic Spasms (ESES) Phases Using EEG Signals: A Vision Transformer Approach Conference: Undergraduate Consortium; Dec, 2024; Zhuhai, Guangdong Price: 15.00 ISBN: 978-1-4503-XXXX-X/23/08 CCS: General and reference Empirical studies CCS: Computing methodologies Neural networks CCS: Computing methodologies Transfer learning CCS: Computing methodologies Machine learning algorithms CCS: Applied computing Bioinformatics CCS: Computing methodologies Knowledge representation and reasoning Wei Gong, Yaru Li Abstract . This work introduces a new approach to the Epileptic Spasms (ESES) detection based on the EEG signals using Vision Transformers (ViT). Classic ESES detection approaches have usually been performed with manual processing or conventional algorithms, suffering from poor sample sizes, single-channel-based analyses, and low generalization abilities. In contrast, the proposed ViT model overcomes these limitations by using the attention mechanism to focus on the important features in multi-channel EEG data, which is contributing to both better accuracy and efficiency. The model processes frequency-domain representations of EEG signals, such as spectrograms, as image data to capture long-range dependencies and complex patterns in the signal. The model demonstrates high performance with an accuracy of 97% without requiring intensive data preprocessing, thus rendering it suitable for real-time clinical applications on a large scale. The method represents a significant development in the advancement of neurological disorders such as ESES in detection and analysis. Introduction Electroencephalogram (EEG) signals are crucial in diagnosing and analyzing neurological disorders, such as Epileptic Spasms (ESES), which are characterized by continuous neuronal discharges during sleep. EEG data, often represented as time series signals, capture intricate patterns of brain activity. However, the complexity and high-dimensional nature of EEG data make it difficult to analyze effectively, requiring sophisticated models that can interpret these complex signals accurately. Traditionally, various machine learning techniques, such as Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks, have been employed for EEG classification tasks, including ESES detection. These methods, while effective, have certain limitations that hinder their performance in real-world applications. CNNs, a popular method for processing spatial data, are well suited for image classification tasks. However, when applied to EEG data, CNNs are often limited by the inability to capture long-range temporal dependencies and the complex, multidimensional patterns inherent in the data. Additionally, CNNs typically require extensive data pre-processing, which can lead to a loss of important signal information. On the other hand, LSTMs, which excel at handling sequential data, can capture long-range dependencies in time-series data. However, they may struggle with high-dimensional EEG data due to the sequential nature of the model, which can limit their ability to efficiently process large-scale datasets. To address these challenges, this paper proposes a novel approach to ESES detection by converting EEG data into image representations and using Vision Transformers (ViT) for classification. Vision Transformers have demonstrated remarkable success in handling complex image data by leveraging self-attention mechanisms, allowing the model to focus on relevant spatial and temporal parts of the signal, making it highly suitable for EEG data analysis ( dosovitskiy2020transformers ) . In our method, we first convert the frequency-domain data into grayscale and RGB images. Transformation of EEG signals into images allows us to take advantage of the power of ViT, a model originally designed for image classification, to handle the high-dimensional data and capture the complex patterns present in EEG signals ( dosovitskiy2020transformers ) . Unlike CNNs and LSTMs, ViTs excel at processing long-range dependencies and capturing intricate relationships within the data using self-attention mechanisms, enabling the model to focus on relevant parts of the signal and enhance classification accuracy. The benefits of using ViT for EEG classification are many. First, the conversion of EEG data into grayscale and RGB images provides a more precise and efficient representation of the signal, reducing the need for extensive manual pre-processing. Second, ViT’s ability to process long-range dependencies within the data allows for more accurate detection of ESES events. The self-attention mechanism in ViT enables the model to capture subtle, distributed patterns across time and frequency bands, which is critical for diagnosing conditions like ESES ( parmar2018image ) . Finally, the ViT model’s ability to handle large datasets efficiently and quickly makes it particularly well suited for real-time clinical applications, offering a promising approach to improving diagnostic accuracy in neurological disorders. In summary, this work introduces a new approach to ESES detection that outperforms traditional methods by converting the frequency data into grayscale images and leveraging the power of Vision Transformers. By converting EEG data into image representations and utilizing ViT’s advanced self-attention mechanism, we provide a more accurate, efficient, and scalable solution for EEG-based diagnosis, setting our model apart from existing methods in the field. 1. Related Work Current Circumstance ESES (Encephalopathy with Status Epilepticus during Sleep) is characterized by continuous or near-continuous epileptiform activity during sleep, typically detected through both time-domain and frequency-domain EEG analysis. Various machine learning and signal processing techniques are currently being utilized to identify ESES patterns in EEG data. A summary of some of the more prominent algorithms and models in use today is as follows: Classic Signal Processing and Machine Learning Before the advent of deep learning models, detection of ESES was primarily performed using classic signal processing techniques combined with machine learning classifiers. These methods include: • Fourier Transform : The Fourier Transform is used to transform EEG signals into the frequency domain, allowing for analysis of specific frequency bands such as delta, theta, alpha, beta, and gamma waves ( fourier2009 ) . • Wavelet Transform : This method is typically employed to detect non-stationary signals, such as seizures and other epileptic activities, due to its ability to analyze both the time and frequency domains simultaneously ( daubechies1992 ) . • Spectrograms : Spectrograms transform EEG signals into time-frequency representations using techniques such as Short-Time Fourier Transform (STFT), which are then classified using machine learning classifiers such as Support Vector Machines (SVM), Random Forests, or k-Nearest Neighbors (K-NN) ( spectrogram2015 ) . Machine Learning Classifiers Once features are extracted using these transformations, classifiers such as Support Vector Machines (SVM), Random Forests, K-Nearest Neighbors (K-NN), or Artificial Neural Networks (ANNs) are commonly employed for detection ( svm1995 ; randomforest2001 ; knn2001 ) . Figure 1. EEG normal vs ESES.png Sample ID EEG Data (16 channels, 500 Hz) ESES Label 1 [X_{1},X_{2},\dots,X_{500}] 0 (N) 2 [X_{1},X_{2},\dots,X_{500}] 1 (Y) 3 [X_{1},X_{2},\dots,X_{500}] 0 (N) 4 [X_{1},X_{2},\dots,X_{500}] 1 (Y) 5 [X_{1},X_{2},\dots,X_{500}] 0 (N) ⋮ ⋮ ⋮ Table 1. EEG data sampled at 500 Hz with corresponding ESES labels (0 = No, 1 = Yes). Figure 2. The Data Pre-processing and the Model Architecture Signal-to-Image Preprocessing Figure 3. The Greyscale Image Formed By Original Signal The EEG data used in this study originates from a dataset containing both EEG signals and other physiological data. For this task, we specifically focus on the EEG data, which consists of 16 channels, each corresponding to a different electrode on the scalp. These 16 channels of EEG data are sampled at a frequency of 500 Hz per second, resulting in a matrix of dimensions 16\times 500 , where each column represents a time-point and each row corresponds to one of the 16 channels. Each EEG sample in the dataset is associated with a label that indicates the presence or absence of Encephalopathy with Status Epilepticus during