SeizureTransformer: Scaling U-Net with Transformer for Simultaneous Time-Step Level Seizure Detection from Long EEG Recordings
1 st Kerui Wu, 2 nd Ziyue Zhao, 3 rd Bülent Yener · 2025
Read SeizureTransformer: Scaling U-Net with Transformer for Simultaneous Time-Step Level Seizure Detection from Long EEG Recordings by 1 st Kerui Wu, 2 nd Ziyue Zhao, 3 rd Bülent Yener (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-00336.txt.
Introduction
Kerui Wu, Ziyue Zhao, and Bulent Yener built in 2025 a large U-shaped transformer model that spots seizures in brain wave records at each time step. Brain waves reflect the brain state and serve many detection tasks. Older models are small and modest, while large models need long, costly pre-training.
Their model mixes local filters with self-focus layers to catch both short and long patterns. It outputs a label for each time step directly, with no messy post-steps. It can also sort whole windows with one added layer. Scaled up, it beat large pre-trained rivals using only the task data for tuning. It won first place in a 2025 seizure detection contest.
Catalog record
| CREATOR | 1 st Kerui Wu, 2 nd Ziyue Zhao, 3 rd Bülent Yener |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-00336
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SeizureTransformer: Scaling U-Net with Transformer for Simultaneous Time-Step Level Seizure Detection from Long EEG Recordings 1 st Kerui Wu, 2 nd Ziyue Zhao, 3 rd Bülent Yener Abstract Epilepsy is a common neurological disorder that affects around 65 million people worldwide. Detecting seizures quickly and accurately is vital, given the prevalence and severity of the associated complications. Recently, deep learning-based automated seizure detection methods have emerged as solutions; however, most existing methods require extensive post-processing and do not effectively handle the crucial long-range patterns in EEG data. In this work, we propose SeizureTransformer, a simple model comprised of (i) a deep encoder comprising 1D convolutions (ii) a residual CNN stack and a transformer encoder to embed previous output into high-level representation with contextual information, and (iii) streamlined decoder which converts these features into a sequence of probabilities, directly indicating the presence or absence of seizures at every time step. Extensive experiments on public and private EEG seizure detection datasets demonstrate that our model significantly outperforms existing approaches (ranked in the first place in the 2025 ”seizure detection challenge” organized in the International Conference on Artificial Intelligence in Epilepsy and Other Neurological Disorders), underscoring its potential for real-time, precise seizure detection. I Introduction Epilepsy is a prevalent neurological disorder distinguished by recurring seizures. Worldwide, there are approximately 65 million people with epilepsy, more than Parkinson’s disease, Alzheimer’s disease, and Multiple Sclerosis combined. One of the most serious complications linked to epilepsy is Sudden Unexpected Death in Epilepsy(SUDEP), which tragically results in the deaths of around 1 in every 1000 epilepsy patients each year [ 1 ] . Given the severity of this risk, early and precise seizure detection is crucial in clinical practice, as prompt intervention can considerably lower mortality rates [ 2 ] . Traditionally, large numbers of multi-channel EEG signals are visually analyzed by neurologists with the goal of understanding when and where the seizures start and how they propagate within the brain. However, there are two main disadvantages of visual analysis of EEG signals: it is time-consuming and prone to subjectivity. Therefore, automation of the detection of the underlying brain dynamics in EEG signals is significant in order to obtain fast and objective EEG analysis. EEG signals can be treated as a batch of time series, a sequence of data points indexed in a discrete-time order, which formulates the automated seizure detection problem to be part of a classification task in time series analysis. In recent years, deep learning models have demonstrated impressive abilities to capture the intricate dependencies within time series data, making them a powerful tool for time series analysis over traditional statistical methods. However, most existing work [ 3 , 4 , 5 , 6 , 7 , 8 , 9 ] implements the classification task at a sliding window level, which involves segmenting a signal recording into distinct windows and predicting a label for each sample. Converting separated predictions into final event prediction in Standardized Computer-based Organized Reporting of EEG (SCORE) standard [ 10 ] that can be used in real life involves extensive time-consuming post-processing, which departs existing algorithms from simultaneous detection. More than that, existing time series analysis research often train and evaluate models using datasets that have a small sequence length [ 11 ] , while EEG studies haven shown that long-range input records can largely benefit accurate prediction [ 12 ] . In contrast to window-level classification models, sequence-to-sequence modeling, a type of encoder-decoder model to map an input sequence to an output sequence, provides a straightforward solution to avoid redundant post-processing steps through time-step-level classification. In the filed of Natural Language Processing(NLP), Transformer-based models have shown remarkable predictive and generative abilities [ 13 , 14 ] . However, studies have shown that CNN-based models achieve better classification ability in time series analysis compared to RNN-based and Transformer-based models [ 15 ] . This lets the focus of scientific signal classification study be on the U-Net [ 16 , 12 , 17 ] , a fully convolutional encoder-decoder network with skip connections that was originally designed for image segmentation [ 18 ] . The drawback of such models also stands out. Firstly, U-Net primarily operates within local receptive fields, making it difficult for U-Net to effectively model long-range dependencies as the input sequence length becomes big. Beyond that, Scaling U-Net to large datasets or high-resolution sequences requires stacking deeper layers, often leading to vanishing gradients, overfitting, and massive memory and computation usage. In this work, we propose a simple U-Net-based architecture, namely, SeizureTransformer, to solve the mentioned challenges. The model comprises of three components (i) a deep encoder comprising 1D convolutions (ii) a residual CNN stack and a transformer encoder to embed previous output into high-level representation with global contextual information, and (iii) streamlined decoder which converts these features into a sequence of probabilities, directly indicating the presence or absence of seizures at every time step. The scaling embedding components makes the model to be easily scalable to build up the model size and to handle long-sequence signals. Experimentally, our model achieves the consistent state-of-the-art performance, efficiency, and generalization across diverse subjects and devices in public and private EEG datasets. Our model has ranked number one in an international competition organized by the International Conference on Artificial Intelligence in Epilepsy and Other Neurological Disorders . Fig. 1: SeizureTransformer Architecture. II Results II-A Model Overview We design model architecture based on the U-Net to do end-to-end learning from raw waveforms for time-step-level classification to achieve simultaneous seizure detection. Our model consists of three primary modules: an encoder, a scaling embedding component, and a decoder, as shown in Fig. 1 . Taking the continuous long-term EEG signals from the epilepsy monitoring unit, the encoder extracts features by recognizing patterns through one-dimensional convolution layers. The feature vectors are further embedded by a ResCNN stack and a Transformer encoder stack with a global attention mechanism to generate high-level representations that capture rich temporal dependencies. The streamlined decoder then converts these representations into a sequence of probability, indicating the presence or absence of seizures at every time step. Residual connections between each encoder layer and decoder layer are used to ease the gradient flow and to avoid degradation problems in the deep neural network. More details about network architecture selection are provided in the methodology section. II-B Model Training Datasets. We use Temple University Hospital EEG Seizure Corpus v2.0.3(TUSZ) [ 19 ] and Siena Scalp EEG Database [ 20 ] to form our training dataset. TUSZ is the largest public dataset for seizure detection that has been manually annotated with data for seizure events. The predefined training set in TUSZ has 910 hours of recording sessions from 579 subjects with various sampling frequencies, from 250 Hz to 1000 Hz. The Siena Scalp EEG Database is a small dataset that contains 128 hours of recording sessions from 14 subjects with a unified sampling rate of 512 Hz. Both datasets contain at least 19 electrodes of the international 10-20 system. We unify the training data from two datasets by resampling signals into 256 Hz and fixing the channel sequence in order (Fig. 2 a). We combine two datasets by concatenating segmented one-minute-long time series windows together, i.e.,