MEGConformer: Conformer-Based MEG Decoder for Robust Speech and Phoneme Classification
Xabier de Zuazo, Ibon Saratxaga, Eva Navas, HiTZ Center, Dept. of Communications Engineering, School of Engineering · 2025
Read MEGConformer: Conformer-Based MEG Decoder for Robust Speech and Phoneme Classification by Xabier de Zuazo, Ibon Saratxaga, Eva Navas, HiTZ Center, Dept. of Communications Engineering, School of Engineering (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01443.txt.
Introduction
Xabier de Zuazo and co-authors decode brain waves for speech tasks. They used MEG, which tracks magnetic fields from the brain. Their entry targets a 2025 contest on the LibriBrain data set. Two tasks count: spotting speech and sorting speech sounds.
Their decoder is based on the Conformer model. It reads raw signals from hundreds of sensors at once. A light front layer shapes the input for each task. A form of data warping made for MEG was also tried. Strong decoders could one day aid speech aids for users who cannot speak.
Catalog record
| CREATOR | Xabier de Zuazo, Ibon Saratxaga, Eva Navas, HiTZ Center, Dept. of Communications Engineering, School of Engineering |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-01443
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MEGConformer: Conformer-Based MEG Decoder for Robust Speech and Phoneme Classification Xabier de Zuazo, Ibon Saratxaga, Eva Navas, HiTZ Center, Dept. of Communications Engineering, School of Engineering Abstract We present Conformer-based decoders for the LibriBrain 2025 PNPL competition, targeting two foundational MEG tasks: Speech Detection and Phoneme Classification. Our approach adapts a compact Conformer to raw 306-channel MEG signals, with a lightweight convolutional projection layer and task-specific heads. For Speech Detection, a MEG-oriented SpecAugment provided a first exploration of MEG-specific augmentation. For Phoneme Classification, we used inverse-square-root class weighting and a dynamic grouping loader to handle 100-sample averaged examples. In addition, a simple instance-level normalization proved critical to mitigate distribution shifts on the holdout split. Using the official Standard track splits and F1-macro for model selection, our best systems achieved 88.9% (Speech) and 65.8% (Phoneme) on the leaderboard, surpassing the competition baselines and ranking within the top-10 in both tasks. For further implementation details, the technical documentation, source code, and checkpoints are available at https://github.com/neural2speech/libribrain-experiments . 1 Introduction In the emerging fields of deep learning and neuroscience, decoding speech from non-invasive brain signals remains one of the central problems in neural signal processing and brain-computer interfaces (BCIs) 27 ; 11 ; 25 . The recently released LibriBrain dataset 26 and its associated PNPL 2025 Competition 18 provide a unique opportunity to address this challenge by enabling large-scale within-subject modeling of magnetoencephalography (MEG) recordings. LibriBrain contains over 50 hours of continuous MEG data from a single participant listening to the complete Sherlock Holmes corpus, accompanied by precise voice activity and phoneme-level alignments. The competition defines two foundational decoding tasks: Speech Detection, which distinguishes between speech and silence, and Phoneme Classification, which predicts the phoneme being perceived from averaged MEG signals. Both tasks include a Standard and an Extended track, where the Standard track restricts participants to using only the official training data. We participated exclusively in the Standard track. Our approach explores an initial adaptation of state-of-the-art Automatic Speech Recognition (ASR) architectures to non-invasive neural signals. Specifically, we leverage the Conformer model 12 , a convolution-augmented Transformer 33 that combines the global context modeling of self-attention 20 with the local feature extraction strength of convolutional networks 19 . Conformer-based architectures have been and continue to be the state-of-the-art in the ASR field 4 ; 15 ; 29 ; 14 ; 30 . We hypothesize that their ability to capture both temporal and spectral dependencies may serve as a good starting point for MEG-based speech tasks, which also exhibit structured spatiotemporal patterns over multiple timescales. In this work, we evaluate a Conformer model adapted to the LibriBrain competition tasks. The models process raw MEG windows to predict either speech activity or phoneme identity, sharing the same backbone while differing in input and output processing. To ensure fair comparison and reproducibility, all experiments follow the official data splits and evaluation metrics provided by the organizers 18 . Our main contributions are fourfold: (i) a unified Conformer architecture jointly optimized for both LibriBrain tasks; (ii) a robust yet straightforward input normalization method that substantially improves holdout generalization; (iii) an effective MEG-specific augmentation; (iv) and a dynamic grouping strategy to classify averaged samples in the Phoneme Task. This study demonstrates that adapting modern ASR architectures to MEG decoding yields competitive and robust results in both voice activity detection and phoneme recognition, emphasizing the growing convergence between speech processing and neural decoding research. 2 Related work Decoding speech from non-invasive neural signals has been explored with both EEG and MEG using supervised learning, from linear baselines to deep architectures. EEG studies reconstruct speech features or spectrograms with CNN/Transformer-style models and subject-independent training, showing steady gains with more data 1 ; 35 . On MEG, prior work demonstrated phrase and word-level decoding and trial-efficiency using wavelet denoising, CNNs, and transfer learning 6 ; 7 ; 5 , as well as compact end-to-end networks adapted to sensor time series 31 , and transformer-based neural encoding that links linguistic context to MEG responses 16 . Other studies have also started to explore phoneme-level decoding in perception and production modalities, comparing traditional and novel model architectures 8 ; 9 . More recently, LibriBrain established an unprecedented within-subject scale and standardized tasks for speech detection and phoneme classification 26 ; 18 . Beyond purely supervised setups, contrastive and self-supervised objectives, along with foundation-model guidance, have advanced non-invasive decoding across perception domains, improving retrieval and generation 10 ; 3 ; 2 . Recent work on non-invasive brain-to-text systems combines discriminative decoders with language-model rescoring and cross-dataset scaling to produce significant score improvements 13 . 3 Methods 3.1 Model We use a single Conformer encoder adapted to MEG for both tracks (Speech Detection and Phoneme Classification). The network takes raw sensor time series (306 channels) preprocessed with the official pipeline (bad-channel interpolation, head-position correction, signal-space separation, notch, and band-pass filtering), where windowed signal segments are downsampled to 250\text{\,}\mathrm{H}\mathrm{z} . For Speech, we use 2.5\text{\,}\mathrm{s} windows (625 samples), and for the Phoneme Task, 0.5\text{\,}\mathrm{s} windows (125 samples). A lightweight 1D convolutional projection adapts the 306 MEG channels to the Conformer input size (144), followed by a dropout of p{=}0.1 , a stack of Conformer blocks, and a linear classifier. Each Conformer block follows the standard macaron layout: feed-forward, multi-head self-attention, depthwise temporal convolution, and a second feed-forward, with residual connections. We keep the design compact with a hidden size of 144. For Speech, based on Conformer Small, we use 16 layers, 4 heads, a hidden layer dimension of FFNs of 576, and the layer’s depthwise convolution layer with a kernel size of 31. For Phonemes, we created a custom-sized Conformer with 7 layers, 12 heads, FFN dimension of 2048, and a kernel size of 127, to better adapt the model to the smaller dataset. 3.2 Speech detection task specifics Figure 1: Conformer-based model architecture used for Speech Task. The Speech model is a binary classifier with a single-logit head, binary cross-entropy loss, and loss-level label smoothing of 0.1 24 . We found that simple, speech-style augmentation was sufficient for this task. Therefore, we develop and apply a light MEG-specific variant of SpecAugment 28 , which we introduce as MEGAugment . It includes two operations: (i) time masking , which zeroes two random temporal spans (max width T{=}180 samples at 250\text{\,}\mathrm{H}\mathrm{z} ) per window; and (ii) bandstop masking , which randomly suppresses narrow frequency bands (theta, alpha, beta, gamma, high-gamma 22 ) using fourth-order infinite impulse response (IIR) notches with probability p{=}0.4 . Moreover, input windows slide with a 60-sample stride during training to increase diversity via overlapping. The final layer outputs a probability of speech, which is smoothed during inference by removing speech sequences shorter than 60 samples ( 240\text{\,}\mathrm{m}\mathrm{s} ). The block diagram in Figure 1 summarizes the pipeline. 3.3 Phoneme classification task specifics The Phoneme model uses a linear classification head that outputs 39 logits and is trained with cross-entropy. At inference, we select the class via argmax . To mitigate imbalance in the Phoneme Task, we use class weights w_{c} with the Inverse of Square root of Number of Samples (ISNS) rule following Equation 1 , where n_{c} is the count of class c in the training set and C is the