MultiDiffNet: A Multi-Objective Diffusion Framework for Generalizable Brain Decoding
Mengchun Zhang, Kateryna Shapovalenko, Yucheng Shao, Eddie Guo, Parusha Pradhan · 2025
Read MultiDiffNet: A Multi-Objective Diffusion Framework for Generalizable Brain Decoding by Mengchun Zhang, Kateryna Shapovalenko, Yucheng Shao, Eddie Guo, Parusha Pradhan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-18294.txt.
Introduction
Mengchun Zhang, Kateryna Shapovalenko, Yucheng Shao, Eddie Guo, and Parusha Pradhan decode brain signals for people the model has never seen. Reading EEG, the scalp signal of brain activity, fails most when a new subject's patterns differ from the training group. Past fixes fake new subjects or pad data in plain ways, and neither scales well.
MultiDiffNet skips fake data growth. It learns a compact code space tuned for many goals at once, and decodes straight from that space. Scores top prior work across tasks when test subjects and sessions are held apart from training. The team also release a shared test suite of four EEG tasks, from steady visual response to imagined speech, with a clean split protocol.
Catalog record
| CREATOR | Mengchun Zhang, Kateryna Shapovalenko, Yucheng Shao, Eddie Guo, Parusha Pradhan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 162 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-18294
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MultiDiffNet: A Multi-Objective Diffusion Framework for Generalizable Brain Decoding Mengchun Zhang, Kateryna Shapovalenko, Yucheng Shao, Eddie Guo, Parusha Pradhan Abstract Neural decoding from electroencephalography (EEG) remains fundamentally limited by poor generalization to unseen subjects, driven by high inter-subject variability and the lack of large-scale datasets to model it effectively. Existing methods often rely on synthetic subject generation or simplistic data augmentation, but these strategies fail to scale or generalize reliably. We introduce MultiDiffNet , a diffusion-based framework that bypasses generative augmentation entirely by learning a compact latent space optimized for multiple objectives. We decode directly from this space and achieve state-of-the-art generalization across various neural decoding tasks using subject and session disjoint evaluation. We also curate and release a unified benchmark suite spanning four EEG decoding tasks of increasing complexity (SSVEP, Motor Imagery, P300, and Imagined Speech) and an evaluation protocol that addresses inconsistent split practices in prior EEG research. Finally, we develop a statistical reporting framework tailored for low-trial EEG settings. Our work provides a reproducible and open-source foundation for subject-agnostic EEG decoding in real-world BCI systems. 1 Introduction Electroencephalography (EEG) is a widely used modality in brain–computer interfaces (BCIs), supporting applications from assistive communication to cognitive monitoring. Deep learning has improved decoding across motor imagery, SSVEP, and speech tasks [ 10 , 1 , 23 ] , yet generalizing to unseen subjects remains challenging due to high inter-subject variability and limited data [ 14 , 4 ] . Subject-specific models require extensive per-user calibration [ 11 , 27 ] , while multi-subject models struggle to generalize [ 31 , 25 , 41 ] . The alternative is to use two-stage pipelines that generate EEG via GANs or diffusion and then train decoders [ 11 , 37 ] , but they suffer from low realism, artifact transfer, and inefficiencies. We propose MultiDiffNet , a unified multi-objective diffusion framework that learns a shared latent space, eliminating the need for synthetic augmentation and enhancing generalization. To benchmark progress, we release a curated suite spanning SSVEP, Motor Imagery, P300, and Imagined Speech tasks, with standardized subject- and session-disjoint evaluation. We also develop a statistical reporting protocol tailored for low-trial EEG research, addressing a persistent gap in reproducibility. Figure 1 : Overview of the MultiDiffNet that jointly optimizes a conditional DDPM, a contrastive encoder, and a generative decoder through a shared latent space z . The encoder produces discriminative features used for both classification and contrastive learning, while the decoder and DDPM reconstruct the input signal. An optional temporal masked mixup module stochastically blends the original, DDPM-denoised, and decoder-reconstructed EEG to improve representation quality. 2 Related work EEG Decoding and Generalization EEG decoding has evolved from handcrafted features to deep architectures, with EEGNet emerging as a widely adopted baseline due to its efficient depthwise–separable convolutions and lightweight design [ 22 , 44 ] . Recent models explore transformers [ 3 , 24 , 33 ] and graph neural networks [ 35 , 13 ] , but EEGNet remains favored for its robustness and simplicity. A key limitation is poor cross-subject generalization, with 20-40% accuracy drops despite strong within-subject performance [ 14 , 4 ] . Attempts to address this require expensive calibration [ 31 , 25 , 41 ] . Scalable BCIs require subject-agnostic models that generalize without per-user retraining. Diffusion Models for EEG Denoising Diffusion Probabilistic Models (DDPMs) model data distributions via iterative denoising and outperform GANs in EEG synthesis by avoiding mode collapse [ 38 , 12 ] . Recent enhancements, such as reinforcement learning [ 2 ] and progressive distillation [ 37 ] , have further improved realism and sampling speed. Diff-E [ 17 ] extended diffusion to imagined-speech decoding via joint reconstruction and classification, but remained task-specific and did not address cross-subject generalization. Broader research suggests that combining generative and discriminative objectives yields stronger representations [ 7 , 9 ] , yet EEG models typically optimize only one. We explore this joint learning paradigm across diverse EEG tasks, aiming to learn generalizable representations that capture both signal structure and task-relevant information. Mixup Methods Signal-level augmentation has evolved from basic jittering and filtering to temporal, spectral, and channel-wise mixup [ 28 , 26 , 16 , 30 , 43 ] , but many variants introduce unrealistic artifacts that hinder generalization [ 5 ] . This motivates our systematic evaluation of weighted and temporal input mixup across encoder layers, along with latent‑space mixing Evaluation Strategies Effective cross-subject EEG decoding requires both rigorous training strategies and standardized evaluation. Leave-one-subject-out (LOSO) validation remains common but is computationally intensive and impractical for real-time deployment [ 8 , 6 , 46 , 4 , 21 ] , while simpler subject splits often neglect session independence and true seen/unseen separation [ 42 ] . We address it in our work by introducing a standardized subject- and session-disjoint evaluation. 3 Methodology 3.1 MultiDiffNet architecture MultiDiffNet is a modular architecture designed to jointly optimize classification, reconstruction, and contrastive structure learning from EEG signals. It consists of a Denoising Diffusion Probabilistic Model (DDPM), a discriminative encoder, a generative decoder, and a classifier (Figure 1 ). Given a raw EEG signal x\in\mathbb{R}^{C\times T} , where C is the number of EEG channels and T is the number of timepoints, the model processes the input in two parallel paths. First, the DDPM denoises the signal via a learned reverse diffusion process, producing a refined version \hat{x}\in\mathbb{R}^{C\times T} . Simultaneously, the same input x is passed through an EEGNet-based encoder (See Section 3.2 ) to extract a latent representation z\in\mathbb{R}^{D} , where D is the embedding dimension. The latent vector z is then used for two purposes: (1) it is passed to a lightweight decoder to reconstruct the denoised signal \hat{x} , resulting in a reconstruction x_{\text{dec}}\in\mathbb{R}^{C\times T} ; and (2) it is passed to a fully connected classification head to predict class logits \hat{y}\in\mathbb{R}^{K} , where K is the number of classes. To further structure the latent space, z is locally normalized (Section 3.3 ) and then projected to z_{\text{proj}}\in\mathbb{R}^{D^{\prime}} , which is optimized with a supervised contrastive loss. All classification and reconstruction are performed directly from z , without relying on generated augmentations. We performed an extensive ablation study across architectural variants, modifying the presence of DDPMs, encoder inputs, decoder pathways, classifier heads, and loss terms. The configuration described here reflects the best-performing combination. 3.2 EEGNet-style encoder with attention pool Given EEGNet’s demonstrated effectiveness across multiple EEG decoding tasks, we adapt its architecture as our discriminative encoder, hypothesizing that its proven feature extraction capabilities can produce powerful latent representations z for our multi-objective framework. Our encoder extracts multi-scale features (dn_{1},dn_{2},dn_{3}) from different layers and applies attention pooling: z=\text{AttentionPool}(dn_{3})\in\mathbb{R}^{D}, 3.3 Subject-wise latent normalization To mitigate inter-subject variability, we apply subject-wise normalization on the encoder output z : z_{\text{norm}}=\frac{z-\mu_{s}}{\sigma_{s}}, where \mu_{s} and \sigma_{s} denote the mean and standard deviation computed per subject s using a subset of training trials. During evaluation, we adopt a two-mode strategy: for seen subjects, normalization uses pre-computed statistics from their training data; for unseen subjects, statistics are estimated on-the-fly using their own calibration trials, simulating realistic deployment scenarios. Algorithm 1 Temporal Masked Mixup 1: Initialize a binary mask M\in\{0,1\}^{C\times T} with all zeros. 2: Flip each 0