Standing on the Shoulders of Giants: Rethinking EEG Foundation Model Pretraining via Multi-Teacher Distillation

Chenqi Li, Yu Liu, Shuo Zhang, Timothy Denison, Tingting Zhu · 2026

Read Standing on the Shoulders of Giants: Rethinking EEG Foundation Model Pretraining via Multi-Teacher Distillation by Chenqi Li, Yu Liu, Shuo Zhang, Timothy Denison, Tingting Zhu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-04478.txt.

Description

Li and colleagues pretrain EEG foundation models by distilling many teachers with MTDP. Masked reconstruction, borrowed from vision and language, struggles on EEG, where data cost dearly and signal to noise runs low. Vision and time series foundation models transfer surprisingly well to EEG, the authors first show. MTDP fuses diverse teacher representations through a learnable gating network and masked latent denoising, then distills the fused representation into an EEG model. Across 9 downstream tasks and 12 datasets, the MTDP model beats self supervised counterparts while using only 25 percent of pretraining data.

Catalog record

CREATORChenqi Li, Yu Liu, Shuo Zhang, Timothy Denison, Tingting Zhu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46337
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.04478

Browse the full library · About the Tower of Babel