Standing on the Shoulders of Giants: Rethinking EEG Foundation Model Pretraining via Multi-Teacher Distillation
Chenqi Li, Yu Liu, Shuo Zhang, Timothy Denison, Tingting Zhu · 2026
Read Standing on the Shoulders of Giants: Rethinking EEG Foundation Model Pretraining via Multi-Teacher Distillation by Chenqi Li, Yu Liu, Shuo Zhang, Timothy Denison, Tingting Zhu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-04478.txt.
Description
Li and colleagues pretrain EEG foundation models by distilling many teachers with MTDP. Masked reconstruction, borrowed from vision and language, struggles on EEG, where data cost dearly and signal to noise runs low. Vision and time series foundation models transfer surprisingly well to EEG, the authors first show. MTDP fuses diverse teacher representations through a learnable gating network and masked latent denoising, then distills the fused representation into an EEG model. Across 9 downstream tasks and 12 datasets, the MTDP model beats self supervised counterparts while using only 25 percent of pretraining data.
Catalog record
| CREATOR | Chenqi Li, Yu Liu, Shuo Zhang, Timothy Denison, Tingting Zhu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46337 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.04478