Beyond Hearing: Learning Task-agnostic ExG Representations from Earphones via Physiology-informed Tokenization
Hyungjun Yoon 1 Seungjoo Lee 2 Yu Yvonne Wu 3 Xiaomeng Chen 4 Taiting Lu 5 Freddy Yifei Liu 2 Taeckyung Lee 1 Hyeongheon Cha 1 Haochen Zhao 6 Gaoteng Zhao 7 Sung-Ju Lee 1 Cecilia Mascolo 3 Dongyao Chen 4 Lili Qiu 8,9 1 KAIST 2 Carnegie Mellon University 3 University of Cambridge 4 Shanghai Jiao Tong · 2025
Read Beyond Hearing: Learning Task-agnostic ExG Representations from Earphones via Physiology-informed Tokenization by Hyungjun Yoon 1 Seungjoo Lee 2 Yu Yvonne Wu 3 Xiaomeng Chen 4 Taiting Lu 5 Freddy Yifei Liu 2 Taeckyung Lee 1 Hyeongheon Cha 1 Haochen Zhao 6 Gaoteng Zhao 7 Sung-Ju Lee 1 Cecilia Mascolo 3 Dongyao Chen 4 Lili Qiu 8,9 1 KAIST 2 Carnegie Mellon University 3 University of Cambridge 4 Shanghai Jiao Tong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20853.txt.
Introduction
Hyungjun Yoon leads thirteen co-authors on learning general body signal representations from earphones worn in daily life. Electrical body signals reveal much about health, yet most recordings come from bulky lab gear, and most models are built for one task with narrow filters.
The team gathered fifty hours of free living recordings with an earphone prototype and built DailySense, a data set spanning five human senses. Their token method splits signals into twelve bands guided by physiology, then learns by reconstruction so features adapt across the full spectrum. Tests on DailySense and four public sets show broad transfer, a step toward quiet everyday health sensing.
Catalog record
| CREATOR | Hyungjun Yoon 1 Seungjoo Lee 2 Yu Yvonne Wu 3 Xiaomeng Chen 4 Taiting Lu 5 Freddy Yifei Liu 2 Taeckyung Lee 1 Hyeongheon Cha 1 Haochen Zhao 6 Gaoteng Zhao 7 Sung-Ju Lee 1 Cecilia Mascolo 3 Dongyao Chen 4 Lili Qiu 8,9 1 KAIST 2 Carnegie Mellon University 3 University of Cambridge 4 Shanghai Jiao Tong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-20853
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Hearing: Learning Task-agnostic ExG Representations from Earphones via Physiology-informed Tokenization Hyungjun Yoon 1 Seungjoo Lee 2 Yu Yvonne Wu 3 Xiaomeng Chen 4 Taiting Lu 5 Freddy Yifei Liu 2 Taeckyung Lee 1 Hyeongheon Cha 1 Haochen Zhao 6 Gaoteng Zhao 7 Sung-Ju Lee 1 Cecilia Mascolo 3 Dongyao Chen 4 Lili Qiu 8,9 1 KAIST 2 Carnegie Mellon University 3 University of Cambridge 4 Shanghai Jiao Tong University 5 Pennsylvania State University 6 UCLA 7 Northwest University 8 University of Texas at Austin 9 Microsoft Research Abstract Electrophysiological (ExG) signals offer valuable insights into human physiology, yet building foundation models that generalize across everyday tasks remains challenging due to two key limitations: (i) insufficient data diversity, as most ExG recordings are collected in controlled labs with bulky, expensive devices; and (ii) task-specific model designs that require tailored processing ( i.e. , targeted frequency filters) and architectures, which limit generalization across tasks. To address these challenges, we introduce an approach for scalable, task-agnostic ExG monitoring in the wild. We collected 50 hours of unobtrusive free-living ExG data with an earphone-based hardware prototype to narrow the data diversity gap. At the core of our approach is Physiology-informed Multi-band Tokenization (PiMT) , which decomposes ExG signals into 12 physiology-informed tokens, followed by a reconstruction task to learn robust representations. This enables adaptive feature recognition across the full frequency spectrum while capturing task-relevant information. Experiments on our new DailySense dataset—the first to enable ExG-based analysis across five human senses—together with four public ExG benchmarks, demonstrate that PiMT consistently outperforms state-of-the-art methods across diverse tasks. 1 Introduction Electrophysiological (ExG) signals, including electroencephalography (EEG), electromyography (EMG), electrooculography (EOG), and electrocardiography (ECG), provide critical insights into neural, muscular, ocular, and cardiovascular activities. They enable a wide range of physiological applications, from gaze tracking ( Merino et al. 2010 ) and emotion recognition ( Gkintoni et al. 2025 ) to sleep staging ( Nguyen et al. 2016 ) and seizure detection ( JW et al. 2016 ) . Recent advances in deep learning have improved ExG analysis by developing data-driven training approaches ( Song et al. 2022 ; Jiang et al. 2024 ) that capture complex temporal and spectral patterns for various physiological tasks. Building on this, foundation models, which have demonstrated remarkable success across domains by leveraging large-scale data to learn general-purpose representations ( Narayanswamy et al. 2025 ) , offer a promising opportunity for advancing everyday ExG analysis. However, ExG foundation models remain underexplored due to two limitations: (i) insufficient dataset diversity and (ii) task-specific model design. First, ExG datasets are typically collected in controlled environments ( Zheng & Lu 2015 ; Katsigiannis & Ramzan 2018 ; Wang et al. 2023 ) using bulky, expensive devices ( e.g. , EEG headsets ( Duvinage et al. 2013 ) ). This setup restricts both scale and diversity across tasks, leaving free-living ExG data largely untapped. Second, existing ExG models are highly task-specific, relying on tailored processing pipelines, i.e. , architectures optimized for a fixed frequency band, which limits their generalization. For example, gaze tracking methods are designed to capture low-frequency bands (0.1 \sim 15 Hz) ( Merino et al. 2010 ) , whereas emotion recognition relies on higher EEG bands (8 \sim 30 Hz) ( Gkintoni et al. 2025 ) . As a result, a model trained for gaze tracking cannot be directly applied to emotion recognition, highlighting the lack of transferability across tasks. To address the first challenge, we collected free-living ExG data in unobtrusive settings, constructing the DailySense dataset. For this, we prototyped NeuroBuds , an earphone-based ExG sensing device. Unlike traditional bulky systems, NeuroBuds is lightweight, low-cost, and portable while still capturing rich physiological signals: near-ear EEG, EMG from facial muscles, and EOG from eye movements. This design enables long-term data collection, overcoming the constraints of lab-based recordings. Leveraging this platform, we collected 50 hours of free-living ExG recordings from 22 participants engaged in unconstrained daily activities. Furthermore, we gathered 20 hours of targeted task-specific data spanning the five human senses ( i.e. , sight, hearing, taste, touch, and smell), establishing the first benchmark for evaluating model performance across diverse tasks. Moreover, we propose Physiology-informed Multi-band Tokenization (PiMT) , an approach designed to learn task-agnostic ExG representations. Instead of relying on a task-specific narrow band or a single wide-band input, PiMT decomposes ExG data into 12 fixed sub-band tokens, each corresponding to distinct physiological modalities. For instance, the [0.5 \sim 4 Hz] band captures to EEG delta waves, which are informative for sleep staging ( Elsaid & Labanowski 2017 ) , whereas the [15 \sim 45 Hz] band reflects low-frequency EMG activities, relevant for muscle activation and motor tasks ( Allison & Fujiwara 2002 ) . These structured tokens provide the encoder with fine-grained access to diverse spectral features, enabling the model to capture task-relevant information while remaining agnostic to any specific task. Coupled with self-supervised reconstruction objectives, we train a robust, transferable representations that generalize effectively across diverse downstream tasks. To evaluate our approach, we benchmark PiMT against the state-of-the-art ExG training approaches. Specifically, we evaluate it on our newly introduced DailySense benchmark, which spans tasks across the five human senses, along with four widely used datasets covering diverse ExG applications, including emotion recognition, sleep staging, and brain–computer interface (BCI) tasks. Extensive experiments demonstrate that PiMT achieves robust performance and strong generalization across both DailySense and public datasets. Our key contributions are as follows: • We identify the key limitations of existing ExG frameworks—insufficient dataset diversity and task-specific model design—that hinder generalization to real-world applications. • We introduce NeuroBuds, an earphone-based prototype for unobtrusive, long-term ExG monitoring. Leveraging NeuroBuds, we curate DailySense, a dataset containing 50 hours of free-living recordings and 20 hours of task-specific data spanning the five human senses. • We propose PiMT, a task-agnostic ExG training approach that incorporates a novel, physiology-informed multi-band tokenization scheme. This enables automatic extraction of task-relevant features across the entire frequency spectrum. • Extensive experiments on DailySense spanning six distinct tasks and four public ExG benchmarks show PiMT achieves state-of-the-art performance, with an average F1 score of 87% over baseline models. Together, these contributions establish the foundation for scalable, real-world ExG analysis, bridging wearable sensing technology and foundation models for deeper human understanding. 2 Related Work To enable effective analysis of ExG signals and uncover valuable physiological patterns, recent approaches can be categorized into following three main groups: (i) Conventional deep learning frameworks , such as EEGNet ( Lawhern et al. 2018 ) and DeepConvNet ( Schirrmeister et al. 2017 ) , leverage temporal and spatial convolutions to extract features directly from raw ExG signals. (ii) Transformer-based models , which capture local and long-term temporal dependencies, are well-suited for complex, high-dimensional ExG signals. Early efforts such as EEGConformer ( Song et al. 2022 ) combine convolution and attention to jointly model local and global patterns. PatchTST ( Nie et al. 2023 ) introduces patch-wise attention and independent channel encoding, while Medformer ( Wang et al. 2024 ) enhances feature extraction through multi-scale patching and cross-channel attention. Most recently, Bidirectional-Mamba ( Zhu et al. 2024 ) applies bidirectional state-space modeling for efficient long-range dynamics. (iii) Self-supervised learning methods aim to learn generalizable representations from unlabeled