T-TIME: Test-Time Information Maximization Ensemble for Plug-and-Play BCIs

Siyang Li, Ziwei Wang, Hanbin Luo, Lieyun Ding, Dongrui Wu · 2024

Read T-TIME: Test-Time Information Maximization Ensemble for Plug-and-Play BCIs by Siyang Li, Ziwei Wang, Hanbin Luo, Lieyun Ding, Dongrui Wu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07228.txt.

Introduction

Siyang Li and four co-authors present T-TIME, a 2024 method making brain-computer interfaces work without calibration sessions. EEG-based interfaces read brain signals to control devices, but each new user normally endures a tedious calibration because brains differ and signals drift. Transfer learning can help, yet mostly in offline settings where all the new user's data exist in advance. T-TIME tackles the online case, where unlabeled trials stream in and answers are due instantly.

It starts several classifiers from aligned past data, predicts each new trial by ensemble, then updates the classifiers by shrinking uncertainty and matching distributions. On three public motor imagery data sets, it beat about twenty rival methods. The authors call it the first test-time adaptation for calibration-free BCIs, a step toward plug-and-play devices.

Catalog record

CREATORSiyang Li, Ziwei Wang, Hanbin Luo, Lieyun Ding, Dongrui Wu
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-07228

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

T-TIME: Test-Time Information Maximization Ensemble for Plug-and-Play BCIs Siyang Li, Ziwei Wang, Hanbin Luo, Lieyun Ding, Dongrui Wu Abstract Objective : An electroencephalogram (EEG)-based brain-computer interface (BCI) enables direct communication between the human brain and a computer. Due to individual differences and non-stationarity of EEG signals, such BCIs usually require a subject-specific calibration session before each use, which is time-consuming and user-unfriendly. Transfer learning (TL) has been proposed to shorten or eliminate this calibration, but existing TL approaches mainly consider offline settings, where all unlabeled EEG trials from the new user are available. Methods : This paper proposes Test-Time Information Maximization Ensemble (T-TIME) to accommodate the most challenging online TL scenario, where unlabeled EEG data from the new user arrive in a stream, and immediate classification is performed. T-TIME initializes multiple classifiers from the aligned source data. When an unlabeled test EEG trial arrives, T-TIME first predicts its labels using ensemble learning, and then updates each classifier by conditional entropy minimization and adaptive marginal distribution regularization. Our code is publicized. Results : Extensive experiments on three public motor imagery based BCI datasets demonstrated that T-TIME outperformed about 20 classical and state-of-the-art TL approaches. Significance : To our knowledge, this is the first work on test time adaptation for calibration-free EEG-based BCIs, making plug-and-play BCIs possible. I Introduction A brain-computer interface (BCI) [ 1 ] enables direct communication between the human brain and a computer by neural activities of the user. It can assist, augment, or even repair human cognitive or sensory–motor functions [ 2 ] . In addition to rehabilitation for disabled people, BCIs have also found applications for able-bodied users in device control, education, gaming, and so on [ 3 ] . BCIs can be categorized into non-invasive, partially invasive, and invasive ones. Among them, non-invasive BCIs, which usually use electroencephalogram (EEG) as the input signal [ 4 ] , are the most convenient. The three classic paradigms in EEG-based BCIs are event-related potentials [ 5 ] , steady-state visual evoked potentials [ 6 ] , and motor imagery (MI) [ 7 ] . In an MI-based BCI, the user imagines the movement of his/her body parts (e.g., left hand, right hand, both feet, or tongue), which modulates different areas of the motor cortex of the brain [ 8 ] . These imageries are then mapped into specific commands for external device control. Despite various advantages, such as noninvasiveness and low cost, EEG demonstrates significant individual differences, i.e., for the same task, different users usually exhibit different EEG responses. The current solution requires a subject-specific calibration session before each use, which is time-consuming and user-unfriendly, hindering their broad real-world applications. Transfer learning (TL) [ 8 , 4 ] , which utilizes knowledge from previous users (source domains) to facilitate the learning for a new user (target domain), is a promising solution to alleviate individual differences and hence to reduce or eliminate the calibration effort. There are three different TL scenarios when the target domain is completely unlabeled: 1. Unsupervised domain adaptation (UDA), where the labeled source data and the unlabeled target data are combined for offline learning. 2. Source-free UDA (SFUDA), where the source model, instead of the source data, is used in UDA, usually for source domain privacy protection. 3. Test-time adaptation (TTA), where the source model is updated online to the target domain, whose test samples are classified in real-time. A comparison of these three settings is shown in Fig. 1 . Fig. 1: Three different TL settings, when the target domain is completely unlabeled. (a) Unsupervised domain adaptation (UDA); (b) source-free unsupervised domain adaptation (SFUDA); and, (c) test-time adaptation (TTA). Existing TL approaches mainly consider offline UDA or SFUDA settings, where all unlabeled target data are available. However, in real-world online TTA applications, such as MI-based wheelchair control, unlabeled EEG trials arrive in a stream, and immediate classification is required. This paper proposes Test-Time Information Maximization Ensemble (T-TIME) to accommodate the most challenging online TTA scenario for calibration-free EEG-based BCIs. T-TIME initializes multiple classifiers from the aligned source data. When an unlabeled test EEG trial arrives, T-TIME first predicts its labels using ensemble learning, and then updates each classifier by conditional entropy minimization and adaptive marginal distribution regularization. Extensive experiments on three public MI-based BCI datasets demonstrated that T-TIME outperformed about 20 classical and state-of-the-art TL approaches. To our knowledge, this is the first study on TTA for plug-and-play EEG-based BCIs. Our code is also publicized. The remainder of this paper is organized as follows: Section II introduces related work. Section III proposes T-TIME. Section IV presents experimental results to demonstrate the performance of T-TIME. Finally, Section V draws conclusions and points out some future research directions. II Related Works This section introduces related works on deep learning based TL. II-A Unsupervised Domain Adaptation UDA [ 9 ] combines the labeled source data and the unlabeled target data for offline learning. The domain discrepancies can be reduced from three perspectives: 1. Input space. CORrelation ALignment (CORAL) [ 10 ] aligns the second-order statistics of different input distributions with a linear transformation. For multi-channel EEG signals, Riemannian geometry-based approach [ 11 ] and Euclidean alignment (EA) [ 12 ] have demonstrated outstanding performance. 2. Feature space. Deep adaptation network (DAN) [ 13 ] and joint adaptation network (JAN) [ 14 ] perform feature alignment by minimizing the maximum mean discrepancies. Domain-adversarial neural network (DANN) [ 15 ] and conditional domain adversarial network (CDAN) [ 16 ] use adversarial training to reduce the feature discrepancies between the source and target domains. 3. Output space. Minimum class confusion (MCC) [ 17 ] utilizes a weighted prediction entropy to minimize the class confusion. Liang et al. [ 18 ] introduced an auxiliary classifier to improve the quality of the target domain pseudo labels. II-B Source-free Unsupervised Domain Adaptation SFUDA [ 19 ] uses the source model, instead of the source data, for UDA. It is appropriate for source domain privacy protection, but generally more challenging to implement. Source HypOthesis Transfer (SHOT) [ 20 ] uses information maximization and self-supervised learning for feature extraction in the target domain. To explicitly consider class-imbalance, imbalanced source-free domain adaptation (ISFDA) [ 21 ] uses intra-class tightening and inter-class separation to form better decision boundaries. SFUDA has also been used in EEG-based BCIs for privacy protection [ 22 ] . Li et al. [ 23 ] proposed a meta-learning strategy for multi-source model transfer. Zhang et al. considered lightweight SFUDA [ 24 ] and multi-source decentralized SFUDA [ 25 ] for privacy-preserving BCIs. II-C Test-time Adaptation Both UDA and SFUDA assume all unlabeled target data are available, i.e., they target at offline applications. On the contrast, TTA [ 26 ] focuses on online applications. In TTA, target data arrive in a stream, and a prediction must be immediately made for each coming sample. Thus, it is more challenging than UDA and SFUDA. Target domain pseudo-labels [ 27 ] can be used for model adaptation in TTA. For example, Chen et al. [ 28 ] combined contrastive learning with self-supervision on refined online pseudo-labels, and test-time template adjuster (T3A) [ 29 ] efficiently integrates few-shot learning with pseudo-labeling. Entropy-based uncertainty reduction has also been employed in

Read the complete text