Unsupervised EEG-based decoding of absolute auditory attention with canonical correlation analysis

Nicolas Heintz, Tom Francart, and Alexander Bertrand · 2025

Read Unsupervised EEG-based decoding of absolute auditory attention with canonical correlation analysis by Nicolas Heintz, Tom Francart, and Alexander Bertrand (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-17724.txt.

Introduction

Nicolas Heintz and co-authors published a research paper in 2025 on an algorithm that can tell from brain waves whether a person is truly listening to a sound. It needs no training labels at all. Knowing when attention is on or off helps hearing tech adapt. Supervised decoders need long, tedious calibration sessions for each user.

Their method pairs a correlation-based feature step with a lightly informed classifier, both unsupervised. On test data it beat a leading supervised model by a clear margin. The win comes from adapting cheaply to each person's drifting signals at test time. The door opens to attention tracking that sets itself up.

Catalog record

CREATORNicolas Heintz, Tom Francart, and Alexander Bertrand
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-17724

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Unsupervised EEG-based decoding of absolute auditory attention with canonical correlation analysis Thanks: N. Heintz and A. Bertrand are with KU Leuven, Department of Electrical Engineering (ESAT), STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics and also with Leuven.AI - KU Leuven institute for AI, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium (e-mail: [email protected], [email protected]). N. Heintz and T. Francart are with KU Leuven, Department of Neurosciences, Research Group ExpORL, Herestraat 49 box 721, B-3000 Leuven, Belgium (e-mail: [email protected]). This research is funded by the Research Foundation - Flanders (FWO) project nr. G0A4918N, the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 802895 and grant agreement No 637424), and the Flemish Government (AI Research Program). The scientific responsibility is assumed by its authors. Nicolas Heintz, Tom Francart, and Alexander Bertrand Abstract We propose a fully unsupervised algorithm that detects from encephalography (EEG) recordings when a subject actively listens to sound, versus when the sound is ignored. This problem is known as absolute auditory attention decoding (aAAD). We propose an unsupervised discriminative CCA model for feature extraction and combine it with an unsupervised classifier called minimally informed linear discriminant analysis (MILDA) for aAAD classification. Remarkably, the proposed unsupervised algorithm performs significantly better than a state-of-the-art supervised model. A key reason is that the unsupervised algorithm can successfully adapt to the non-stationary test data at a low computational cost. This opens the door to the analysis of the auditory attention of a subject using EEG signals with a model that automatically tunes itself to the subject without requiring an arduous supervised training session beforehand. I Introduction As the sound travels through the ear, it transforms into an electrical stimulus that eventually reaches the brain. The brain then processes this electrical stimulus through a complex cascade of electrical responses, which can be measured using electroencephalography (EEG) [ 1 , 2 , 3 ] . For amplitude-modulated sounds, such as speech or music, the cortical response tends to track the energy envelope of the incoming sound, which can be quantified using the correlation between the EEG and the speech envelope. This correlation is significantly higher when the subject actively listens to the incoming sound compared to when the subject ignores it [ 4 , 5 ] . This makes it possible to decode from EEG signals to what degree a person understands and actively listens to a given sound (either or not in the presence of other competing sound sources) [ 6 , 7 , 8 , 9 , 10 , 11 , 12 , 5 ] . Such decoders have a plethora of potential applications. For example, in audiology, the decoders yield objective markers of speech intelligibility [ 13 , 14 , 15 ] . Moreover, these decoders can also be used to improve contemporary hearing aids with the introduction of adaptive and automatic hearing aid fitting [ 16 ] or can control a noise reduction algorithm to amplify the attended speaker in a multi-talker scenario [ 6 , 10 , 9 , 17 , 18 ] . Finally, Roebben et al. recently demonstrated that such decoders can provide an accurate estimate of the extent to which a person actively pays attention to a given speaker [ 5 ] . Many of these decoders are based on regression algorithms that model the interaction between attended sound and measured EEG signals [ 6 , 7 , 8 , 9 , 19 , 10 , 11 ] . This interaction is different for each subject [ 7 ] and for different listening scenarios [ 20 ] . Although it is possible to train models that can generalise over multiple subjects and conditions, this leads to a substantial decrease in performance [ 7 , 6 , 21 ] . Alternatively, the model can be calibrated for each subject separately during a calibration session, although such a time-consuming practice would be impractical in real-life applications. Moreover, such subject-specific decoders are fixed and inherently assume that the neural response is stationary in time. This is not true in reality. Over time, the cortical response can change, electrodes can move, deteriorate and fail, the conductance between the electrodes and skin can vary, etc. More and more brain-computer interfaces, therefore, make use of adaptive models [ 22 , 23 , 24 ] . These models are continuously retrained on the incoming data, which allows them to adapt to the changes mentioned earlier. Since they can even be initialised with a random model, there is no need for a lengthy calibration session [ 23 ] . However, hearing devices are generally used outside a supervised lab environment, where there are no ground-truth labels indicating whether/to whom the subject is listening. In contrast, most state-of-the-art algorithms decoding auditory attention require such ground-truth labels to be trained [ 21 ] . This problem was first tackled in [ 18 , 23 ] , where an unsupervised linear regression model decodes to which of multiple competing speaker a subject is listening. This method was improved in [ 25 ] , which corrected a bias in the original algorithm that hampered convergence on smaller datasets. However, [ 18 , 23 , 25 ] all exclusively focused on selective Auditory Attention Decoding (sAAD, often simply called AAD), where the algorithm decodes to which of multiple competing speakers a subject is listening. This paper proposes an unsupervised algorithm that tackles the absolute Auditory Attention Decoding (aAAD) problem. Here, the objective is to decode whether or not a subject is actively paying attention to a single speaker [ 5 , 4 , 26 ] . The proposed model is based on the state-of-the-art supervised canonical correlation analysis (CCA) algorithm proposed in [ 10 , 27 ] for feature extraction, though adapted to make it more discriminative and unsupervised. These features are then classified using an unsupervised variant of linear discriminant analysis called minimally informed linear discriminant analysis (MILDA) [ 28 ] and thresholded using a Gaussian mixture model (GMM). Remarkably, the proposed unsupervised algorithm significantly outperforms the state-of-the-art supervised CCA model, even when the unsupervised algorithm is initialised randomly. This has two complementary causes: • EEG is highly non-stationary. The unsupervised algorithm is automatically tuned to the changing statistics of the incoming unlabelled test data, while this is not possible for supervised algorithms. • The ground-truth labels are imperfect. It is easy for a subject to briefly pay attention to the incoming sound when they shouldn’t and vice versa. This negatively affects the performance of the supervised model, which uses these imperfect labels [ 5 ] . The focus on a CCA+LDA model is motivated by its strong performance for match-mismatch and AAD tasks compared to other linear models [ 27 ] . Furthermore, the linearity allows for an efficient updating strategy with low computational cost, which makes them perfect for use in a wearable setting, where adaptive models are the most relevant. The paper is structured as follows. We first explain the supervised CCA algorithm on which our model is based in Section II . We then convert this model into an unsupervised self-adaptive model in Section III . Section IV discusses the experimental procedures and data that

Read the complete text