Auditory Attention Decoding without Spatial Information: A Diotic EEG Study
Masahiro Yoshino 1 , Haruki Yokota 2 , Junya Hara 2 , Yuichi Tanaka 2 , Hiroshi Higashi 2 · 2026
Read Auditory Attention Decoding without Spatial Information: A Diotic EEG Study by Masahiro Yoshino 1 , Haruki Yokota 2 , Junya Hara 2 , Yuichi Tanaka 2 , Hiroshi Higashi 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16442.txt.
Introduction
Masahiro Yoshino and three coauthors read which voice a hearer tracks from brain waves. Their 2026 preprint deals with the cocktail party task. In a room of voices, a hearer picks one voice to track. Brain wave tools can tell which voice that is. Such tools back smart aids that lift the tracked voice.
Past tools lean on where each voice sits in the room. This test drops that place cue and uses mixed sound. Both ears get the same mix in the diotic case. A tool that works sans place cues fits more real rooms. The work tests brain reads in that hard case.
Catalog record
| CREATOR | Masahiro Yoshino 1 , Haruki Yokota 2 , Junya Hara 2 , Yuichi Tanaka 2 , Hiroshi Higashi 2 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-16442
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Auditory Attention Decoding without Spatial Information: A Diotic EEG Study Masahiro Yoshino 1 , Haruki Yokota 2 , Junya Hara 2 , Yuichi Tanaka 2 , Hiroshi Higashi 2 Abstract Auditory attention decoding (AAD) identifies the attended speech stream in multi-speaker environments by decoding brain signals such as electroencephalography (EEG). This technology is essential for realizing smart hearing aids that address the cocktail party problem and for facilitating objective audiometry systems. Existing AAD research mainly utilizes dichotic environments where different speech signals are presented to the left and right ears, enabling models to classify directional attention rather than speech content. However, this spatial reliance limits applicability to real-world scenarios, such as the “cocktail party” situation, where speakers overlap or move dynamically. To address this challenge, we propose an AAD framework for diotic environments where identical speech mixtures are presented to both ears, eliminating spatial cues. Our approach maps EEG and speech signals into a shared latent space using independent encoders. We extract speech features using wav2vec 2.0 and encode them with a 2-layer 1D convolutional neural network (CNN), while employing the BrainNetwork architecture for EEG encoding. The model identifies the attended speech by calculating the cosine similarity between EEG and speech representations. We evaluate our method on a diotic EEG dataset and achieve 72.70% accuracy, which is 22.58% higher than the state-of-the-art direction-based AAD method. I Introduction Auditory attention decoding (AAD) [ 17 , 22 , 5 ] aims to identify the attended speech stream in multi-speaker environments by decoding brain signals such as electroencephalography (EEG) [ 8 ] . This technology is essential for realizing smart hearing aids [ 16 ] , capable of selectively enhancing a specific speech source in noisy environments, thereby addressing the so-called cocktail party problem [ 7 ] . Furthermore, AAD is expected to improve understanding of neural mechanisms of selective listening and facilitates the development of objective audiometry systems [ 25 , 24 ] . Existing AAD research is primarily classified into two approaches based on the decoding strategy. The first approach detects the attended speech by analyzing the correspondence between EEG and speech signals [ 2 , 10 ] . The second approach classifies the attended direction of the speech based on only the EEG signals [ 27 , 6 , 18 , 13 ] . DARNet [ 6 ] , a state-of-the-art deep learning model, adopts the directional strategy and achieves 96.2% accuracy on the standard KUL dataset [ 9 ] . These AAD approaches have supporsed and evaluated with dichotic environments. In this environment, speech signals are clearly separated into the left and right ears [ 4 , 15 , 23 ] . This separation makes it easier to classify based on spatial cues, which effectively serve as explicit directional information to distinguish the target speaker. The AAD models can capture this directional information and decode the direction of the target speaker. However, these spatial cue-based models may not work in real-world cocktail party scenarios where speakers often stand close together or move dynamically. Consequently, methods using spatial cues may have limited applicability. To build AAD systems that identify the attended speech even when sound sources overlap or move, we need to consider a diotic scenario. In this scenario, the exact same mixture of speech streams is simultaneously presented to both ears. The spatial cues are identical across streams and provide no discriminative information for source localization. In this paper, to address the AAD problem in the diotic setting, we propose a framework based on speech features. Our approach maps EEG and speech signals into a shared latent space using independent encoders. Specifically, we extract speech features using a pre-trained wav2vec 2.0 model [ 3 ] and encode them using a 2-layer 1D convolutional neural network (CNN). Simultaneously, we employ the BrainNetwork architecture [ 28 ] to encode EEG signals. In the shared latent space, we calculate the cosine similarity between the EEG and each speech stream. By evaluating this direct correspondence, our model identifies the attended speech based on speech content without using spatial information. We evaluate the proposed method using the diotic EEG dataset in [ 26 ] . Our method achieves 72.70% accuracy on five-second segments. In contrast, DARNet performs 50.12% in the diotic setting, which is close to chance level. This result validates that our proposed framework effectively solves the diotic AAD problem based on speech content without relying on spatial cues. Furthermore, we validate the neuroscientific plausibility of our model through two analyses. Based on the Late Selection Theory [ 11 ] , we validate that our model distinguishes speeches at the attentional selection stage rather than the early acoustic processing stage. Additionally, using SHAP analysis [ 20 ] in the context of Top-down Gain Control [ 14 ] , we show that our model specifically highlights EEG channels associated with attentional control. These analyses suggest that our framework captures the neural mechanisms of selective speech attention. II Diotic Auditory Attention Decoding This section presents our proposed diotic AAD framework designed to overcome the spatial reliance of conventional AAD methods in dichotic settings. II-A Task Figure 1 illustrates the concept of the diotic AAD task along with the dichotic setting. The goal of the diotic AAD task is to identify the attended speech stream from a set of multiple speech streams. Unlike dichotic environments where different speech signals are presented to each ear, the diotic setup presents a mixture of N speech streams to both ears simultaneously. Let \mathbf{E}\in\mathbb{R}^{C\times T} denote an EEG segment consisting of C channels and T time instances, and let \{\mathbf{S}_{1},\dots,\mathbf{S}_{N}\} be a set of N candidate speech streams. Here, each stream \mathbf{S}_{i}\in\mathbb{R}^{F\times T} has a feature dimension F . Given these inputs, the task is to predict the target index \hat{y}\in\{1,\dots,N\} corresponding to the attended speech. Using a classifier function g(\cdot;\boldsymbol{\phi}) parameterized by learnable weights \boldsymbol{\phi} , the AAD task is formulated as \hat{y}=g(\mathbf{E},\mathbf{S}_{1},\mathbf{S}_{2},\dots,\mathbf{S}_{N};\boldsymbol{\phi}). (1) II-B Framework Figure 2 shows the overview of our proposed diotic AAD framework. The framework consists of three main stages: feature extraction, similarity calculation, and training. II-B 1 Feature Extraction The model maps a preprocessed EEG segment \mathbf{E} and candidate speech streams \mathbf{S}_{i} into a shared latent space using independent encoders. The detailed preprocessing procedures for EEG and speech signals are described in Sections III-B and III-C , respectively. For the speech streams, we utilize wav2vec 2.0 features as inputs, which are known to improve the detection of synchronization between EEG and audio signals [ 28 ] . These features are transformed by a speech encoder consisting of a 2-layer 1D convolutional neural network (CNN). For the EEG inputs, we employ the BrainNetwork architecture [ 28 ] , which consists of a spatial attention layer and a 5-block residual network (ResNet). As a result, we obtain feature maps \mathbf{Z}_{E} for the EEG signals and \mathbf{Z}_{i} for the i th speech stream from the respective encoders, where \mathbf{Z}_{l}\in\mathbb{R}^{F\times T} for l\in\{E,1,\dots,N\} , F denotes the feature dimension, and T denotes the time length. II-B 2 Similarity Calculation We apply a learnable weight vector \mathbf{w}\in\mathbb{R}^{F} to the feature maps via channel-wise scaling. This allows the model to emphasize important features in the latent representation. The weighted representation