SYSTEMATIC EVALUATION OF TIME-FREQUENCY FEATURES FOR BINAURAL SOUND SOURCE LOCALIZATION

Davoud Shariat Panah, Alessandro Ragano, Dan Barry, Jan Skoglund, Andrew Hines · 2025

Read SYSTEMATIC EVALUATION OF TIME-FREQUENCY FEATURES FOR BINAURAL SOUND SOURCE LOCALIZATION by Davoud Shariat Panah, Alessandro Ragano, Dan Barry, Jan Skoglund, Andrew Hines (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-13487.txt.

Introduction

Davoud Shariat Panah and 4 co-authors systematically test which time-frequency features help a neural network locate sounds with two ears. Binaural localization can use loudness cues, phase cues, or mixes of both, and the best mix may shift with the listening setting. Their study holds the network fixed and varies only the input features.

Tests on matched and mismatched head shapes show that feature choice often beats added model complexity. A loudness-difference plus phase-difference pair sufficed for matched data, while richer mixes with full channel spectra generalized better to new content. Their compact network stayed competitive with the best feature sets, giving builders clear design guidance.

Catalog record

CREATORDavoud Shariat Panah, Alessandro Ragano, Dan Barry, Jan Skoglund, Andrew Hines
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-13487

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SYSTEMATIC EVALUATION OF TIME-FREQUENCY FEATURES FOR BINAURAL SOUND SOURCE LOCALIZATION Davoud Shariat Panah, Alessandro Ragano, Dan Barry, Jan Skoglund, Andrew Hines Abstract † † address: 1 School of Computer Science, University College Dublin, Dublin, Ireland 2 Google LLC, San Francisco, CA This study presents a systematic evaluation of time-frequency feature design for binaural sound source localization (SSL), focusing on how feature selection influences model performance across diverse conditions. We investigate the performance of a convolutional neural network (CNN) model using various combinations of amplitude-based features (magnitude spectrogram, interaural level difference - ILD) and phase-based features (phase spectrogram, interaural phase difference - IPD). Evaluations on in-domain and out-of-domain data with mismatched head-related transfer functions (HRTFs) reveal that carefully chosen feature combinations often outperform increases in model complexity. While two-feature sets such as ILD + IPD are sufficient for in-domain SSL, generalization to diverse content requires richer inputs combining channel spectrograms with both ILD and IPD. Using the optimal feature sets, our low-complexity CNN model achieves competitive performance. Our findings underscore the importance of feature design in binaural SSL and provide practical guidance for both domain-specific and general-purpose localization. 1 Introduction Binaural audio captures spatial hearing cues by recording or synthesizing sound with two channels to replicate the auditory information humans use for three-dimensional localization. Binaural SSL leverages interaural cues such as time and level differences, along with spectral shaping from the outer ear, to determine the spatial position of sounds in three dimensions [ 4 ] . Accurate source localization is crucial for natural hearing, enabling applications in hearing aids [ 21 ] , virtual reality [ 19 ] , auditory scene analysis [ 14 ] , and humanoid robots [ 11 ] . SSL has seen significant advances in recent years, with deep learning models achieving state-of-the-art accuracy and robustness in complex acoustic environments. A variety of architectures have been successfully applied to the SSL problem, among which CNNs (e.g., [ 23 , 6 , 22 , 20 ] ) and convolutional recurrent neural networks (CRNNs) (e.g., [ 1 , 7 , 15 ] ) are among the most widely used. More recently, a variety of Transformer models have been applied to the binaural SSL problem. For instance, Phokhinanan et al. [ 17 ] proposed an adapted vision Transformer (ViT) model for binaural SSL in noisy conditions, using ILD and IPD as input feature representations. In another study [ 18 ] , they introduced a ViT model with a spectral attention mechanism for binaural speech localization designed to handle mismatches in HRTFs between training and testing data. This model leveraged ILD, IPD, and magnitude spectrograms as input features. Kuang et al. [ 13 ] proposed a Transformer model based on Mamba ViT [ 9 ] for binaural SSL in both anechoic and reverberant conditions. This work used magnitude spectrograms as input features. As the above examples demonstrate, previous work has employed a variety of feature representations with binaural audio data. The choice of features directly influences how effectively a binaural SSL model can extract spatial cues, remain robust in noisy and reverberant conditions, and generalize across environments [ 5 , 8 ] . Therefore, thoughtful feature design is often as important as the choice of model architecture. However, research in this area remains very limited, and to our knowledge, only one work has examined how the choice of input features affects SSL performance. Krause et al. [ 12 ] compared amplitude-based, phase-based, and covariance matrix-based features in terms of sound event detection and localization performance when used with a CRNN architecture. However, the data used in this study were collected with a tetrahedral microphone array, and therefore, the findings may not generalize to binaural data. Moreover, the study restricted feature combinations to pairs and did not explore larger sets that could potentially result in more effective representations and improved performance. To address this gap, this study systematically investigates which feature set is more effective for binaural SSL using controlled experiments and cross-condition evaluation. Specifically, we present a comparative analysis of amplitude- and phase-based features, evaluating them individually and in combinations of up to four. Given that CNNs are among the most commonly used architectures in the field, we use these features as inputs to a CNN model. To assess the robustness of the features to content variability, we evaluate the models on both in-domain and out-of-domain content. Furthermore, we employ different sets of HRTFs from different subjects for training and testing to examine generalization across listening conditions. Finally, by making our source code and datasets publicly available, we provide the community with a benchmark resource that can accelerate future research, facilitate reproducibility, and support fair comparison of new methods. 2 EXPERIMENTAL SETUP To investigate the impact of feature choice on SSL performance, we train multiple models using various feature combinations and systematically compare their performance on in-domain and out-of-domain data. Below, we detail the datasets, feature representations, model architecture, model training, and evaluation. 2.1 Datasets We used the Binamix library [ 3 ] to synthesize datasets for model training and evaluation. This library enables binaural mixing using the SADIE II database [ 2 ] , which provides head-related impulse responses (HRIRs) and binaural room impulse responses (BRIRs) across 20 subjects. To handle angles where measured impulse responses are unavailable, Binamix applies a modified Delaunay triangulation for interpolation. The training set was generated by rendering randomly selected samples from the TSP speech dataset [ 10 ] with HRIRs from subjects D1, H3–H6, and H11–H13 in the SADIE database, covering the full sphere at 5° increments in both azimuth and elevation. This set contains 21,312 recordings. The validation set was generated in the same way but used subjects H3 and H4. This set includes 5,328 recordings. We also use three test sets in our experiments. The first is a speech-only test set, TSP–SSL, generated using the same procedure but with subjects H7–H10. To avoid using the same content, we ensured that speakers from the TSP dataset did not overlap across the training, validation, and test sets. The TSP–SSL test set includes 10,656 recordings. The second test set is the localization sensitivity subset from the SynBAD dataset [ 16 ] , which contains a variety of natural (e.g., castanets) and synthetic (e.g., pink noise) content spatialized using HRIRs from subject D2 of the SADIE database. This set has a variable angular resolution that becomes coarser toward the periphery, aligned with the characteristics of human perceptual localization accuracy, which is highest in the frontal region. It includes 11,500 recordings, and we refer to it as SynBAD–Var. The third test set is derived from the same content as SynBAD–Var, but rendered at a uniform 5° resolution across both azimuth and elevation. This test set includes 16,560 recordings, and we refer to it as SynBAD–Fix. We use the TSP–SSL as an in-domain test set and both SynBAD sets as out-of-domain sets. As mentioned above, the data in all sets cover the full azimuthal range (0° – 360°). We map azimuth angles to the frontal plane (-90° – 90°) by wrapping angles appropriately. All sets were originally sampled at 48 kHz but were resampled to 16 kHz for processing. The generated datasets will be published upon paper

Read the complete text