CNN-based Robust Sound Source Localization with SRP-PHAT for the Extreme Edge

Jun Yin, Marian Verhelst · 2025

Read CNN-based Robust Sound Source Localization with SRP-PHAT for the Extreme Edge by Jun Yin, Marian Verhelst (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-02046.txt.

Introduction

Jun Yin and Marian Verhelst wrote this engineering paper, published in two thousand twenty-five, on finding where a sound comes from using tiny devices. Hearing aids and small robots need this skill in real time, but the best neural methods are too heavy for edge hardware.

The team starts from a robust network that reads SRP-PHAT features, a classic sound-location signal, and trims it for the extreme edge. Their paired methods cut the cost of building the features and shrink the network itself. Across settings, the network's compute cost fell by up to about three quarters, and its stored weights by far more. The most balanced version still matched top accuracy while running in under nine milliseconds per frame on a Raspberry Pi. That is more than seven times faster than the starting model. The work targets always-on listening on small, cheap chips.

Catalog record

CREATORJun Yin, Marian Verhelst
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-02046

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CNN-based Robust Sound Source Localization with SRP-PHAT for the Extreme Edge Jun Yin, Marian Verhelst Abstract . Robust sound source localization for environments with noise and reverberation are increasingly exploiting deep neural networks fed with various acoustic features. Yet, state-of-the-art research mainly focuses on optimizing algorithmic accuracy, resulting in huge models preventing edge-device deployment. The edge, however, urges for real-time low-footprint acoustic reasoning for applications such as hearing aids and robot interactions. Hence, we set off from a robust CNN-based model using SRP-PHAT features, Cross3D ( Diaz-Guerra et al. 2020 ) , to pursue an efficient yet compact model architecture for the extreme edge. For both the SRP feature representation and neural network, we propose respectively our scalable LC-SRP-Edge and Cross3D-Edge algorithms which are optimized towards lower hardware overhead. LC-SRP-Edge halves the complexity and on-chip memory overhead for the sinc interpolation compared to the original LC-SRP ( Dietzen et al. 2020 ) . Over multiple SRP resolution cases, Cross3D-Edge saves 10.32 ∼ \sim 73.71% computational complexity and 59.77 ∼ \sim 94.66% neural network weights against the Cross3D baseline. In terms of the accuracy-efficiency tradeoff, the most balanced version ( EM ) requires only 127.1 MFLOPS computation, 3.71 MByte/s bandwidth, and 0.821 MByte on-chip memory in total, while still retaining competitiveness in state-of-the-art accuracy comparisons. It achieves 8.59 ms/frame end-to-end latency on a Rasberry Pi 4B, which is 7.26x faster than the corresponding baseline. 1. Introduction Sound source localization (SSL) targets to derive the relative position of sound sources against the origin, which is typically the recording device. The most recent research focuses on the calculation of the Direction of Arrival (DoA) towards the microphone array, i.e. the source’s relative azimuth and elevation angles. In the past few decades, SSL techniques have been exploited with various types of sound, such as ocean acoustics ( Niu et al. 2017 ) , ultrasonic signals ( Kundu 2014 ) , and anisotropic-material conduction ( Kundu et al. 2012 ) . Nowadays, SSL on human-audible sounds is emerging rapidly for public or domestic usage, for instance in speech recognition ( Dávila-Chacón et al. 2018 ) , speech enhancement ( Xenaki et al. 2018 ) , noise control ( Chiariotti et al. 2019 ) , and robotic perception ( Rascon and Meza 2017 ) . There has been a long history of solving SSL problems with conventional signal processing methods, including the beamformer-based search ( Dmochowski et al. 2007 ) , subspace methods ( Schmidt 1986 ) , probabilistic generative mixture models ( Rickard and Yilmaz 2002 ) , and independent component analysis ( Sawada et al. 2003 ) . However, it is hard to generalize these methods to perform well under real-world conditions with complex noise-reverberation interference and sources’ spatial-temporal alternation. Thanks to the advent of deep learning ( LeCun et al. 2015 ) , one can further distill information inside the features which are extracted by these conventional methods. Since 2015, the number of DNN models for SSL is increasing explosively, covering all major types of network layer types, such as Multi-Layer Perceptrons (MLP) ( Tsuzuki et al. 2013 ) , convolutional neural networks (CNN) ( Hirvonen 2015 ) , convolutional recurrent neural networks (CRNN) ( Adavanne et al. 2019a ) , encoder-decoder neural networks ( Le Moing et al. 2019 ) , attention-based neural networks ( Cao et al. 2021 ) , etc. These state-of-the-art models will be further detailed in Section 2 . In many applications, these cutting-edge algorithms are required to be processed locally, for latency or privacy reasons. Such edge applications are for example drone navigation ( Hoshiba et al. 2017 ) , hearing aids ( Van den Bogaert et al. 2011 ) , and interactive robots ( Trifa et al. 2007 ) . These applications are expected to provide robust performance against harsh or varying environments during execution. Yet, these devices also suffer from limited space available for the computational unit, resulting in a need for a compute, memory, and energy-efficient design. This requires a new class of SSL models, optimized to run on a resource-constrained embedded device, yielding real-time outputs with limited computational performance, memory bandwidth, and power. Obviously, the mentioned requirements of SSL robustness and computation al efficiency create a tradeoff case, especially for the typically compute-heavy DNN models. In terms of SSL robustness, methods based on steered response power features with phase transform filter (SRP-PHAT) ( DiBiase et al. 2001 ) lead the SotA in SSL applications in harsh environments . Derived from the generalized cross-correlation (GCC) of microphone signals, the search among candidate locations with maximal SRP power is capable of tolerating noisy and reverberant environments. However, the original SRP-PHAT is computationally expensive which makes it impossible to meet the real-time requirements in edge devices . To relieve this, many modifications have been proposed to reduce SRP’s complexity ( Dietzen et al. 2020 ) , enhance the parallelism ( Minotto et al. 2013 ) , optimize localization mechanism ( Lima et al. 2015 ) and etc. Yet, computational requirements remain far above the capabilities of extreme edge devices. Moreover, recently, the combination of Deep Neural Network (DNN) models with the SRP features, results in cascaded SRP-DNN models ( Salvati et al. 2018 ; Diaz-Guerra et al. 2020 ) that show further SSL accuracy and robustness improvements. Yet, this again comes at an increased computational burden. Besides, SRP-PHAT sacrifices the spectral information of the source signal for its outstanding robustness. This further stresses the resource constraints, i.e. low-complexity demand, if auxiliary blocks are required to make up for such loss in complex missions. For instance, in sound event localization and detection (SELD) ( Adavanne et al. 2018b ) , event classification DNN is jointly built beside localization to resolve overlapped multiple targets. As such, while providing excellent robustness and SSL performance, the challenge of bringing the SRP-DNN method for SSL to the edge is twofold: (1) The computation overhead: Both the SRP-map grid search and DNN inference are time-consuming due to the computation amount and the data dependencies. (2) The extremely-mixed acoustic scenes: Although SRP is designed to handle noisy and reverberant cases with robustness, it is challenging to make compact, computational-efficient DNN models converge on mixed cases with miscellaneous acoustic environments and randomly moving sources. In this paper, we will start from the Cross3D model ( Diaz-Guerra et al. 2020 ) , which is designed to robustly solve the challenge-(2) for single-source localization at indoor environments. To our knowledge, currently the Cross3D’s dataset simulator can synthesize the widest range of indoor acoustic scenes, with randomness on noise levels, reverberation levels, source trajectories, sensor locations, room parameters and etc. On this dataset, Cross3D shows robust performance over random cases and outperforms other state-of-the-art models. However, the resulting Cross3D model becomes gigantic and fails for the challenge-(1), which will be elaborated in Section 3 and 4 . Therefore, we propose an optimized version of the Cross3D model towards edge-deployment hardware requirements. Firstly, we reveal the baseline Cross3D’s bottlenecks in algorithm and computation. Secondly, we assess and exploit the model trade-off point between algorithmic accuracy and low-complexity computation. For the SRP part, we propose LC-SRP-Edge based on LC-SRP ( Dietzen et al. 2020 ) for lower hardware overhead.

Read the complete text