DOA Estimation with Lightweight Network on LLM-Aided Simulated Acoustic Scenes

\authorblockN Haowen Li \authorrefmark 1, Zhengding Luo \authorrefmark 1, Dongyuan Shi \authorrefmark 2, Boxiang Wang \authorrefmark 1, Junwei Ji \authorrefmark 1, Ziyi Yang \authorrefmark 1and Woon-Seng Gan \authorrefmark 1 \authorblockA \authorrefmark 1 Nanyang Technological University, Singapore · 2025

Read DOA Estimation with Lightweight Network on LLM-Aided Simulated Acoustic Scenes by \authorblockN Haowen Li \authorrefmark 1, Zhengding Luo \authorrefmark 1, Dongyuan Shi \authorrefmark 2, Boxiang Wang \authorrefmark 1, Junwei Ji \authorrefmark 1, Ziyi Yang \authorrefmark 1and Woon-Seng Gan \authorrefmark 1 \authorblockA \authorrefmark 1 Nanyang Technological University, Singapore (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08012.txt.

Introduction

Haowen Li and co-authors revisit direction of arrival estimation, the task of finding where a sound came from. Most models learn from synthetic scenes made by mixing clean speech with room responses, which limits acoustic variety. A newer data set built with help from large language models offers richer, more realistic spatial audio scenes.

The authors benchmark several neural methods on that data and propose LightDOA, a light model using depthwise separable convolutions. It accepts multi-channel input and aims at varied rooms without heavy computation. Results show sound accuracy and robustness across scenes, making the approach suitable for devices with limited resources.

Catalog record

CREATOR\authorblockN Haowen Li \authorrefmark 1, Zhengding Luo \authorrefmark 1, Dongyuan Shi \authorrefmark 2, Boxiang Wang \authorrefmark 1, Junwei Ji \authorrefmark 1, Ziyi Yang \authorrefmark 1and Woon-Seng Gan \authorrefmark 1 \authorblockA \authorrefmark 1 Nanyang Technological University, Singapore
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-08012

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DOA Estimation with Lightweight Network on LLM-Aided Simulated Acoustic Scenes \authorblockN Haowen Li \authorrefmark 1, Zhengding Luo \authorrefmark 1, Dongyuan Shi \authorrefmark 2, Boxiang Wang \authorrefmark 1, Junwei Ji \authorrefmark 1, Ziyi Yang \authorrefmark 1and Woon-Seng Gan \authorrefmark 1 \authorblockA \authorrefmark 1 Nanyang Technological University, Singapore \authorrefmark 2 Northwestern Polytechnical University, China E-mail: [email protected], [email protected], [email protected], {boxiang001,junwei002, ziyi016}@e.ntu.edu.sg, [email protected] Abstract Direction-of-Arrival (DOA) estimation is critical in spatial audio and acoustic signal processing, with wide-ranging applications in real-world. Most existing DOA models are trained on synthetic data by convolving clean speech with room impulse responses (RIRs), which limits their generalizability due to constrained acoustic diversity. In this paper, we revisit DOA estimation using a recently introduced dataset constructed with the assistance of large language models (LLMs), which provides more realistic and diverse spatial audio scenes. We benchmark several representative neural-based DOA methods on this dataset and propose LightDOA, a lightweight DOA estimation model based on depthwise separable convolutions, specifically designed for mutil-channel input in varying environments. Experimental results show that LightDOA achieves satisfactory accuracy and robustness across various acoustic scenes while maintaining low computational complexity. This study not only highlights the potential of spatial audio synthesized with the assistance of LLMs in advancing robust and efficient DOA estimation research, but also highlights LightDOA as efficient solution for resource-constrained applications. 1 Introduction Spatial sound perception and directional cues are crucial in real-world applications [ 4 , 26 , 28 ] . As a core technique in array signal processing, Direction-of-Arrival (DOA) estimation has recently shown great promise for enhancing learning-based ANC systems by providing spatial priors that improve noise suppression and robustness [ 17 , 16 , 23 ] . In recent years, neural network-based and other data-driven approaches have gained popularity due to their ability to model complex spatial cues from acoustic signals. Numerous architectures have been explored for this task, including multi-layer perceptrons (MLPs) [ 12 , 25 , 30 ] , convolutional neural networks (CNNs) [ 8 , 7 , 2 ] , and convolutional recurrent neural networks (CRNNs) [ 19 , 3 ] , all of which have shown promising results under controlled settings. However, these methods face a key limitation: their success heavily relies on the diversity and realism of training data. Most existing datasets are constructed by convolving clean speech with simulated room impulse responses (RIRs), resulting in acoustically constrained and semantically limited scenarios [ 14 , 29 ] . Such simplified data often fails to reflect the complexities of real-world environments, where background noise, reverberation, and diverse sound events are common. As a result, models trained on these datasets tend to struggle with generalization in unseen or acoustically diverse conditions. To better bridge the gap between synthetic and real-world acoustic conditions, we leverage a recently proposed spatial audio dataset constructed with the assistance of large language models (LLMs)- Both Ears Wide Open (BEWO) [ 24 ] . Compared to traditional RIR-based corpora, this dataset introduces significantly greater diversity in both acoustic and semantic dimensions, including non-speech content, various environments, and dynamic contexts. This not only offers a more faithful proxy for real-world sound scenes, but also presents new challenges by exposing models to more complex conditions, raising the bar for network robustness and adaptability. In this paper, we investigate single-source (SS) DOA estimation on the BEWO dataset, systematically evaluate several representative neural DOA models under this challenging setting, and propose LightDOA , a lightweight CNN-based architecture that leverages depthwise separable convolutions to efficiently extract spatial cues from two-channel input. Inspired by the MobileNet family [ 9 , 20 , 10 , 22 ] , LightDOA achieves a favorable balance between accuracy and computational cost. Experimental results show that our method achieves comparable accuracy to existing approaches while significantly reducing model complexity. Therefore, LightDOA holds promise for deployment in downstream tasks such as ANC system [ 15 , 18 ] , where both source localization and real-time noise suppression are required on edge devices. Our contributions are threefold: • We benchmark several representative neural DOA models on a realistic and diverse two-channel dataset synthesized with LLM assistance; • We propose LightDOA , a lightweight CNN-based architecture based on depthwise separable convolutions, • We demonstrate that LightDOA maintains competitive accuracy with significantly lower complexity, benefiting from the diverse spatial data generated with LLM assistance. This highlights its potential for efficient DOA estimation and deployment in downstream applications. 2 Dataset Description 2.1 Dataset Overview Figure 1 : Pipeline for Constructing the SS Subset from AudioCaps [ 11 ] . We utilize the SS set of the BEWO dataset [ 24 ] 1 1 1 Dataset: https://huggingface.co/datasets/spw2000/BEWO-1M , which is a large-scale synthetic spatial audio corpus created by combining captions generated by LLMs with physically based acoustic rendering. The process of constructing the SS-set from AudioCaps is illustrated in Fig. 1 , including caption filtering, activity detection, CLAP [ 27 ] -based audio-text alignment filtering, spatial attribute enrichment via GPT-4(o) [ 1 ] , and stereo audio simulation using Pyroomacoustics [ 21 ] or gpuRIR [ 5 ] . Table 1 : DOA dataset composition used in this study. Subset Samples Description Training 43,213 For model training Validation 1,981 For early stopping, tuning Test 4,301 For final evaluation Source AudioCaps [ 11 ] (caption-based retrieval) Spatialization Simulated with room acoustics and IPD/ILD Content Type Single dominant source per scene Labels Azimuth DOA angles The DOA dataset used in our study is summarized in Table 1 , including train/val/test splits, audio source, and labels. All audio samples in the SS-subset contain only a single dominant source per scene, ensuring spatial labels are well-defined. Limitation on Front–Back Resolution: Despite being dual-channel, the BEWO dataset does not incorporate head-related transfer functions (HRTFs) or any form of anatomical filtering. Consequently, directional cues are limited to interaural phase difference (IPD) and interaural level difference (ILD). As shown in Fig. 2 (a), the raw DOA angles can result in front–back ambiguity. To mitigate this, a front–back mapping is applied to fold all angles into the [0^{\circ},180^{\circ}] range. The resulting class distributions for the training, validation, and test subsets are shown in Fig. 2 (b). Imbalanced Class Distribution: Another limitation lies in the distribution of DOA angles across the dataset. Although the train, validation, and test splits maintain similar overall statistics, the number of samples per class (i.e., per direction) is not uniformly distributed. Certain angles are overrepresented, while others have relatively few examples. This internal class imbalance can affect model training and evaluation, especially in tasks requiring fine-grained angular resolution. This imbalance is illustrated in Figure 2 . (a) KDE of training angles with sparse regions highlighted (b) Histogram of angle classes across train/val/test splits Figure 2 : Angle distribution of the BEWO dataset. (a) shows the kernel density estimation (KDE) of raw DOA angles in the training set, where sparse regions (e.g., around 22°, 67°, 112°, and 157°) are visually identifiable. (b) shows the histogram of DOA angles after applying the front–back mapping across training, validation, and test sets, which are globally consistent but imbalanced within each set. 2.2 Room and Microphone Configuration Each scene is simulated within a cuboid room [R_{0},R_{1},R_{2}] based on a scene type-specific base size r , perturbed with uniform

Read the complete text