Analytic Incremental Learning for Sound Source Localization with Imbalance Rectification

Zexia Fan, Yu Chen, Qiquan Zhang, Kainan Chen, Xinyuan Qian · 2026

Read Analytic Incremental Learning for Sound Source Localization with Imbalance Rectification by Zexia Fan, Yu Chen, Qiquan Zhang, Kainan Chen, Xinyuan Qian (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18335.txt.

Introduction

Zexia Fan and co-authors teach sound localization systems to keep learning without forgetting. Their 2026 preprint deals with sound source localization, which finds where a sound came from. The method shines in controlled settings but struggles in the real world. Two imbalance problems bite. Within a task, some directions are far more common than others. Across tasks, new data skew and overlap with old. Both drive catastrophic forgetting as the system learns over time.

The authors propose analytic incremental learning with imbalance rectification. Analytic methods can update in closed form, which suits steady learning. Fixing the skew at both levels should keep old directions sharp. Hearing aids, robots, and smart rooms all need such ears.

Catalog record

CREATORZexia Fan, Yu Chen, Qiquan Zhang, Kainan Chen, Xinyuan Qian
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-18335

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Analytic Incremental Learning for Sound Source Localization with Imbalance Rectification Zexia Fan, Yu Chen, Qiquan Zhang, Kainan Chen, Xinyuan Qian Abstract Sound source localization (SSL) demonstrates remarkable results in controlled settings but struggles in real-world deployment due to dual imbalance challenges: intra-task imbalance arising from long-tailed direction-of-arrival (DoA) distributions, and inter-task imbalance induced by cross-task skews and overlaps. These often lead to catastrophic forgetting , significantly degrading the localization accuracy. To mitigate these issues, we propose a unified framework with two key innovations. Specifically we design a GCC-PHAT-based data augmentation (GDA) method that leverages peak characteristics to alleviate intra-task distribution skews. We also propose an Analytic dynamic imbalance rectifier (ADIR) with task-adaption regularization, which enables analytic updates that adapt to inter-task dynamic. On the SSLR benchmark, our proposal achieves state-of-the-art (SoTA) results of 89.0% accuracy, 5.3 ∘ mean absolute error, and 1.6 backward transfer, demonstrating robustness to evolving imbalances without exemplar storage. Code and data will be released. † † address: 1 University of Science and Technology Beijing, Beijing, China 2 The Chinese University of Hong Kong, Shenzhen, China 3 Tongyi Speech Lab, Alibaba, China 4 Eigenspace GmbH, Germany 1 Introduction Sound Source Localization (SSL) aims to estimate the DoA of sound sources from multichannel audio. In the context of human-robot interaction, SSL is fundamental for selective auditory attention [ 1 ] , automatic speech recognition [ 2 ] , speech enhancement [ 3 , 4 , 5 ] , and speaker extraction [ 6 ] in multi-speaker and noisy scenarios. Traditional signal processing approaches rely on idealized acoustic assumptions and derive closed-form solutions, such as GCC-PHAT [ 7 ] , MUSIC [ 8 ] , and SRP-PHAT [ 9 ] . They often perform well under controlled scenarios, but struggle with challenging acoustic conditions such as noisy and strong reverberant environments. Deep learning-based supervised neural SSL has recently demonstrated significant advancements. Existing approaches can be broadly grouped into hybrid [ 10 , 11 , 12 , 13 ] and pure neural solutions [ 14 , 15 , 16 , 17 ] . The former leverages a deep neural network (DNN) to enhance specific modules of traditional pipelines. While the latter optimizes DNNs to estimate DoA in an end-to-end fashion. In practice, labeled SSL data is scarce and arrive incrementally, as in robotic audition where DoA annotations evolve over time. A critical problem in this incremental setting is catastrophic forgetting, where the model loses previously acquired knowledge when adapting to new tasks. Class-incremental learning (CIL) has been explored to address this issue, which seeks to integrate new classes while preserving old knowledge. Existing CIL generally spans three paradigms: regularization-based approaches [ 18 , 19 ] that constrain weight updates to retain information, replay-based approaches [ 20 , 21 ] that store exemplars for knowledge distillation, and analytic approaches [ 22 , 23 , 24 , 25 ] that exploit closed-form solutions for computational efficiency. Building on these foundations, SSL-CIL [ 26 ] introduced the first CIL framework for SSL, yet CIL method remain challenged by complex real-world conditions such as repeated class occurrences and class imbalance problems. These limitations become particularly acute in the generalized CIL (GCIL) setting for SSL, which faces a fundamental challenge: the long-tailed class distributions of DoA data. This manifests in two forms: 1) Intra-task imbalance arises within each task, where certain directions dominate training data while minority directions remain severely underrepresented [ 27 , 28 ] ; 2) Inter-task imbalance occurs across tasks, with uneven or overlapping class distributions, which induces dynamic biases and exacerbates forgetting [ 29 ] . Together, these imbalances make GCIL for SSL particularly difficult, especially when privacy constraints prohibit storing past data. To tackle these challenges, we propose a novel unified framework, namely SSL-GCIL, that addresses both intra-task and inter-task imbalances. For intra-task imbalance, we introduce a GDA method that exploits GCC-PHAT peak statistics to synthesize samples for tail classes, preserving inter-microphone correlations without requiring external data. For inter-task imbalance, we design ADIR, an analytic classifier whose core adaptive regularization is guided by task-specific statistics (e.g., the Gini coefficient) to effectively mitigate cross-task skews and forgetting [ 28 ] . Together, these components form a lightweight, adaptive SSL framework tailored for GCIL. In summary, our contributions are summarized as: • Our proposal is the first SSL framework that address both intra-task and inter-task imbalances in the GCIL setting, providing an effective solution for long-tailed distributions which alleviate catastrophic forgetting. • We introduce GDA that exploits GCC-PHAT peak statistics to synthesize tail-class samples, alleviating intra-task imbalance while maintaining statistical consistency and privacy. • We propose an ADIR, a unified analytic framework that dynamically responds to inter-task class overlaps and distribution skews through its task-adaptive regularization. We conduct comprehensive experiments on the SSLR benchmark, the experimental results demonstrate the superiority of our proposed SSL-GCIL over SoTA baselines, achieving improvements of 3.2% in accuracy, 1.2 ∘ in mean absolute error, and 2.5 in backward transfer. Figure 1 : (a) Block Diagram of our proposed method and (b) GCC-PHAT-based Data Augmentation ( \bigoplus indicates addition). 2 Method In this section, we first formalize the problem of GCIL for SSL and then introduce our proposed method. As illustrated in Fig. 1 (a), the framework is composed of two main components: a GDA module for feature enrichment and an ADIR module with a frozen MLP feature extractor to map GCC-PHAT features to DoA probability. 2.1 Problem Formulation We formulate SSL as a GCIL problem, where data arrives in tasks, each containing new and old sound direction classes with imbalanced distributions. Let \mathcal{D}_{t}=\{(\mathbf{x}_{i}^{(t)},\mathbf{y}_{i}^{(t)},\mathbf{z}_{i}^{(t)})\}_{i=1}^{N_{t}} denote the training set of task t\in\{1,\dots,T\} , where \mathbf{x}_{i}^{(t)}\in\mathbb{R}^{P\cdot D_{a}} is the GCC-PHAT feature derived from a P -pair microphone array, D_{a} being the feature dimension per pair, \mathbf{y}_{i}^{(t)}\in\{0,1\}^{360} is the one-hot label indicating the true DoA in [0^{\circ},360^{\circ}) , and \mathbf{z}_{i}^{(t)}\in\mathbb{R}^{360} is the Gaussian-distributed label generated from \mathbf{y}_{i}^{(t)} with a peak at the true DoA and standard deviation \sigma=5^{\circ} . The classes in \mathcal{D}_{t} follow a long-tailed distribution with up to 60 classes per task. The goal is to train a model f_{\theta}:\mathbb{R}^{P\cdot D_{a}}\to\mathbb{R}^{360} minimizing cumulative loss over tasks without revisiting prior data: \mathcal{L}=\sum_{t=1}^{T}\mathbb{E}_{(\mathbf{x},\mathbf{z})\sim\mathcal{D}_{t}}\left[\ell(f_{\theta}(\mathbf{x}),\mathbf{z})\right], (1) where \ell is binary cross-entropy loss, addressing catastrophic forgetting and intra-task imbalance without exemplar storage. 2.2 GCC-PHAT-based Data Augmentation The GCC-PHAT estimates source directions from inter-microphone time delays. For signals with Fourier transforms X(f) and Y(f) , it is defined as R_{xy}(\tau)=\int_{-\infty}^{\infty}\frac{X(f)Y^{*}(f)}{|X(f)Y^{*}(f)|}e^{j2\pi f\tau}\,df, (2) where \tau denotes the lag. Empirically, the peaks of GCC-PHAT dominate SSL performance, as they encode critical time-difference cues [ 15 ] . To mitigate intra-task imbalance under long-tailed distributions [ 30 ] , we design a domain-specific augmentation that manipulates these peaks while preserving their statistical structure. For each task t , the feature vector \mathbf{x}_{i}^{(t)}\in\mathbb{R}^{P\cdot D_{a}} is partitioned into P segments (one per microphone pair), each of dimension D_{a} . Let N_{c}^{(t)} denote the number of samples in class c at task t . For class c , we compute mean and variance of peak positions \{p_{c,k}\} and amplitudes \{a_{c,k}\} across segments. Let M_{t}=\max_{c}N_{c}^{(t)} be the largest class size and \alpha=0.5 the augmentation rate. For any class with N_{c}^{(t)}<\alpha M_{t} , we generate K_{c}=\lceil\alpha M_{t}-N_{c}^{(t)}\rceil new samples using Algorithm 1 .

Read the complete text