On Ambisonic Source Separation with Spatially Informed Non-negative Tensor Factorization
Mateusz Guzik, Student Member, IEEE, Konrad Kowalczyk, Senior Member, IEEE · 2025
Read On Ambisonic Source Separation with Spatially Informed Non-negative Tensor Factorization by Mateusz Guzik, Student Member, IEEE, Konrad Kowalczyk, Senior Member, IEEE (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-10305.txt.
Introduction
Mateusz Guzik and Konrad Kowalczyk present a method for pulling apart sounds recorded in all directions at once, in this 2025 paper. Their recordings come from Ambisonic microphones, which capture a full sphere of sound. Listeners and engineers often want one voice or instrument alone from such a mix. Knowing roughly where each source sits can help the split.
The method breaks the sound into a grid of parts that cannot go below zero, then solves for the mix. Prior knowledge of each source's direction enters as a soft rule on how the sound spreads across channels. The authors derive four versions of the method from two error measures and two kinds of prior rule. Tests compare these against a version that uses no prior knowledge.
Catalog record
| CREATOR | Mateusz Guzik, Student Member, IEEE, Konrad Kowalczyk, Senior Member, IEEE |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-10305
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On Ambisonic Source Separation with Spatially Informed Non-negative Tensor Factorization Mateusz Guzik, Student Member, IEEE, Konrad Kowalczyk, Senior Member, IEEE Abstract This article presents a Non-negative Tensor Factorization based method for sound source separation from Ambisonic microphone signals. The proposed method enables the use of prior knowledge about the Directions-of-Arrival (DOAs) of the sources, incorporated through a constraint on the Spatial Covariance Matrix (SCM) within a Maximum a Posteriori (MAP) framework. Specifically, this article presents a detailed derivation of four algorithms that are based on two types of cost functions, namely the squared Euclidean distance and the Itakura-Saito divergence, which are then combined with two prior probability distributions on the SCM, that is the Wishart and the Inverse Wishart. The experimental evaluation of the baseline Maximum Likelihood (ML) and the proposed MAP methods is primarily based on first-order Ambisonic recordings, using four different source signal datasets, three with musical pieces and one containing speech utterances. We consider under-determined, determined, as well as over-determined scenarios by separating two, four and six sound sources, respectively. Furthermore, we evaluate the proposed algorithms for different spherical harmonic orders and at different reverberation time levels, as well as in non-ideal prior knowledge conditions, for increasingly more corrupted DOAs. Overall, in comparison with beamforming and a state-of-the-art separation technique, as well as the baseline ML methods, the proposed MAP approach offers superior separation performance in a variety of scenarios, as shown by the analysis of the experimental evaluation results, in terms of the standard objective separation measures, such as the SDR, ISR, SIR and SAR. † † aftertitle: ©©2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. DOI: 10.1109/TASLP.2024.3399618 I Introduction The field of spatial sound processing has recently been greatly influenced by the Ambisonic technology, owing to its useful qualities, which often make it the preferred choice for audio recording and reconstruction [ 1 , 2 ] . In particular, the recording procedure based on the Spherical Harmonic (SH) decomposition enables uniform coverage of the entire sound scene, which makes Ambisonics very well suited for spatial audio in general. Moreover, since the knowledge about the composition of the sound field is acquired, rather than the specific values of sound pressure in a fixed microphone setup, the Ambisonic technology also enables to conveniently decouple the recording and reconstruction stages. This quality is useful with respect to easily-scalable and versatile immersive audio playback systems, because the decoder can be straightforwardly determined for various loudspeaker setups. As a result, the interest in the Ambisonic audio format for 360 audio-video, augmented and virtual reality [ 3 ] , has attracted the attention of many researchers. The classical Ambisonic linear decoding suffers from low-order limitations, arising as a consequence of an insufficient spatial resolution, which may cause a notable degradation in perceived sound quality [ 4 , 5 , 6 , 7 ] . This has led to the formulation of alternative, more advanced, signal-dependent methods that involve parametric representation of the sound field [ 8 ] , such as Directional Audio Coding (DirAC) [ 9 ] and Higher‐Order Directional Audio Coding (HO-DirAC) [ 10 ] , High Angular Resolution Planewave Expansion (HARPEX) [ 11 ] or COding and Multidirectional Parameterization of Ambisonic Sound Scenes (COMPASS) [ 12 ] . Depending on the adopted model, these signal-dependent methods describe the sound field with various parameters, which need to be estimated from the Spherical Harmonic Domain (SHD) signals and are later utilized during the reconstruction stage. Among different parameters, the source power spectral densities and the individual source signals are of key importance, since the accuracy of their estimation determines the level of audio quality and control over the reproduction of the spatial sound scene. One other important parameter, which often appears in the context of spatial sound and its reproduction, are the Directions of Arrival (DOAs) associated with the positions of sound sources relative to the array position. Apart from the classical methods, such as fixed and adaptive spatial filtering [ 13 , 14 ] , an effort in terms of the methods for sound source separation in the SHD has already been observed. This includes the deep learning based approaches [ 15 , 16 , 17 , 18 ] , the Non-negative Tensor Factorization (NTF) based methods [ 19 , 20 , 21 , 22 ] , and many other [ 23 , 24 , 25 , 26 ] . While it is well established in the literature that the prior DOA knowledge can be utilized to substantially raise the quality of source separation [ 27 , 28 , 29 , 30 ] , only a limited literature concerns the usage of prior DOA information in the SHD for sound source separation [ 31 , 32 , 15 ] . Concerning methods that operate directly on the microphone signals, rather than in the SHD, there exist a multitude of approaches to sound source separation [ 33 , 34 , 35 ] and some of them incorporate prior localization knowledge [ 30 , 28 ] . While these methods could potentially be applied directly to the raw microphone signals, as opposed to the encoded Ambisonic recordings, the existing research indicates that an enhanced performance can be achieved by performing separation in the SHD [ 21 ] . Moreover, direct application of the sound source separation methods, formulated for the classical microphone arrays, is only possible if these methods do not assume prior knowledge about the microphone array, they are not direction-aware, i.e. the form of the steering vector is unknown. Unfortunately, in the case of the prior DoA incorporating methods, the steering vectors must be defined explicitly. While in the classical array setup, the steering vector is defined as a phase shift between the microphones, the Ambisonic signals are not characterized by a phase shift. This prevents the direct usage of the already existing prior DoA incorporating methods with the Ambisonic signals, unless they are explicitly adapted to the SHD. On the other hand, the methods formulated for the classical microphone arrays have to adopt frequency-dependent steering vectors, as opposed to the case of Ambisonic signals. By utilization this frequency-independence, broadband estimation is seamlessly enabled, owing to the fact that the Ambisonic SCMs can be averaged over frequency bands. This can often result in an enhanced performance, while at the same time enabling significant computational speedup, because the update equations for the spatial components can be calculated once, as opposed to calculating them independently per each frequency band. This, connected with the so far limited interest in incorporation of prior DoA information for Ambisonic source separation, constitutes the motivation underlying this research. In this work, we develop on our conference publication [ 32 ] , in which we presented an extension of an NTF-based algorithm from [ 22 ] . In its basic form, this algorithm employs a Parallel Factor Analysis (PARAFAC) NTF for the source spectrogram estimation [ 36 ]