Do Music Source Separation Models Preserve Spatial Information in Binaural Audio?
2025
Read Do Music Source Separation Models Preserve Spatial Information in Binaural Audio? (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-00155.txt.
Introduction
Richa Namballa, Agnieszka Roginska, and Magdalena Fuentes test whether music source separation models preserve spatial cues in binaural audio, in a two thousand twenty-five audio research paper. Binaural recordings matter for virtual reality and access, yet separation research mostly ignores them.
The authors synthesize a binaural data set from MUSDB18-HQ stems, placing instruments around a flat circle with head-related transfer functions. Popular separation models process the two channels, and the separated stems are scored with signal-processing and interaural cue metrics. Stereo models fail to preserve the spatial information that makes binaural audio feel real. The damage depends on architecture and instrument, pointing to concrete chances for spatially aware separation.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-00155
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Do Music Source Separation Models Preserve Spatial Information in Binaural Audio? Abstract Binaural audio remains underexplored within the music information retrieval community. Motivated by the rising popularity of virtual and augmented reality experiences as well as potential applications to accessibility, we investigate how well existing music source separation (MSS) models perform on binaural audio. Although these models process two-channel inputs, it is unclear how effectively they retain spatial information. In this work, we evaluate how several popular MSS models preserve spatial information on both standard stereo and novel binaural datasets. Our binaural data is synthesized using stems from MUSDB18-HQ and open-source head-related transfer functions by positioning instrument sources randomly along the horizontal plane. We then assess the spatial quality of the separated stems using signal processing and interaural cue-based metrics. Our results show that stereo MSS models fail to preserve the spatial information critical for maintaining the immersive quality of binaural audio, and that the degradation depends on model architecture as well as the target instrument. Finally, we highlight valuable opportunities for future work at the intersection of MSS and immersive audio. 1 Introduction In recent years, immersive experiences have gained popularity in various forms of media such as video games, concerts, and movies. The shift to virtual and augmented reality (VR/AR) requires not only realistic visual stimuli, but authentic auditory cues as well. One common form of spatial audio used to provide the listener with directionality of sound is binaural audio. Binaural audio goes beyond traditional gain-based stereo panning by filtering two-channel audio to create interaural cues differing in level, time, and spectral content to simulate the location of a source in space [ 1 ] . Furthermore, binaural audio requires reproduction through headphones or loudspeakers equipped with crosstalk cancellation to maintain spatial imaging integrity. Level differences resulting from the “head-shadow effect” and the Time Difference of Arrival (TDOA) of a sound at each ear provide directional cues. Frequency-dependent filtering, determined by the form of the listener’s head and specific ear (pinna) shape, causes two identical sound sources positioned differently to exhibit slightly different spectral content at each ear, further assisting localization. The two common methods of producing binaural audio are recording with a binaural dummy head and signal processing with a Head-Related Transfer Function (HRTF). Beyond the increasing demand for immersive VR/AR experiences, binaural audio has significant potential applications in accessibility. For instance, individuals who identify as neuro-divergent or hard of hearing often benefit from enhanced auditory clarity, enabling them to isolate and focus on specific sound sources in complex acoustic environments, facilitating independent navigation and interaction in social and public settings. Binaural source separation has been shown to significantly enhance auditory accessibility by reducing background noise and emphasizing relevant auditory signals in real-time with the use of microphone-enabled headphones [ 2 ] . In this context, music source separation (MSS) in binaural audio could substantially improve how individuals engage with and enjoy musical environments such as concerts, festivals, and other live performances, enabling users to isolate specific musical elements or instruments and thus enhance their listening experience and overall participation in music events. These tools can further be utilized for recorded binaural content such as spatial audio captures of live performances or binaural field recordings. Despite these potential benefits and growing interest, binaural audio processing has received limited attention within the music information retrieval (MIR) community, particularly concerning MSS. In this work, we investigate whether existing MSS models are able to separate binaural mixtures into their respective stems while preserving the spatial characteristics, which are crucial for the immersive experience provided by binaural audio. We create a binaural MSS dataset based off of the well-established MUSDB18-HQ dataset [ 3 ] , and leverage several metrics that quantify separation quality, spatial distortion, and immersiveness to evaluate these models. Our results show that there is a considerable gap in binaural MSS performance compared with MSS in simpler stereo settings, and that this gap depends on model architecture and target source. Lastly, we discuss the shortcomings of current metrics and identify opportunities for future research. 2 Related Work Until now, most work on binaural source separation has been completed in the speech domain, often overlapping with the similar task of target source extraction (TSE). In particular, the speech research community describes the task as two-fold: source separation and localization [ 4 ] . We focus on prior studies concerning the former. Early two-channel source separation models were primarily signal processing-based, with a focus on mathematical and theoretical techniques [ 5 ] . As the focus moved towards capturing directionality, models began using psychoacoustic spatial cues to improve the performance of the signal processing-based source estimation methods [ 6 , 7 , 8 , 9 , 10 , 11 ] . With the technological progress made in computational resources, binaural source separation models shifted to using deep learning approaches to perform source extraction in more complex environments and in real-time [ 12 , 13 , 14 , 15 , 2 ] . Recent deep learning systems have proposed novel loss functions aimed at preserving the level, phase, and time differences between binaural channels, cues which are critical to the immersive nature of binaural audio [ 16 , 17 ] . To the best of our knowledge, the only published work on binaural MSS thus far concerns vocal separation of binaural audio recorded with a dummy head [ 18 ] . Their approach uses various hybrid combinations of single- and multichannel-source separation algorithms to extract the vocal stems, with a focus on signal-processing methods [ 19 , 20 , 21 , 22 , 23 ] . The results are evaluated with standard source separation metrics [ 24 ] and subjective listener ratings. Based on the limited existing research in binaural MSS, we believe that there is a significant opportunity to explore this task using deep learning methods, inspired by recent progress in the speech community. Regarding performance, the most common metric reported for evaluating source separation models is the Signal to Distortion Ratio (SDR), measured in decibels (dB) [ 24 ] . Specifically, for MSS, researchers often benchmark their models on the test set of MUSDB18-HQ and report the SDR both overall and by instrument type [ 25 ] . SDR (and its scale-invariant version, SI-SDR [ 26 ] ) aim to reflect what portion of the estimated stem corresponds to the reference stem versus any error introduced by interference from other instruments, noise, and artifacts. While SDR is well-established for evaluating mono and stereo tracks [ 27 ] , it does not specify the amount of spatial error introduced between channels in the model’s estimated output, which is essential for evaluating the quality of binaural source separation. Therefore, we leverage other metrics from the literature which reflect spatial quality. In the immersive audio research community, there are several models used to quantify the quality of a binaural signal, such as BAM-Q [ 28 ] and MoBi-Q [ 29 ] , trained on a combination of extracted binaural features and subjective quality ratings. We save the use of these models for future work in binaural MSS and choose to focus on more accessible and interpretable metrics, further explained in Section 4.1 ,