Source Separation of Small Classical Ensembles: Challenges and Opportunities

2025

Read Source Separation of Small Classical Ensembles: Challenges and Opportunities (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17823.txt.

Introduction

Roa-Dabike and co-authors present a 2025 paper on separating the instruments in small classical music ensembles. Splitting pop songs into parts works well with deep learning, but classical pieces resist. Their sound varies more, ground-truth recordings are scarce, and similar instruments blur together. The Cadenza project tackles the problem to remix music for listeners with hearing loss.

The team built a synthesized woodwind data set to fix gaps in an older one. Separate neural models each pull out one string or woodwind voice. Both live, causal processing and offline modes were tested, since hearing aids need the live kind. Results were graded on two small real-music sets.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17823

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Source Separation of Small Classical Ensembles: Challenges and Opportunities Abstract Musical source separation (MSS) of western popular music using non-causal deep learning can be very effective. In contrast, MSS for classical music is an unsolved problem. Classical ensembles are harder to separate than popular music because of issues such as the inherent greater variation in the music; the sparsity of recordings with ground truth for supervised training; and greater ambiguity between instruments. The Cadenza project has been exploring MSS for classical music. This is being done so music can be remixed to improve listening experiences for people with hearing loss. To enable the work, a new database of synthesized woodwind ensembles was created to overcome instrumental imbalances in the EnsembleSet. For the MSS, a set of ConvTasNet models was used with each model being trained to extract a string or woodwind instrument. ConvTasNet was chosen because it enabled both causal and non-causal approaches to be tested. Non-causal approaches have dominated MSS work and are useful for recorded music, but for live music or processing on hearing aids, causal signal processing is needed. The MSS performance was evaluated on the two small datasets (Bach10 and URMP) of real instrument recordings where the ground-truth is available. The performances of the causal and non-causal systems were similar. Comparing the average Signal-to-Distortion (SDR) of the synthesized validation set (6.2 dB causal; 6.9 non-causal), to the real recorded evaluation set (0.3 dB causal, 0.4 dB non-causal), shows that mismatch between synthesized and recorded data is a problem. Future work needs to either gather more real recordings that can be used for training, or to improve the realism and diversity of the synthesized recordings to reduce the mismatch. Furthermore, while the SDRs for the synthesized validation set are promising, there is still scope to further improve the models or to expand the training sets to improve performance. 1 Introduction People with hearing loss can struggle to identify and distinguish instruments in classical ensemble music [ 1 , 2 , 3 ] . Could music source separation (MSS) help? If MSS can separate music recordings into individual instrument lines, a remix can be made that increases the salience of some instruments. For instance, amplification, EQ or dynamic range compression could be applied to some instruments before remixing. The Cadenza Project is running machine learning challenges to improve music processing for people with hearing loss. This is needed because 68% of hearing aid users report difficulties when listening to music [ 4 ] . The second Cadenza Challenge (CAD2) included a task to rebalance musical instruments in a small classical ensemble [ 5 ] . MSS is an established area of research and there have been previous demixing challenges [ 6 , 7 , 8 ] . These have focused on non-causal separation of pre-recorded, studio-produced Western popular music. High performance can be achieved with the Signal-to-Distortion-Ratio (SDR) exceeding 9 dB [ 9 , 10 , 11 ] for the MUSDB18 benchmark [ 12 ] . People with hearing loss tend to be older and listen to more classic music [ 13 ] . If MSS is to help with music processing in hearing aids, it needs to work with classical music. But classical music offers a number of additional challenges for MSS compared to Western popular music. These include: (i) A lack of datasets to train deep learning algorithms. (ii) Classical ensembles often include more instruments. (iii) Some instruments in an ensemble have similar timbres that are harder to distinguish (e.g. violin and viola). (iv) Classical ensemble recording is usually done with all musicians in a performance space, which adds room acoustics to the signals. MSS for classical music is much less explored than popular music. Chiu et al. [ 14 ] trained an Open-Unmix model [ 15 ] to separate piano-violin duets. For 6 tracks from the MedleyDB dataset, they achieved SDRs of 9.66 and 1.56 dB for piano and violin, respectively. Sarkar et al. created the synthesized EnsembleSet data and then used it to train a duet separator using a dual-path transformer architecture [ 16 ] . For evaluation, they extracted duets of real recordings from the URMP dataset [ 17 ] and got an SDR of 11.37 dB. A recent paper created SynthSOD, a synthesized dataset that is better balanced across instruments than Ensembleset [ 18 ] . To test their new dataset, Garcia-Martinez et al. trained four independent Open-Unmix separator models for (i) strings, (ii) woodwind, (iii) brass, and (iv) percussion. Tests on the URMP data gave SDRs ranging from 0.0 dB for the tuba to 4.51 dB for the cello. These lower SDRs show that MSS with more than two instruments and real recorded test sets is still unsolved for classical music. Finally, previous MSS work has mostly developed non-causal algorithms. This is fine for recorded music, but for live music and hearing aids, causal approaches are needed. This paper arises from the MSS part of the CAD2 Cadenza challenge for classical music ensembles. (The remixing and hearing aid amplification parts of the challenge are not reported.) Below, the data are outlined, including the new synthetic woodwind dataset developed for the challenge, and the causal and non-causal MSS models used that formed the challenge baseline. Results follow, along with discussions around the unsolved problems in MSS for classical ensembles. 2 Materials and Methods 2.1 Datasets Real recordings of classical ensembles with completely isolated instrument tracks are scarce, see [ 18 ] for a summary. They are needed so the ground truth of the separated instrument signals are known. The recorded datasets that do exist are too small for training a deep neural network (DNN). Consequently, systems were trained on synthesized audio and evaluated on real recordings. For strings, we used the synthesized EnsembleSet [ 16 ] during training. This contains 80 pieces from 17 composers. EnsembleSet has synthesized audio for 11 different instruments and 18 virtual microphones in a concert hall. We used the professionally mixed Mix_1 render, which is a balance of the Decca tree, outriggers, ambient, balcony, mids, and close microphones. One limitation is that there are relatively few woodwind examples, and so the set has an unbalanced representation of the different instruments. Hence, we complemented with the CadenzaWoodwind, a new dataset of synthesized woodwind quartets [ 19 ] . This audio was generated by selecting 19 music scores from the OpenScore String Quartet corpus [ 20 ] . These were rendered for two ensembles of (i) flute, oboe, clarinet and bassoon; and (ii) flute, oboe, alto saxophone and bassoon. Synthesis was done by a professional music producer using virtual instruments that interpreted the expression markings in the score. Convolution reverberation simulated a performance space. For the validation set, a subset of tracks from the training dataset was held out. This contained 8 tracks from EnsembleSet and 6 from CadenzaWoodwind. These tracks were divided into consecutive 15-second segments and split to generate 261 samples, totalling just over one hour of data. Validation samples included 87 samples of string quartets from EnsembleSet and woodwind quartets from CadenzaWoodwind, and random mixtures (174 samples) combining string and woodwind instruments because mixed ensembles appear in the evaluation data. This should also aid training through

Read the complete text