Source Separation for A Cappella Music
Luca A. Lanzendörfer, Constantin Pinkl, Florian Grötschla · 2025
Read Source Separation for A Cappella Music by Luca A. Lanzendörfer, Constantin Pinkl, Florian Grötschla (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-26580.txt.
Introduction
Luca Lanzendorfer, Constantin Pinkl, and Florian Grotschla teach machines to split blended voices in a cappella song. Their 2025 paper faces a shifting cast: the number of active singers changes from mix to mix, and training sets of many-singer stems are small. A power-set trick answers the data gap, recombining stems into far more training mixes.
Their model adapts a top speaker-splitting design with wave-shaped activations and a loss that still learns when a stem sits silent. On a known Japanese a cappella set, the method set new best scores for full casts and for subsets alike. It beat spectrum-based rivals and held up on mixes with varied singer counts.
Catalog record
| CREATOR | Luca A. Lanzendörfer, Constantin Pinkl, Florian Grötschla |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-26580
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Source Separation for A Cappella Music Luca A. Lanzendörfer, Constantin Pinkl, Florian Grötschla Abstract In this work, we study the task of multi-singer separation in a cappella music, where the number of active singers varies across mixtures. To address this, we use a power set-based data augmentation strategy that expands limited multi-singer datasets into exponentially more training samples. To separate singers, we introduce SepACap, an adaptation of SepReformer, a state-of-the-art speaker separation model architecture. We adapt the model with periodic activations and a composite loss function that remains effective when stems are silent, enabling robust detection and separation. Experiments on the JaCappella dataset demonstrate that our approach achieves state-of-the-art performance in both full-ensemble and subset singer separation scenarios, outperforming spectrogram-based baselines while generalizing to realistic mixtures with varying numbers of singers. † † address: ETH Zurich 1 Introduction The field of music source separation has recently seen rapid progress [ 1 , 2 , 3 , 4 ] , primarily focusing on separating different instruments from each other by learning spectral masks that isolate the desired source from the mixture. In contrast, separating multiple singers in purely vocal (a cappella) recordings remains underexplored. A cappella ensembles range from small groups (duets and quartets) to chamber choirs, often organized by part (soprano, alto, tenor, and bass), as well as subset variants [ 5 , 6 , 7 ] . These recordings exhibit dense harmonic overlap, frequent unison/octave doubling within a part, voice crossing, vibrato and portamento, tightly aligned consonant onsets, and breath and sibilant noise. These properties reduce timbral diversity and make source separation more challenging than instrument-wise separation [ 8 ] . Contemporary a cappella may also include vocal percussion or beatboxing, further increasing spectral overlap despite all sources being human voices [ 9 , 10 ] . Despite applications in transcription, remixing, and choir analysis, singer separation has received relatively little attention. A major bottleneck is the limited availability of multi-singer datasets. These constraints make it difficult to train large neural network-based separation models and data augmentation strategies are needed to obtain more training samples. In this work, we present SepACap, an adaptation of the recently proposed SepReformer [ 11 ] , a state-of-the-art speaker separation model, for the a cappella setting where the number of active singers may vary across mixtures. We adapt the model by (i) introducing periodic activations [ 12 ] as we find them to perform better compared to the default ReLU activations; (ii) replacing the default training loss with a silence-aware composite objective that remains well-defined when stems are absent, combining waveform, multi-scale mel, and multi-resolution spectral losses; and (iii) coupling the model with a power set-based data augmentation scheme that creates mixtures for all subsets of stems, enabling joint separation and detection of active singers. We evaluate SepACap on JaCappella [ 13 ] in two scenarios: all-stems and subset-stems, and report both separation quality and silent-stem suppression quality, along with detection metrics. Across these settings, SepACap outperforms strong spectrogram-masking and waveform baselines, achieving state-of-the-art results in full-ensemble separation while markedly reducing bleed-through and correctly outputting silence for inactive stems. Our contributions can be summarized as follows: • We propose SepACap, an a cappella source separation model achieving state-of-the-art performance on multi-singer separation while operating in the waveform domain. • To increase available training data, and to enable model generalizability to subset singer separation, we use a power set-based data augmentation method that transforms a standard multi-singer dataset into exponentially more training samples, including cases with absent singers, enabling more robust separation performance. • We extend the loss function to allow for stable training with empty stems, ensuring that models operating on the waveform learn to handle silent signals in the data. 2 Related Work Audio separation. Research in music source separation has advanced along two principal directions. Spectrogram-based methods typically predict time-frequency masks applied to the mixture. Early CNN- and RNN-based systems have been surpassed by Transformer architectures that better capture spectral structure. For example, Band-Split RoFormer (BS-RoFormer) [ 14 ] introduced subband projections and interleaved Transformers, achieving first place in the Sound Demixing Challenge 2023 [ 15 ] , while its successor, Mel-Band RoFormer [ 4 ] , replaced empirical band splits with mel-scale projections for improved perceptual resolution. Similarly, TF-Locoformer [ 16 ] combined local convolutional and global Transformer modeling in the time-frequency domain. In contrast, waveform-based approaches estimate sources directly without masks. Conv-TasNet [ 17 ] pioneered learnable time-domain encoders and decoders, Demucs [ 1 ] extended this with a U-Net-like architecture, and SepReformer [ 18 ] combined global and local Transformer blocks with an early split into speaker-specific features, enabling efficient long-sequence processing. Together, these methods highlight the effectiveness of both spectrogram- and waveform-based modeling for music separation. Separation of singers. Compared to instrument separation, singer separation has received less attention despite its relevance for applications such as transcription, remixing, and choir analysis. Ensemble singing is especially challenging due to overlapping harmonics between singers. Therefore, most work in this domain focuses on filtering out the lead vocal. The authors of JaCappella [ 13 ] use a variety of models that are capable of separating singers which are singing at various pitch levels. They compare the performance of CrossNet-Open-Unmix (X-UMX) [ 19 ] , DPT-Net [ 20 ] , and MRDLA [ 21 ] on JaCappella. It should be noted that all tested approaches rely on all singers being present in the music mixture. Datasets for singer separation. Compared to other separation domains, singer separation datasets are highly limited. JaCappella [ 13 ] provides 6 vocal stems and finger snapping, totaling 34 minutes of audio across 35 songs in diverse genres such as jazz, punk rock, bossa nova, popular, reggae and enka. In addition to JaCappella, other a cappella datasets exist but are less diverse, less clean, or smaller in size: Dagstuhl ChoirSet [ 22 ] with 55 minutes of music from 2 classical songs performed by a choir of 16 singers. It includes the stems soprano, alto, tenor, and bass. Another dataset is ESMUC Choir [ 23 ] , comprising three songs, one each from the pop, folk, and classical choral genres, totalling approximately 31 minutes of audio. Both datasets suffer from bleed-through between singers and limited coverage of stems. 3 Methodology Model. To separate arbitrary singers in an a cappella setting, we introduce SepACap, an adaptation of SepReformer [ 11 ] , a recently proposed state-of-the-art speaker separation model. We found that updating the existing ReLU activation functions to the SNAKE activation [ 12 ] markedly increased model performance. Additionally, we changed the training objective from SNR-based training to a composite time and time-frequency loss. Many separation models from the speech domain use a variant of signal-to-noise ratio (SNR) or signal-to-distortion ratio (SDR) as their loss during training [ 11 , 24 , 25 ] , such as the scale invariant SDR (SI-SDR) [ 26 ] defined as SI-SDR \displaystyle:=10\log_{10}\frac{\|e_{target}\|^{2}}{\|e_{\text{res}}\|^{2}}=10\log_{10}\frac{\left\|\frac{\hat{s}^{T}s}{\|s\|}s\right\|^{2}}{\left\|\frac{\hat{s}^{T}s}{\|s\|}s-\hat{s}\right\|^{2}}, (1) where s is the target signal and {\hat{s}} is the predicted signal. The problem when using SI-SDR as a loss function is that, for stems without a signal, SI-SDR provides no informative gradient, making it ineffective as a training objective in such cases. Therefore, we utilize a