Sanidha: A Studio Quality Multi-Modal Dataset for Carnatic Music

2025

Read Sanidha: A Studio Quality Multi-Modal Dataset for Carnatic Music (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-06959.txt.

Introduction

Venkatakrishnan Krishnan and three co-authors release the first studio-grade multi-track data set for Carnatic music, the classical tradition of South India. Their 2025 paper serves source separation, the task of unmixing a recording into voice, drums, and melody lines. Training data for that task come almost wholly from Western studio music, so models falter on Carnatic performances.

Live Carnatic recordings bleed across tracks, which poisons training. The new set, Sanidha, was recorded in a studio with clean separation between stems, plus high-definition video of the artists. The team fine-tuned a popular separation model on the set and beat fine-tuning on an older Carnatic collection. A listening study judged the outputs. The data set is open to all researchers.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-06959

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sanidha: A Studio Quality Multi-Modal Dataset for Carnatic Music Abstract Music source separation demixes a piece of music into its individual sound sources (vocals, percussion, melodic instruments, etc.), a task with no simple mathematical solution. It requires deep learning methods involving training on large datasets of isolated music stems. The most commonly available datasets are made from commercial Western music, limiting the models’ applications to non-Western genres like Carnatic music. Carnatic music is a live tradition, with the available multi-track recordings containing overlapping sounds and bleeds between the sources. This poses a challenge to commercially available source separation models like Spleeter and Hybrid Demucs. In this work, we introduce Sanidha , the first open-source novel dataset 1 1 1 Sanidha dataset (Licensed under CC-BY-4.0) is hosted in the server: https://ccml.gtcmt.gatech.edu/data/Sanidha for Carnatic music, offering studio-quality, multi-track recordings with minimal to no overlap or bleed. Along with the audio files, we provide high-definition videos of the artists’ performances. Additionally, we fine-tuned Spleeter, one of the most commonly used source separation models, on our dataset and observed improved SDR performance compared to fine-tuning on a pre-existing Carnatic multi-track dataset. The outputs of the fine-tuned model with Sanidha are evaluated through a listening study. 1 Introduction Carnatic music is a traditional "art music" genre from the Southern part of India. Carnatic Music is largely improvised, requiring all musicians to utilize a complex understanding of the melodic and rhythmic structures of the music to improvise coherently. Carnatic performances generally feature four to five musicians centered around a vocalist in the lead role. The core instruments are the violin, in both supportive and lead roles; the mridangam , a tonal two-sided drum that provides rhythmic support; and the ghatam , a clay pot instrument that contributes rhythmic patterns to complement the mridangam in a higher frequency range. Carnatic Music performances are also accompanied by a tanpura , which constantly oscillates the sa , the tonic, and either the pa , the fifth or sometimes ma , the fourth. All the instruments are tuned to these frequencies, including the percussion instruments, which, too, have tonal qualities [ 1 ] . This leads to a significant overlap of frequency content, making Carnatic Music source separation almost impossible with simple dictionary learning methods [ 2 ] . Like most traditional music genres, Carnatic Music is performed live [ 1 ] . Thus, recordings of Carnatic Music lack multi-track isolation, as microphones inevitably capture signals from multiple instruments as well as the audience—these unwanted signals are known in music production as leakage or “bleed.” This contrasts with Western pop music, where completely isolated multi-tracks are commonplace, and many source separation datasets are available [ 3 , 4 , 5 , 6 ] . The most extensive open-source Music Information Retrieval (MIR) dataset of Indian art music—the Saraga dataset [ 7 ] —exhibits significant leakage between different audio tracks: For example, the sound of the violin is audible in the vocal track - The bleeding of other sources into other microphones is significant [ 8 , 9 , 10 ] . 1.1 Leakage Problem Consider a signal \mathbf{s} , noise \mathbf{n} , and a mix \mathbf{x} , at 0 dB Signal-to-Noise Ratio (SNR): \mathbf{x}=\mathbf{s}+\mathbf{n} , where \mathbf{x},\mathbf{s},\mathbf{n}\in\mathbb{R}^{d} . Let \mathbf{s_{t}},\mathbf{n_{t}}\in\mathbb{R}^{d} such that they represent ground truth signal and noise with bleed. Assume no microphone sensor noise and no Room Impulse Response (RIR). Then \mathbf{x=s_{t}+n_{t}} (1) \mathbf{s_{t}}=f(\mathbf{s},\mathbf{n})=\alpha\mathbf{s}+\beta\mathbf{n} (2) where \alpha\in[0,1] and \beta\in[0,1] , using Eq. 1 , it follows that \mathbf{n_{t}}=g(\mathbf{s},\mathbf{n})=(1-\alpha)\mathbf{s}+(1-\beta)\mathbf{n} (3) Assume that functions f and g are linear time-invariant functions for all audios. However, the \alpha and \beta values will vary for different signals in a general unclean dataset. Figure 1 : Problem of Poor Ground Truth Let the source separation function trained with (\mathbf{s_{t}},\mathbf{n_{t}}) as the ground truth be \mathbf{F} , such that \mathbf{F}(\mathbf{x})=(\mathbf{\hat{s}},\mathbf{\hat{n}}) For simplicity, let us assume \mathbf{\hat{s}} , \mathbf{s_{t}} and \mathbf{n_{t}} lie in the same subspace as \mathbf{s} and \mathbf{n} ; \mathbf{s} and \mathbf{n} are orthogonal to each other i.e. \mathbf{s}^{T}\mathbf{n}=0 as seen in Figure 1 . The most common metric used for evaluation and loss in the source separation community is the Signal-to-Distortion Ratio (SDR) and, more recently, the Scale-Invariant version of SDR called SI-SDR [ 11 ] . For simplicity, let’s consider using SDR for evaluation since the idea can easily be extended to SI-SDR. SDR is defined by [ 12 ] for the BSS_eval toolbox (which is the same as classical SNR) as: \text{SDR}_{s}=10\log_{10}\bigg(\frac{||\mathbf{s}||}{||\mathbf{s}-\mathbf{\hat{s}}||}\bigg) Given that f and g functions vary for each audio, the SDR formula above is modified for data with bleed as: \text{SDR}_{s,mod}=10\log_{10}\bigg(\frac{||\mathbf{s_{t}}||}{||\mathbf{s_{t}}-\mathbf{\hat{s}}||}\bigg) These objective results from SDR, however good, will never truly represent what the original source must sound like. Training on data with a significant bleed will never push the predicted \mathbf{\hat{s}} towards the actual source \mathbf{s} , since the loss function will be trained on the modified function dependent on sources with bleeding. Furthermore, the result will be subpar after incorporating scale invariance [ 11 ] . If we calculate the norm of \mathbf{s_{t}} and \mathbf{n_{t}} , using Eq. 2 , 3 , and the triangle inequality, we can prove: ||\mathbf{s_{t}}||+||\mathbf{n_{t}}||\leq||\mathbf{s}||+||\mathbf{n}|| (4) This means that if we had to calculate the average SI-SDR of the signal and the noise with respect to the sources with bleed, the error would be significant. This error will be large when compared to calculating it with respect to "true" sources, which are inaccessible. It is also important to note that this was based on the assumption that all were in the same subspace, but that is never true in real scenarios, resulting in increased error. Hence, the Saraga dataset cannot be used as accurate ground truth data for supervised source separation models for both training and especially evaluation, hindering the development of such models for Carnatic Music. As a workaround, some have attempted using source separation models like Spleeter [ 13 ] , presumably trained on a few or no Carnatic Music examples [ 9 ] , directly on the vocal multi-track with bleeding for certain MIR tasks [ 8 , 10 ] . However, attempts toward source separation for Carnatic using the currently available datasets have been made [ 2 , 9 , 14 ] . The stems obtained for Western Music datasets [ 3 , 4 , 5 , 6 ] are all from studio recordings, recorded separately and mixed, resulting in zero bleeds of other instruments in the multi-tracks. This allows for evaluation metrics such as SDR, Scale-Invariant SDR [ 11 ] , Signal-to-Aritfacts Ratio, Signal-to-Interference Ratio (SIR), etc., to be used without problems. However, there is no such available dataset for Carnatic Music [ 2 , 9 ] , and since it is a live tradition, it is impossible to record the artists at separate times. There have been a lot of datasets for Carnatic Music and Hindustani Music, which provide clean studio-quality data for individual instruments [ 15 , 16 ] . However, there have been no completely isolated full live concert recordings of studio quality. To directly address this requirement, we present a new dataset of well-isolated multi-track recordings of Carnatic Music: Sanidha . The Sanidha dataset features audio and video recordings of Carnatic

Read the complete text