Dissecting Performance Degradation in Audio Source Separation under Sampling Frequency Mismatch
Kanami Imamura, Tomohiko Nakamura, Kohei Yatabe, Hiroshi Saruwatari · 2026
Read Dissecting Performance Degradation in Audio Source Separation under Sampling Frequency Mismatch by Kanami Imamura, Tomohiko Nakamura, Kohei Yatabe, Hiroshi Saruwatari (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-14684.txt.
Introduction
Kanami Imamura, Tomohiko Nakamura, Kohei Yatabe, and Hiroshi Saruwatari study audio source separation in this 2026 paper. Source separation splits a mixed recording into its parts, such as one voice among many. Deep neural network methods for this task are typically trained at a single sampling frequency. To handle untrained frequencies, the input is usually resampled.
Resampling can degrade performance. The harm is worst when the input sampling frequency is lower than the trained one. The team dissects that performance degradation under sampling frequency mismatch. Readers get a careful breakdown of where the loss comes from, a step toward separation models that handle real-world files of mixed rates.
Catalog record
| CREATOR | Kanami Imamura, Tomohiko Nakamura, Kohei Yatabe, Hiroshi Saruwatari |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-14684
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dissecting Performance Degradation in Audio Source Separation under Sampling Frequency Mismatch Kanami Imamura, Tomohiko Nakamura, Kohei Yatabe, Hiroshi Saruwatari Abstract Audio processing methods based on deep neural networks are typically trained at a single sampling frequency (SF). To handle untrained SFs, signal resampling of the input signal is commonly employed, but it can degrade performance, particularly when the input SF is lower than the trained SF. This paper investigates the causes of this degradation through two hypotheses: (i) the lack of high-frequency components introduced by up-sampling, and (ii) the greater importance of their presence than their spectral contents. To examine these hypotheses, we compare conventional resampling with three alternatives: post-resampling noise addition, which adds Gaussian noise to the resampled signal; noisy-kernel resampling, which perturbs the kernel with Gaussian noise to enrich high-frequency components; and trainable-kernel resampling, which adapts the interpolation kernel through training. Experiments on music source separation show that noisy-kernel and trainable-kernel resampling alleviate the degradation observed with conventional resampling. We further demonstrate that noisy-kernel resampling is effective across diverse models, highlighting it as a simple yet practical option to alleviate the performance degradation. † † address: 1 The University of Tokyo, Japan 2 The National Institute of Advanced Industrial Science and Technology (AIST), Japan 3 Tokyo University of Agriculture and Technology, Japan 1 Introduction Audio signal processing methods based on deep neural networks (DNNs) are typically trained and evaluated at a single sampling frequency (SF). To handle untrained SFs, the common solution is to resample the input signal to the trained SF, process it by the DNN, and resample the output back to the original SF. Recent studies have shown that such resampling can degrade performance in DNN-based audio source separation methods, particularly when the input SF is lower than the trained SF [ 12 , 3 , 18 , 19 ] . These studies mainly focus on specific models and conditions. Thus, a comprehensive analysis is missing. As a result, the mechanism behind this degradation remains unclear, which motivates a systematic study of its causes and remedies. To address this issue, we consider two hypotheses about the cause of the performance degradation. The first hypothesis is that the lack of high-frequency components caused by up-sampling leads to performance degradation. When the input SF is lower than the trained SF, up-sampling produces signals whose frequency components above the input Nyquist frequency are set to zero. This absence of high-frequency components mismatches the training distribution and can impair performance. The second hypothesis is that the presence of high-frequency components contributes more to separation performance than their spectral contents . DNN-based source separation models generally focus on low-frequency components, with coarse resolution at high frequencies. For example, in Conv-TasNet, the encoder allocates more channels to process the low-frequency components [ 7 ] . Similar trends appear in its extensions [ 2 , 12 , 4 ] . Furthermore, the band-split recurrent neural network (BSRNN) [ 8 ] explicitly assigns finer resolution to low-frequency bands and coarser resolution to high-frequency bands. These models potentially place less emphasis on high-frequency components. Despite the necessity of their presence, their spectral contents may be less critical. To examine these hypotheses, we introduce three resampling methods and compare them with the conventional resampling. The first method adds random noise directly to the resampled signal to supplement missing high-frequency components. The second method perturbs the interpolation kernel with random noise, which enriches the high-frequency components of the resampled signal. The third method parameterizes the interpolation kernel with a DNN. While the original separation network is kept frozen, only the network representing the kernel is trained so that the resampling process adapts to the model and mitigates performance degradation. Through music source separation experiments, we demonstrate that our two hypotheses are supported and that noisy-kernel resampling consistently mitigates degradation across diverse models. (a) Processing pipeline of DNN handling untrained SF with resampling. (b) Post-resampling noise addition. (c) Noisy-kernel resampling. (d) Trainable-kernel resampling. Figure 1 : Overview of processing pipeline and kernels in proposed resampling methods. 2 Related Works 2.1 Interpolation-Based Resampling Interpolation-based resampling is one of the most widely used approaches. Let (x[n])_{n=0}^{N-1} denote the input signal of length N sampled at the SF F_{\text{s}}^{\text{(in)}} . After resampling to the SF F_{\text{s}}^{\text{(out)}} , we obtain (y[m])_{m=0}^{M-1} of length M=\lfloor F_{\text{s}}^{\text{(out)}}N/F_{\text{s}}^{\text{(in)}}\rfloor , where \lfloor\cdot\rfloor denotes the floor function: y[m]=\sum_{n=0}^{N-1}x[n]\>k\left(\dfrac{m}{F_{\text{s}}^{\text{(out)}}}-\dfrac{n}{F_{\text{s}}^{\text{(in)}}};F_{\text{s}}^{\text{(in)}}\right), (1) where k(t,F_{\text{s}}) is the interpolation kernel. A common choice is the windowed sinc function, k_{\text{(win-sinc)}}(t;F_{\text{s}})=z(t)\text{sinc}(F_{\text{s}}t),\quad\text{sinc}(t)=\dfrac{\sin(\pi t)}{\pi t}, (2) with t\in\mathbb{R} denoting continuous time and z(t) a window function. We use a non-negative smooth window z(t) and assume z(t)=0 for t<-L/(2F_{\text{s}}) or t>L/(2F_{\text{s}}) , where L is a positive integer. The Kaiser window is a typical choice. The window length L controls the trade-off between interpolation accuracy and computational cost. Since the sinc function acts as an ideal low-pass filter with cutoff frequency F_{\text{s}}/2 , the windowed sinc kernel suppresses frequency components above the Nyquist frequency. 2.2 Impact of Resampling on DNN-Based Methods As described in Section 1 , the performance degradation due to signal resampling has been reported. However, it has been examined only in a few studies. In [ 12 ] , a Conv-TasNet adaptation for music source separation was evaluated under different SFs, and degradation was observed especially when the input SF was lower than the trained SF. In related work on speech enhancement, [ 18 ] evaluated resampling effects on the performance of BSRNN. In addition, [ 19 ] reported results across different SFs for models submitted to the speech enhancement competition. The reported impact varied across architectures, and degradation was not always but unpredictably observed. In [ 1 ] , resampling effects were also analyzed, but the scope was limited to recurrent neural networks for neural audio effect processing. 3 Methods To examine the hypotheses described in Section 1 , we introduce three resampling methods. Figure 1 shows an overview of the methods. We denote the input and trained sampling frequencies by F_{\text{s}}^{\prime} and F_{\text{s}} , respectively, and focus on the case F_{\text{s}}^{\prime}<F_{\text{s}} , where degradation due to resampling has been observed. (e) vocals (f) bass (g) drums (h) other Figure 2 : SDRs of conventional resampling, post-resampling noise addition, noisy-kernel resampling, and trainable-kernel resampling. 3.1 Post-Resampling Noise Addition The first method is a simple baseline that directly adds Gaussian noise to the signal obtained after conventional resampling. The output signal is given as y_{\text{(noise)}}[m]=\sum_{n=0}^{N}x[n]\>k_{\text{(win-sinc)}}\left(\dfrac{m}{F_{\text{s}}}-\dfrac{n}{F_{\text{s}}^{\prime}};F_{\text{s}}^{\prime}\right)+\eta_{\sigma}[m], (3) where \eta_{\sigma}[m] is zero-mean Gaussian noise with variance \sigma^{2} , sampled independently at each time index. This operation injects energy across the entire frequency range and supplements the components above the Nyquist frequency. 3.2 Noisy-Kernel Resampling The second method modifies the conventional windowed-sinc interpolation kernel by adding Gaussian noise. In this method, Gaussian noise is added to a discretized kernel in the interpolation computation. Let r denote the discrete-time index of the kernel. The kernel k_{\text{(noisy)}}[r;F_{\text{s}}^{\prime},\sigma] is defined as k_{\text{(noisy)}}[r;F_{\text{s}}^{\prime},\sigma]=k_{\text{(win-sinc)}}[r;F_{\text{s}}^{\prime}]+\eta_{\sigma}[r]. (4) The frequency response of k_{\text{(noisy)}}[r;F_{\text{s}}^{\prime},\sigma] is the sum of the responses of k_{\text{(win-sinc)}}[r;F_{\text{s}}^{\prime}] and \eta_{\sigma}[r] . Consequently, compared with the conventional kernel, k_{\text{(noisy)}}[r;F_{\text{s}}^{\prime},\sigma] shows a larger magnitude in the frequency band above the cutoff frequency F_{\text{s}}^{\prime}/2 . The strength of these high-frequency components depends on \sigma . 3.3 Trainable-Kernel Resampling The third method is the only data-driven