Local Equivariance Error-Based Metrics for Evaluating Sampling-Frequency-Independent Property of Neural Network

Kanami Imamura 1,2 , Tomohiko Nakamura 2 , Norihiro Takamune 1 , Kohei Yatabe 3 , and Hiroshi Saruwatari 1 · 2025

Read Local Equivariance Error-Based Metrics for Evaluating Sampling-Frequency-Independent Property of Neural Network by Kanami Imamura 1,2 , Tomohiko Nakamura 2 , Norihiro Takamune 1 , Kohei Yatabe 3 , and Hiroshi Saruwatari 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-03550.txt.

Introduction

Kanami Imamura, Tomohiko Nakamura, Norihiro Takamune, Kohei Yatabe, and Hiroshi Saruwatari wrote this research paper in two thousand twenty-five. It deals with audio tools that use deep neural networks. Such tools are most often trained at one sampling rate, so they need resampling when the rate changes.

The authors propose three new metrics that test how well a network holds up when the sampling rate changes. The metrics build on a measure of how much a network output shifts when its input is changed. Tests on music source separation show the metrics track the drop in quality at new rates well. The work gives a way to check this weak spot before a tool is put to use.

Catalog record

CREATORKanami Imamura 1,2 , Tomohiko Nakamura 2 , Norihiro Takamune 1 , Kohei Yatabe 3 , and Hiroshi Saruwatari 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-03550

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Local Equivariance Error-Based Metrics for Evaluating Sampling-Frequency-Independent Property of Neural Network Thanks: This work was supported by JSPS KAKENHI under Grant JP23K28108 and JST ACT-X under Grant JPMJAX210G. Kanami Imamura 1,2 , Tomohiko Nakamura 2 , Norihiro Takamune 1 , Kohei Yatabe 3 , and Hiroshi Saruwatari 1 Abstract Audio signal processing methods based on deep neural networks (DNNs) are typically trained only at a single sampling frequency (SF) and therefore require signal resampling to handle untrained SFs. However, recent studies have shown that signal resampling can degrade performance with untrained SFs. This problem has been overlooked because most studies evaluate only the performance at trained SFs. In this paper, to assess the robustness of DNNs to SF changes, which we refer to as the SF-independent (SFI) property, we propose three metrics to quantify the SFI property on the basis of local equivariance error (LEE). LEE measures the robustness of DNNs to input transformations. By using signal resampling as input transformation, we extend LEE to measure the robustness of audio source separation methods to signal resampling. The proposed metrics are constructed to quantify the SFI property in specific network components responsible for predicting time-frequency masks. Experiments on music source separation demonstrated a strong correlation between the proposed metrics and performance degradation at untrained SFs. I Introduction Deep neural networks (DNNs) have been widely used for various audio signal processing tasks [ 1 ] . In these tasks, DNNs are typically trained and evaluated only at a single sampling frequency (SF). To apply the trained models to signals with different SFs, signal resampling is usually required. Although resampling is well-grounded in signal processing theory [ 2 ] , recent studies have shown that it can degrade performance at untrained SFs in DNN-based audio source separation methods [ 3 , 4 , 5 , 6 , 7 ] . This observation underscores the need to assess not only separation performance at trained SFs but also how robust these methods are to SF changes—that is, how independent they are of SFs. We refer to this independence as the SFI property and address its evaluation in this paper. Although there are well-established evaluation metrics for separation performance [ 8 , 9 ] , there is currently no standard metric for evaluating the SFI property. For separation performance, source-to-distortion ratio (SDR) [ 8 ] and scale-invariant SDR (SI-SDR) [ 9 ] are commonly utilized in research on audio source separation literature [ 10 ] . Previous studies have measured the SFI property by calculating these metrics at various SFs [ 3 , 11 , 5 , 6 , 4 , 7 , 12 ] . However, while this evaluation is useful for comparing the performance of audio source separation methods at each test SF, it does not quantify how SFI a method is. Since SDR and SI-SDR depend on SFs by definition, their values at different SFs cannot be directly compared. Therefore, we need to construct an evaluation metric that quantifies the SFI property and explains the performance degradation at untrained SFs. In this paper, we propose evaluation metrics to quantify the SFI property by measuring the robustness of audio source separation methods to signal resampling. The proposed metrics are based on local equivariance error (LEE) [ 13 ] . Originally introduced for image processing tasks, LEE measures the robustness of DNNs to input transformations such as rotation and translation. Since signal resampling is also a form of input transformation, we extend LEE to quantify the SFI property. Through this extension, we find that directly applying LEE does not explain the performance degradation at untrained SFs, as discussed in Section IV-C . To resolve this issue, the proposed evaluation metrics target not the entire network but specifically the network components responsible for predicting time-frequency masks. Through experiments on music source separation, we will analyze the relationship between the proposed metrics and performance degradation at untrained SFs. II Related Works II-A Local Equivariance Error LEE is defined on the basis of equivariance , a property of a function where a transformation applied to the input space results in an equivalent transformation in the output space. Equivariance in DNNs has proven effective for tasks such as image line drawing and semantic segmentation [ 14 ] . Let f:\mathbb{V}\rightarrow\mathbb{V^{\prime}} be a function and x\in\mathbb{V} be its input sampled from a data distribution \mathcal{D} , where \mathbb{V} and \mathbb{V^{\prime}} are input and output vector spaces, respectively. We say that f is equivariant under the action of a group \mathcal{G} if the following equation holds for any \mathsf{g}\in\mathcal{G} : f(\tau(\mathsf{g})x)=\tau^{\prime}(\mathsf{g})(f(x)), (1) where \tau(\mathsf{g}):\mathbb{V}\rightarrow\mathbb{V} and \tau^{\prime}(\mathsf{g}):\mathbb{V^{\prime}}\rightarrow\mathbb{V^{\prime}} are the representations of \mathsf{g} for the input and output of f , respectively. For example, we consider the rotation group \mathcal{G} for images and a group element \mathsf{g}_{r} smoothly connected on \mathcal{G} by the parameterization of angle r . The realizations of \mathsf{g}_{r} in the input and output domains ( \tau(\mathsf{g}_{r}) and \tau^{\prime}(\mathsf{g}_{r}) ) are rotation matrices corresponding to angle r . When r=0 , they reduce to identity matrices, i.e., no rotation. For simplicity, we denote \tau(\mathsf{g}_{r}) and \tau^{\prime}(\mathsf{g}_{r}) as g_{r} and g^{\prime}_{r} . LEE measures the degree of equivariance in a DNN. We assume that g^{\prime}_{r} is invertible and define h_{r}(x)=g^{\prime-1}_{r}(f(g_{r}(x))) . The difference between h_{r}(x) and f(x) serves as a measure of deviation from equivariance. Taking the limit as r\to 0 , LEE is given as \displaystyle\text{LEE}(f) \displaystyle=\dfrac{1}{V}\,\mathbb{E}[||\mathcal{L}f(x)||^{2}], (2) \displaystyle\mathcal{L}f \displaystyle=\lim_{r\to 0}\dfrac{h_{r}-f}{r}=\left.\dfrac{d}{dr}h_{r}\right|_{r=0}, (3) where \mathbb{E} denotes the expectation with respect to x over the distribution \mathcal{D} and V is the output dimension of f(x) . \mathcal{L}f is called the Lie derivative of f , which measures the degree of change in f under an infinitesimal transformation. LEE was extended to approximately compute the contributions of specific layers using the chain rule of the Lie derivative [ 13 ] . We refer to this extension as layerwise LEE . Let f_{\text{NN}}(x)=(f_{I}\circ f_{I-1}\circ\ldots\circ f_{1})(x) be a DNN composed of I layers, where f_{i} (i=1,\ldots,I) denotes the function representing layer i . By applying the chain rule, the Lie derivative of f_{\text{NN}} can be decomposed as \mathcal{L}f_{\text{NN}}=\sum_{i=1}^{I}J_{I:i+1}\mathcal{L}f_{i}, (4) where J_{I:i+1} is the Jacobian matrix of f_{I}\circ\ldots\circ f_{i+1} and J_{I:I+1} is an identity matrix. For notational simplicity, we omit the input of each Lie derivative and Jacobian matrix. From eq. 4 , we derive the upper bound of the norm of \mathcal{L}f_{\text{NN}} : ||\mathcal{L}f_{\text{NN}}||\leq\sum_{i=1}^{I}||J_{I:i+1}\mathcal{L}f_{i}||. (5) Each term on the right-hand side can be interpreted as the contribution of an individual layer. The layerwise LEE of layer i is defined as the expectation of the term associated with f_{i} in eq. 5 and serves as a tool for analyzing the influence of each layer on LEE. See [ 13 ] for further details on LEE and layerwise LEE. II-B TasNet Architecture The TasNet architecture [ 15 ] is one of the foundational network architectures for DNN-based audio source separation and is widely utilized in literature [ 16 , 3 , 17 ] . It consists of an encoder f_{\text{enc}} , a decoder f_{\text{dec}} , and a mask predictor f_{\text{mask}} . The encoder and decoder serve as a trainable time-frequency transform and its inverse transform, respectively. The encoder transforms the input signal into a pseudo time-frequency representation,

Read the complete text