LipsAM: Lipschitz-continuous Amplitude Modifier for Audio Signal Processing and Its Application to Plug-and-Play Dereverberation

Kazuki Matsumoto, Ren Uchida, Kohei Yatabe · 2026

Read LipsAM: Lipschitz-continuous Amplitude Modifier for Audio Signal Processing and Its Application to Plug-and-Play Dereverberation by Kazuki Matsumoto, Ren Uchida, Kohei Yatabe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21684.txt.

Introduction

Matsumoto and co-authors make a common audio network block provably stable. Amplitude modifiers are a standard design for audio signals, but their robustness had no Lipschitz proof. Lipschitz continuity caps how much an output can move when the input moves a little. The authors prove a condition that makes an amplitude modifier Lipschitz. They give two designs, called LipsAM, that meet it. Dropped into a speech dereverberation method, the new blocks improved stability in numeric tests, they report.

Catalog record

CREATORKazuki Matsumoto, Ren Uchida, Kohei Yatabe
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27129
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.21684

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LipsAM: Lipschitz-continuous Amplitude Modifier for Audio Signal Processing and Its Application to Plug-and-Play Dereverberation Kazuki Matsumoto, Ren Uchida, Kohei Yatabe Abstract The robustness of deep neural networks (DNNs) can be certified through their Lipschitz continuity, which has made the construction of Lipschitz-continuous DNNs an active research field. However, DNNs for audio processing have not been a major focus due to their poor compatibility with existing results. In this paper, we consider the amplitude modifier (AM), a popular architecture for handling audio signals, and propose its Lipschitz-continuous variants, which we refer to as LipsAM . We prove a sufficient condition for an AM to be Lipschitz continuous and propose two architectures as examples of LipsAM. The proposed architectures were applied to a Plug-and-Play algorithm for speech dereverberation, and their improved stability is demonstrated through numerical experiments. † † address: Tokyo University of Agriculture and Technology, Tokyo, Japan 1 Introduction Theoretical guarantees of the robustness and reliability of deep neural networks (DNNs) have been an active research topic. Certifying robustness against adversarial examples [ 31 , 5 ] and the stability analysis from a dynamic system perspective [ 4 , 3 ] are some examples in this line of research. One powerful tool for ensuring robustness is the Lipschitz continuity of DNNs. Several approaches have been proposed to design Lipschitz-bounded DNNs, e.g., projection [ 19 , 24 , 15 ] , regularization [ 32 , 21 ] , and structurally 1-Lipschitz layers [ 16 , 14 , 30 , 22 , 18 , 6 ] . Such techniques have been applied to design DNN architectures, including recurrent neural networks [ 7 ] , self-attention [ 12 ] , and transformers [ 23 ] . Computing the Lipschitz constant of a trained DNN is another important subject [ 25 , 11 ] . Although interest in Lipschitz-continuous DNNs has grown across many fields, audio signal processing has benefited less from these advances. This is mainly because typical architectures for audio signals are often incompatible with existing theoretical frameworks. For example, DNN-based time-frequency-domain processing, which transforms audio signals using the short-time Fourier transform (STFT) and then processes the obtained spectrograms, often modifies only the amplitude of the spectrograms [ 20 , 8 ] . In this case, the entire system is generally not Lipschitz continuous, even when the DNN that modifies the amplitude is Lipschitz continuous. To ensure Lipschitz continuity in such audio-specific architectures, additional constraints or tailored designs are required. In this paper, we focus on a DNN that modifies the amplitude of the spectrogram, which we call the amplitude modifier (AM), and we prove a sufficient condition for an AM to be Lipschitz continuous. We also propose two Lipschitz-continuous AMs according to the obtained condition and name them LipsAM . Our contribution can be summarized as follows: (1) We prove the condition for Lipschitz-continuous AM; (2) We propose two Lipschitz-continuous AMs; (3) We provide the bounds on their Lipschitz constants; and (4) We apply the proposed DNN architectures to a Plug-and-Play (PnP) method [ 2 , 24 , 21 ] for speech dereverberation. Notation: \mathbb{R}_{+} , \mathbb{R} , and \mathbb{C} denote the sets of nonnegative, real, and complex numbers, respectively, and \mathbb{F}\in\{\mathbb{R},\mathbb{C}\} . Lower bold letters represent vectors, e.g., \mathbf{z}=(z_{n})_{n=1}^{N}\in\mathbb{C}^{N} , and capital bold letters denote matrices, e.g., \mathbf{A}=(a_{ij})_{i=1,\,j=1}^{I,\,J}\in\mathbb{C}^{I\times J} . 2 Preliminary 2.1 Lipschitz Continuity of DNNs A mapping f:\mathbb{F}^{N}\to\mathbb{F}^{N} is said to be L -Lipschitz continuous if there exists a constant L\geq 0 such that \displaystyle\forall\mathbf{x},\ \mathbf{y}\in\mathbb{F}^{N},\quad\|f(\mathbf{x})-f(\mathbf{y})\|_{2}\leq L\|\mathbf{x}-\mathbf{y}\|_{2}, (1) and the smallest L is called the Lipschitz constant , \mathrm{Lip}(f) , given by \mathrm{Lip}(f)=\sup_{\mathbf{x}\neq\mathbf{y}\in\mathbb{F}^{N}}\left(\frac{\|f(\mathbf{x})-f(\mathbf{y})\|_{2}}{\|\mathbf{x}-\mathbf{y}\|_{2}}\right), (2) where \|\cdot\|_{2} is the \ell_{2} -norm. Lipschitz continuity and smallness of the Lipschitz constant indicate robustness to the input perturbations. Many theoretical guarantees of DNNs rely on the assumption of their Lipschitz continuity [ 31 , 5 , 4 , 3 , 24 , 21 ] , and several techniques have been proposed to construct Lipschitz-continuous DNNs [ 19 , 24 , 15 , 32 , 21 , 16 , 14 , 30 , 22 , 18 , 6 , 7 , 12 , 23 ] . Figure 1: Amplitude-Modifying DNNs. Red layers are trainable, blue layers are introduced to enforce Lipschitz continuity. Layers “mul” represent the element-wise multiplication. While the popular architectures, AM-SE and AM-RE, are generally not Lipschitz continuous, Lipschitz constants of our LipsAM-SE and LipsAM-RE can be bounded by \sqrt{(\mathrm{Lip}(\mathcal{S}))^{2}+1} and \mathrm{Lip}(\mathcal{R})+1 , respectively. 2.2 Amplitude Modifier (AM) In audio applications, signals are often processed in the STFT domain. Let a signal in the STFT domain be vectorized and denoted as \mathbf{z}\in\mathbb{C}^{N} . Typical DNN architectures process this signal by modifying its amplitude, |\mathbf{z}|\in\mathbb{R}_{+}^{N} , while the phase-related components, \mathrm{sign}(\mathbf{z})\in\mathbb{C}^{N} , remain intact, where the absolute value and the complex-valued sign function ( \mathrm{sign}(z):=z/|z|=\exp(\mathrm{i}\,\arg z) if z\neq 0 and 0 otherwise) are applied element-wise [ 20 , 8 ] . Here, we call such a DNN Amplitude Modifier (AM) : \text{AM :}\quad\mathcal{D}_{\mathcal{A}}(\mathbf{z})=\mathcal{A}(|\mathbf{z}|)\odot\mathrm{sign}(\mathbf{z}), (3) where \mathcal{A}:\mathbb{R}_{+}^{N}\to\mathbb{R}_{+}^{N} is the DNN trained for processing, and \odot denotes the element-wise multiplication. In this paper, we consider two types of AM. The signal estimator (AM-SE), \mathcal{D}_{\mathcal{S}} , directly estimates the amplitude spectrogram of the clean signal using \mathcal{S}:\mathbb{R}_{+}^{N}\to\mathbb{R}^{N} , and the residual estimator (AM-RE), \mathcal{D}_{\mathcal{R}} , estimates the residual component using \mathcal{R}:\mathbb{R}_{+}^{N}\to\mathbb{R}^{N} and then subtracts it from the input, as follows: AM-SE : \displaystyle\mathcal{D}_{\mathcal{S}}(\mathbf{z})=(\mathcal{S}(|\mathbf{z}|))_{+}\odot\mathrm{sign}(\mathbf{z}), (4) AM-RE : \displaystyle\mathcal{D}_{\mathcal{R}}(\mathbf{z})=\mathrm{(}|\mathbf{z}|-\mathcal{R}(|\mathbf{z}|))_{+}\odot\mathrm{sign}(\mathbf{z}), (5) where (\cdot)_{+}=\max(\cdot,0) represents ReLU. With existing approaches for constructing Lipschitz continuous DNNs, the mappings \mathcal{S} and \mathcal{R} can be readily designed to be Lipschitz continuous. Nevertheless, the entire systems \mathcal{D}_{\mathcal{S}} and \mathcal{D}_{\mathcal{R}} are generally not Lipschitz continuous. To ensure Lipschitz continuity of AMs, a special technique is required, which we introduce next. 3 Proposed Method In this section, we establish a sufficient condition that guarantees the Lipschitz continuity of AMs. We then propose Lipschitz-continuous variants for AM-SE and AM-RE and derive theoretical bounds on their Lipschitz constants. To emphasize the necessity of our method, we first show that AMs are generally not Lipschitz continuous. 3.1 AMs are not Lipschitz Continuous The AMs \mathcal{D}_{\mathcal{A}} , \mathcal{D}_{\mathcal{S}} and \mathcal{D}_{\mathcal{R}} in Eqs. ( 3 )–( 5 ) are not Lipschitz continuous in general, even when the amplitude-modifying parts \mathcal{A} , \mathcal{S} and \mathcal{R} are Lipschitz continuous. We demonstrate this fact through the following two examples: Example 1 (Bias) . The mapping \mathcal{A}:\mathbb{R}_{+}\to\mathbb{R}_{+}:x\mapsto x+1 is 1-Lipschitz continuous. However, \mathcal{D}_{\mathcal{A}}:z\mapsto(|z|+1)\cdot\mathrm{sign}(z) is not Lipschitz continuous. 1 1 1 Let z\in\mathbb{C} such that |z|=\epsilon>0 and let w=-z , then |\mathcal{D}_{\mathcal{A}}(z)-\mathcal{D}_{\mathcal{A}}(w)|/|z-w|=2(\varepsilon+1)/(2\varepsilon)\to+\infty with \varepsilon\to+0 . Example 2 (Permutation) . The mapping \mathcal{A}:\mathbb{R}_{+}^{2}\to\mathbb{R}_{+}^{2}:(x_{1},x_{2})\mapsto(x_{2},x_{1}) is 1-Lipschitz continuous. However, \mathcal{D}_{\mathcal{A}}:\mathbf{z}\mapsto\mathcal{{A}}(|\mathbf{z}|)\odot\mathrm{sign}(\mathbf{z}) is not Lipschitz continuous. 2 2 2 Let \mathbf{z}=(\varepsilon,1)\in\mathbb{C}^{2} , \mathbf{w}=(-\varepsilon,1)\in\mathbb{C}^{2} , where \varepsilon>0 . Then \|\mathcal{D}_{\mathcal{A}}(\mathbf{z})-\mathcal{D}_{\mathcal{A}}(\mathbf{w})\|_{2}/\|\mathbf{z}-\mathbf{w}\|_{2}=2/(2\varepsilon)\to+\infty with \varepsilon\to+0 . These examples lead to the following question: How can we impose Lipschitz continuity on AMs? We provide an answer to this question in the following subsection. 3.2 LipsAM: Lipschitz-continuous Amplitude Modifier Aiming to provide a tool to enforce Lipschitz continuity for AMs, we introduce some assumptions on the mapping \mathcal{A} in Eq. ( 3 ): Assumption 3 . For any \mathbf{x} , \mathbf{y}\in\mathbb{R}_{+}^{N} and for all n\in\{1,\ \ldots,N\} , the mapping \mathcal{A}:\mathbb{R}_{+}^{N}\to\mathbb{R}_{+}^{N} satisfies the following two conditions with some constants L_{1},L_{2}\geq 0 . 1. \mathcal{A} is L_{1} -Lipschitz continuous, i.e.,

Read the complete text