SELEBI : Percussion-aware Time Stretching via Selective Magnitude Spectrogram Compression by Nonstationary Gabor Transform

Natsuki Akaishi, Nicki Holighaus, Kohei Yatabe · 2026

Read SELEBI : Percussion-aware Time Stretching via Selective Magnitude Spectrogram Compression by Nonstationary Gabor Transform by Natsuki Akaishi, Nicki Holighaus, Kohei Yatabe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-16421.txt.

Introduction

Natsuki Akaishi, Nicki Holighaus, and Kohei Yatabe introduce SELEBI, a signal-adaptive phase vocoder for percussion-aware stretching. Phase vocoder time-stretching is widely used to change audio time scales, but standard implementations suffer smearing, an artifact that badly degrades percussive sounds. The authors trace it to a time-scale mismatch between the smeared magnitude spectrogram and the new phase.

SELEBI reduces that smearing while keeping stability, and unlike rule-based processing or component splitting, it uses the nonstationary Gabor transform. By varying analysis window lengths, giving short windows to parts with strong transient energy, it computes a localized spectrogram directly from the time-domain signal. That keeps phase structures more consistent. Its perfect reconstruction property guarantees stable, high-fidelity synthesis, unlike earlier approaches. Experiments show the method mitigates smearing and yields natural sound quality.

Catalog record

CREATORNatsuki Akaishi, Nicki Holighaus, Kohei Yatabe
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-16421

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SELEBI : Percussion-aware Time Stretching via Selective Magnitude Spectrogram Compression by Nonstationary Gabor Transform Natsuki Akaishi, Nicki Holighaus, Kohei Yatabe Abstract Phase vocoder-based time-stretching is a widely used technique for the time-scale modification of audio signals. However, conventional implementations suffer from “percussion smearing,” a well-known artifact that significantly degrades the quality of percussive components. We attribute this artifact to a fundamental time-scale mismatch between the temporally smeared magnitude spectrogram and the localized, newly generated phase. To address this, we propose SELEBI , a signal-adaptive phase vocoder algorithm that significantly reduces percussion smearing while preserving stability and the perfect reconstruction property. Unlike conventional methods that rely on heuristic processing or component separation, our approach leverages the nonstationary Gabor transform. By dynamically adapting analysis window lengths to assign short windows to intervals containing significant energy associated with percussive components, we directly compute a temporally localized magnitude spectrogram from the time-domain signal. This approach ensures greater consistency between the temporal structures of the magnitude and phase. Furthermore, the perfect reconstruction property of the nonstationary Gabor transform guarantees stable, high-fidelity signal synthesis, in contrast to previous heuristic approaches. Experimental results demonstrate that the proposed method effectively mitigates percussion smearing and yields natural sound quality. I Introduction Time stretching, a process that modifies the time-scale of a signal without altering its pitch, is a fundamental tool in modern music production, with applications ranging from audio remixing to transcription [ 16 , 27 ] . The ideal goal is to obtain a time-stretched signal that perfectly preserves the musical characteristics of the original, such as its timbre, clarity, and dynamics. To achieve high-quality results, a wide variety of time-stretching methods have been proposed [ 11 , 22 , 8 , 29 , 19 , 14 , 21 , 6 , 20 , 9 , 28 , 26 ] . The phase vocoder (PV) [ 11 , 22 ] is one of the most widely used techniques for time-stretching. This approach operates in the time-frequency (T-F) domain: It computes the short-time Fourier transform (STFT), generates a new phase appropriate for the modified time-scale, and resynthesizes the signal using the overlap-add (OLA) technique with an extended hop size. Although the PV is a well-established technique that has been continuously refined [ 8 , 6 ] , its fundamental logic builds on the sinusoidal signal model, i.e., it assumes that the signal is a sum of sinusoids. This model is effective for tonal content because time-shifts are represented by a well-defined, predictable phase shift. However, it is ill-suited for percussive components, which cannot be represented by a sum of few sinusoids. Therefore, time-shifting of percussive components cannot be modeled as such a phase shift. As a result, the PV often introduces a well-known artifact known as percussion smearing . As illustrated in Fig. 1 (top-right), the stretched percussive components suffer significant degradation, losing their original temporal characteristics due to undesired leakage. To alleviate this, various “percussion-aware” methods have been proposed. However, they either rely on an a priori separation of the signal into tonal and percussive components for individual processing [ 6 , 20 ] (Class A), which invariably introduces artifacts due to imperfect separation, or they suffer, to various degrees, from magnitude-phase mismatch [ 21 , 9 , 28 , 26 ] (Class B), inhibiting the elimination of percussion smearing. Fig. 1: Block diagrams of the basic method, PV with identity phase locking [ 18 ] (left), and the proposed method (right). By leveraging NSDGT , the proposed method synthesizes the target signal from T-F representations in which both the magnitude and phase spectrograms of percussive components are highly concentrated in the time direction. In this paper, we address the fundamental limitation of Class B methods 1 1 1 Our approach does not rely on signal separation, thereby avoiding the artifacts associated with Class A methods. This allows us to focus solely on the limitations of Class B methods. : the inconsistency between magnitude and phase. While these methods often succeed in preserving the phase relationships required for transients (i.e., vertical phase coherence), the corresponding magnitude spectrograms are inevitably smeared due to the time-stretching process. This results in percussive components being spread across a dilated time interval, which contradicts the localized phase information. We posit that this mismatch is the primary cause of percussion smearing, as illustrated in Fig. 1 (left). To address this, we propose computing spectrograms with improved temporal localization specifically where percussive components are present, thereby aligning the magnitude representation with the localized phase (Fig. 1 , right). While our previous work [ 2 ] attempted to mitigate this issue through heuristic time-frequency bin shifting, the proposed method provides a theoretical foundation for energy preservation and stable synthesis. To achieve this magnitude squeezing in a mathematically rigorous manner, this paper proposes a method named SELEBI ( SELE ctive window compression with sta B le I nversion). Our method leverages the Nonstationary Gabor Transform ( NSDGT ) [ 3 ] , a time-frequency representation framework that allows for adaptive, non-uniform windowing and sampling while retaining perfect reconstruction properties. We exploit this flexibility by adaptively assigning shorter windows to percussive regions, thereby directly obtaining a magnitude spectrogram with a desired temporal resolution. Crucially, the underlying mathematical framework of the NSDGT guarantees that this adaptive processing results in a stable and high-fidelity synthesis. The rest of the paper is organized as follows. Section II reviews the fundamentals of the NSDGT and PV-based time stretching. Section III discusses the core component of the proposed method: Sharply resolving percussive events to reduce percussion smearing in time-stretched audio. Section IV presents the proposed SELEBI algorithm. Section V discusses the feasibility of a bounded-delay implementation for on-line applications. Section VI provides experimental results validating our method, and Section VI concludes this paper. II Preliminaries II-A Notation The sets of natural numbers, integers, real numbers, and complex numbers are denoted by \mathbb{N} , \mathbb{Z} , \mathbb{R} , and \mathbb{C} , respectively. Matrices are denoted by bold capital letters (e.g., \mathbf{A} ), and their element at the i -th row and j -th column is denoted by A[i,j] . Vectors are denoted by bold lower-case letters (e.g., \mathbf{v} ), and their i -th element is denoted by v[i] . For the purpose of this paper, we identify the cyclic group of order N , denoted by \mathbb{Z}_{N} , with the integers 0,\ldots,N-1 . The following operations are always considered pointwise: \odot (multiplication) \oslash (division), |\cdot| (modulus / absolute value), and \text{Arg}(\cdot) (complex argument). A sequence indexed by k\in\mathscr{K}\subset\mathbb{N} is denoted by \{\cdot_{k}\}_{k\in\mathscr{K}} . The floor, ceiling, and nearest-integer (rounding) functions are denoted by \lfloor\cdot\rfloor , \lceil\cdot\rceil , and \lfloor\cdot\rceil , respectively. The Frobenius norm is denoted by \|\cdot\|_{\text{F}} . II-B Discrete Gabor Transform (DGT) The DGT coefficients \mathbf{X}\in\mathbb{C}^{M\times N} of a signal \mathbf{x}\in\mathbb{R}^{L} with a window function \mathbf{g}\in\mathbb{R}^{L} are defined as [ 13 ] X[m,n]=\sum^{L-1}_{l=0}x[l]\,g[l-an]\,\text{e}^{-\text{i}2\pi m(l-na)/M}, (1) where \text{i}=\sqrt{-1} , a\in\mathbb{N} is the hop size, l\in\mathbb{Z}_{L} is the time index, n\in\mathbb{Z}_{N} is the time-frame index, m\in\mathbb{Z}_{M} is the frequency index. The signal length L is set to satisfy aN=bM=L , where b\in\mathbb{N} is the frequency decimation parameter. The inverse DGT (iDGT) is given by overlap-adding the individual, synthesized time frames: \widehat{x}[l]=\sum_{n=0}^{N-1}\widetilde{g}[l-na]\cdot\sum_{m=0}^{M-1}X[m,n]\,\text{e}^{\text{i}2\pi ml/M}, (2) where

Read the complete text