Drum-to-Vocal Percussion Sound Conversion and Its Evaluation Methodology
\authorblockN Rinka Nobukawa \authorrefmark 1 \authorrefmark 2 \authorrefmark 4, Makito Kitamura \authorrefmark 1, Tomohiko Nakamura \authorrefmark 2, Shinnosuke Takamichi \authorrefmark 3 \authorrefmark 1 \authorrefmark 2 and Hiroshi Saruwatari \authorrefmark 1 \authorblockA \authorrefmark 1 Gradua · 2025
Read Drum-to-Vocal Percussion Sound Conversion and Its Evaluation Methodology by \authorblockN Rinka Nobukawa \authorrefmark 1 \authorrefmark 2 \authorrefmark 4, Makito Kitamura \authorrefmark 1, Tomohiko Nakamura \authorrefmark 2, Shinnosuke Takamichi \authorrefmark 3 \authorrefmark 1 \authorrefmark 2 and Hiroshi Saruwatari \authorrefmark 1 \authorblockA \authorrefmark 1 Gradua (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-16862.txt.
Introduction
Rinka Nobukawa and co-authors turn drum sounds into vocal percussion in a 2025 audio paper. Singers in a cappella groups often mimic drums with their voices. That art is called vocal percussion. The team define a new task: converting a drum track into a vocal version of the same rhythm.
They build two baseline converters with a neural sound maker. One adds a quantization step, the other does not. Listeners judged both outputs plausible. The quantized version gave more consistent results. A shared task and test method could spur better tools for music makers who sketch beats with drums but perform them by voice.
Catalog record
| CREATOR | \authorblockN Rinka Nobukawa \authorrefmark 1 \authorrefmark 2 \authorrefmark 4, Makito Kitamura \authorrefmark 1, Tomohiko Nakamura \authorrefmark 2, Shinnosuke Takamichi \authorrefmark 3 \authorrefmark 1 \authorrefmark 2 and Hiroshi Saruwatari \authorrefmark 1 \authorblockA \authorrefmark 1 Gradua |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-16862
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Drum-to-Vocal Percussion Sound Conversion and Its Evaluation Methodology \authorblockN Rinka Nobukawa \authorrefmark 1 \authorrefmark 2 \authorrefmark 4, Makito Kitamura \authorrefmark 1, Tomohiko Nakamura \authorrefmark 2, Shinnosuke Takamichi \authorrefmark 3 \authorrefmark 1 \authorrefmark 2 and Hiroshi Saruwatari \authorrefmark 1 \authorblockA \authorrefmark 1 Graduate School of Information Science and Technology, University of Tokyo, Tokyo, Japan \authorrefmark 2 National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan \authorrefmark 3 Faculty of Science and Technology, Keio University, Yokohama, Japan \authorrefmark 4 E-mail: [email protected] Abstract This paper defines the novel task of drum-to-vocal percussion (VP) sound conversion. VP imitates percussion instruments through human vocalization and is frequently employed in contemporary a cappella music. It exhibits acoustic properties distinct from speech and singing (e.g., aperiodicity, noisy transients, and the absence of linguistic structure), making conventional speech or singing synthesis methods unsuitable. We thus formulate VP synthesis as a timbre transfer problem from drum sounds, leveraging their rhythmic and timbral correspondence. To support this formulation, we define three requirements for successful conversion: rhythmic fidelity , timbral consistency , and naturalness as VP . We also propose corresponding subjective evaluation criteria. We implement two baseline conversion methods using a neural audio synthesizer, the real-time audio variational autoencoder (RAVE), with and without vector quantization (VQ). Subjective experiments show that both methods produce plausible VP outputs, with the VQ-based RAVE model yielding more consistent conversion. 1 Introduction Vocal percussion (VP) is a vocal technique that emulates percussive instrument sounds using articulations of the human vocal tract. It has a cultural background, having been used as a means of transmitting drum sounds in drum dance in West Africa [ 1 ] and as a method of teaching the playing of tabla drums in Northern India [ 2 , 3 ] . VP is also widely employed in contemporary a cappella music, where any style of contemporary music is performed using only the human voice and/or body [ 4 ] . To computationally handle contemporary a cappella music, the analysis and synthesis of VP sounds is essential due to its central role in rhythm and timbre reproduction. In the context of music information retrieval, a few studies on VP sound analysis have been conducted; for example, rhythm-centric music information retrieval [ 5 ] , VP sound classification [ 6 ] , and music generation [ 7 ] . However, the synthesis aspect remains largely underexplored, despite its considerable potential to support practice and arrangement, particularly for novice vocalists. We thus address the problem of VP sound synthesis in this paper. Tackling the VP sound synthesis problem poses two major challenges. The first challenge is how to synthesize VP sounds. For voice parts with lyrics, we have previously proposed a method that synthesizes multi-part singing voices by allowing each part to refer the scores of the others [ 8 ] . This approach enhances inter-part synchrony and improves the naturalness of the synthesized ensemble singing voices. While such a method is effective for language-bearing vocal parts, VP does not convey linguistic contents and functions as a vocal surrogate for percussive instruments. It often involves articulatory movements outside the standard phonemic inventory, such as unvoiced trills, ingressive airstreams, and fricative bursts [ 9 ] . These features are not adequately captured by conventional phoneme-based synthesis methods. Therefore, VP sound synthesis requires a distinct modeling approach beyond standard speech synthesis. The second challenge is how to evaluate synthesized VP sounds. Conventional evaluation methods for synthesized audio include the mean opinion score [ 10 ] for naturalness, mel cepstral distortion [ 11 ] for spectral similarity, and metrics such as Fréchet audio distance and inception score [ 12 ] for generative audio quality. However, these evaluation methods are not directly applicable to VP synthesis. Unlike speech or melodic music signals, VP events are typically short, aperiodic, and spectrally noisy, making conventional metrics poorly suited to capture their perceptual qualities. To accurately assess the quality of synthesized VP sounds, a dedicated evaluation framework that accounts for these perceptual attributes is therefore necessary. Figure 1 : Conceptual illustration of drum-to-VP sound conversion. “RQ” denotes a research question. To address the aforementioned challenges in synthesis and evaluation, we define a new task: the drum-to-VP conversion task, which aims to convert drum sounds into the corresponding VP renditions (see fig. 1 ). In this task, we frame the VP synthesis of our interest not as a conventional speech synthesis problem but as a timbre transfer problem from drum sounds to VP sounds. This framing is expected to enable natural acoustic transformations while maintaining rhythmic consistency. As a baseline model, we adopt the real-time audio variational autoencoder (RAVE) [ 13 ] , a generative model capable of low-latency waveform conversion. In our setting, RAVE is trained on VP sounds and applied to transform input drum signals into their corresponding VP renditions. Furthermore, we introduce a dedicated evaluation framework tailored to VP synthesis. This framework consists of three perceptual criteria derived from requirements for our desired timbre transfer: rhythmic fidelity , timbral consistency , and naturalness as vocal percussion . These criteria inform the design of a structured subjective evaluation procedure for the drum-to-VP synthesis. 2 Related Works 2.1 Vocal Percussion and Human Beatboxing VP and human beatboxing are often conflated but have distinct functional and cultural contexts. While beatboxing is typically a solo performance technique that involves a broad range of sound effects, including non-percussive sounds, VP refers to mouth-generated percussive sounds performed in ensemble settings such as contemporary a cappella groups [ 14 ] . In these contexts, VP serves as a substitute for drums and is characterized by its faithful imitation of percussive patterns. A guidebook on contemporary a cappella singing [ 4 ] encourages vocal drummers, like their instrumental counterparts, to practice at a tempo where they can execute beats comfortably and accurately. This highlights the expectation that VP requires rhythmic control similar to that of real drums. Given this close functional relationship, drum-to-VP conversion can be regarded as a timbre transfer task rather than a symbolic-to-audio synthesis problem. 2.2 Acoustic Characteristics of Vocal Percussion Several acoustic analyses of VP have been conducted, particularly in relation to pronunciation corresponding to percussion among beatboxing [ 9 , 15 , 16 , 17 ] . VP sounds are typically generated through the vocal tract but differ from conventional speech in their articulatory and acoustic properties. Many VP sounds are aperiodic, unvoiced, and rich in turbulent noise components [ 9 ] . This fact suggests prioritizing the magnitude spectrogram over the phase for VP sound synthesis. Although some VP sounds can be transcribed using the international phonetic alphabet (IPA) [ 15 ] , especially those produced by less experienced performers, the most skilled VP sounds often defy phonetic categorization [ 9 , 16 , 17 ] . In such cases, articulatory descriptions highlight intentional deviations from speech-like gestures. These findings suggest that phoneme-conditioned synthesis is insufficient and support the use of audio-to-audio models to capture the non-linguistic, instrument-like nature of VP. 3 Proposed Task: Drum-to-VP Sound Conversion 3.1 Task Definition In this section, we frame the VP sound synthesis task as a problem of converting drum sounds into VP sounds, i.e., the drum-to-VP sound conversion task. As discussed in