Towards Realistic Synthetic Data for Automatic Drum Transcription

Pierfrancesco Melucci, Paolo Merialdo, Taketo Akama, [1.0em] Sapienza University of Rome, Italy · 2026

Read Towards Realistic Synthetic Data for Automatic Drum Transcription by Pierfrancesco Melucci, Paolo Merialdo, Taketo Akama, [1.0em] Sapienza University of Rome, Italy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-09520.txt.

Introduction

Pierfrancesco Melucci, Paolo Merialdo, and Taketo Akama work on teaching machines to write down drum parts. Their 2026 arXiv paper targets automatic drum transcription, turning audio into MIDI notes for each drum hit. Deep learning leads the field, but it needs large paired audio-MIDI datasets, which are scarce.

Synthetic data is the usual workaround, yet it often sounds fake. Cheap SoundFont libraries lack the acoustic variety of real kits, leaving a domain gap. The authors aim for more realistic synthetic data to close that gap. Better training data would make transcription work on real recordings, not just clean studio stems.

Catalog record

CREATORPierfrancesco Melucci, Paolo Merialdo, Taketo Akama, [1.0em] Sapienza University of Rome, Italy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-09520

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Realistic Synthetic Data for Automatic Drum Transcription Pierfrancesco Melucci, Paolo Merialdo, Taketo Akama, [1.0em] Sapienza University of Rome, Italy Abstract Deep learning models define the state-of-the-art in Automatic Drum Transcription (ADT), yet their performance is contingent upon large-scale, paired audio-MIDI datasets, which are scarce. Existing workarounds that use synthetic data often introduce a significant domain gap, as they typically rely on low-fidelity SoundFont libraries that lack acoustic diversity. While high-quality one-shot samples offer a better alternative, they are not available in a standardized, large-scale format suitable for training. This paper introduces a new paradigm for ADT that circumvents the need for paired audio-MIDI training data. Our primary contribution is a semi-supervised method to automatically curate a large and diverse corpus of one-shot drum samples from unlabeled audio sources. We then use this corpus to synthesize a high-quality dataset from MIDI files alone, which we use to train a sequence-to-sequence transcription model. We evaluate our model on the ENST and MDB test sets, where it achieves new state-of-the-art results, significantly outperforming both fully supervised methods and previous synthetic-data approaches. The code for reproducing our experiments is publicly available at https://github.com/pier-maker92/ADT_STR . 1 Introduction Automatic Drum Transcription (ADT) is the task of generating a symbolic music transcription from an audio recording of a drum performance. The task can be specified for several scenarios, including drum-only recordings (DTD), drums with other percussion (DTP), and drums within full musical arrangements (DTM) [ 13 ] . Since the adoption of Deep Neural Networks for ADT, a significant challenge has been the scarcity of large-scale, paired audio-MIDI corpora required for supervised training. As MIDI-only datasets are far more abundant, synthetic data generation has been widely explored as a solution [ 12 , 7 , 2 ] . However, these synthetic approaches introduce a critical domain gap and effectively reframe the task as an out-of-distribution generalization problem. Indeed, previous works have shown that synthetic data is most effective when used in combination with real-world data, e.g. when pretrained models on synthetic datasets are subsequently finetuned on real-world distributions. This observation indicates that the distribution shift introduced by synthetic data is too large to rely solely on synthetic data generation during training. We hypothesize that this issue stems from inherent limitations of SoundFont-based synthesis, which tends to produce low-quality audio from MIDI representations. High-quality one-shot drum samples provide a more realistic and diverse sound source, and their use has been shown to be effective for training models for music transcription [ 9 ] , but in the context of ADT their practical use is hindered by a lack of standardization. Publicly available one-shot libraries suffer from two main issues: (i) ambiguous or incorrect labels for similar instruments (e.g., ride vs. crash cymbals), and (ii) absence of a consistent naming convention, which must be inferred through further analysis of the spectral content, as proposed in [ 2 ] . In addition to the synthetic-to-real distribution gap, we highlight another limitation in the current state of the art of ADT systems from an architectural perspective. Sequence-to-sequence models have been shown to perform very well in Music Information Retrieval (MIR) tasks, such as music transcription [ 5 , 9 , 3 ] . However, these works report drum transcription results primarily for ablation purposes, with the evaluation of drum signals lying outside the main scope of the study, which is instead focused on the broader task of multi-instrument music transcription. Consequently, a systematic evaluation of this type of architecture when trained specifically for automatic drum transcription (ADT) tasks is still lacking. Motivated by this, we investigate an efficient synthetic data generation pipeline that aims to reduce the distributional gap between synthetic and real data by incorporating one-shot sample libraries, as proposed in [ 9 , 2 ] . We further validate our approach by training a sequence-to-sequence transformer specifically tailored for the ADT task. We summarize the contributions of this study as follows: • We propose a scalable semi-supervised pipeline for creating a large, diverse, and standardised corpus of one-shot drum samples, starting from unstructured, publicly available one-shot libraries. • We empirically validate our approach by training a model exclusively on synthetic data that outperforms previous methods on real drum signal distributions, as evaluated on the ENST and MDB datasets. Table 1 : MIDI Key to GM-Percussion Mapping Key Source Label Source Instrument ID Label Target Key Source Label Source Instrument ID Label Target 35 Acoustic Bass Drum 0 Acoustic Bass Drum 58 Vibraslap 18 Vibraslap 36 Bass Drum 1 1 Bass Drum 1 59 Ride Cymbal 2 13 Ride Cymbal 37 Side Stick 2 Side Stick 60 Hi Bongo 19 Congas & Timbales 38 Acoustic Snare 3 Acoustic Snare 61 Low Bongo 19 Congas & Timbales 39 Hand Clap 4 Hand Clap 62 Mute Hi Conga 19 Congas & Timbales 40 Electric Snare 5 Electric Snare 63 Open Hi Conga 19 Congas & Timbales 41 Floor Tom 6 Floor Tom 64 Low Conga 19 Congas & Timbales 42 Closed Hi Hat 7 Closed Hi Hat 65 High Timbale 19 Congas & Timbales 43 High Floor Tom 6 Floor Tom 66 Low Timbale 19 Congas & Timbales 44 Pedal Hi-Hat 8 Pedal Hi-Hat 67 High Agogo 17 Cowbell 45 Low Tom 6 Floor Tom 68 Low Agogo 17 Cowbell 46 Open Hi-Hat 9 Open Hi-Hat 69 Cabasa 20 Shaker 47 Low Mid Tom 10 Mid Tom 70 Maracas 20 Shaker 48 High Mid Tom 10 Mid Tom 71 Short Whistle 21 Whistle 49 Crash Cymbal 11 Crash Cymbal 72 Long Whistle 21 Whistle 50 High Tom 12 High Tom 73 Short Guiro 22 Guiro 51 Ride Cymbal 13 Ride Cymbal 74 Long Guiro 22 Guiro 52 Chinese Cymbal 14 Chinese Cymbal 75 Claves 23 Claves 53 Ride Bell 13 Ride Cymbal 76 Hi Wood Block 23 Claves 54 Tambourine 15 Tambourine 77 Low Wood Block 23 Claves 55 Splash Cymbal 16 Splash Cymbal 78 Mute Cuica 24 Cuica 56 Cowbell 17 Cowbell 79 Open Cuica 24 Cuica 57 Crash Cymbal 2 11 Crash Cymbal 80 Mute Triangle 25 Triangle 81 Open Triangle 25 Triangle 2 Related works The field of ADT has been systematically reviewed in [ 13 ] , which outlined the primary transcription scenarios (DTD, DTP, DTM) and identified the Convolutional Recurrent Neural Network (CRNN) followed by a peak-picking algorithm as the dominant architectural paradigm. While alternative sequence-to-sequence Transformer models have been proposed [ 5 ] for music transcription, in the context of ADT, only a few works have explored different architectures other than CRNNs, limiting the analysis to employing self-attention in such convolutional systems [ 15 ] . In response to the lack of paired audio–MIDI datasets, a substantial body of prior work has adopted data-centric approaches to address the data scarcity. Rather than focusing on architectural designs tailored to the ADT task, these studies primarily investigate how large-scale data can be leveraged through synthetic data generation strategies or crowd-sourced collections. Cartwright and Bello demonstrated that combining a small real dataset with a larger synthetic one improves performance, expanding the transcription vocabulary [ 2 ] . Following this direction, Vogl et al. introduced the TMIDT dataset, a large synthetic corpus, and found that pre-training on synthetic data

Read the complete text