Beat and Downbeat Tracking in Performance MIDI Using an End-to-End Transformer Architecture

Sebastian Murgul ( [email protected] ), Michael Heizmann ( [email protected] ) · 2025

Read Beat and Downbeat Tracking in Performance MIDI Using an End-to-End Transformer Architecture by Sebastian Murgul ( [email protected] ), Michael Heizmann ( [email protected] ) (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-00466.txt.

Introduction

Sebastian Murgul and Michael Heizmann track beats and downbeats in performed MIDI music with a transformer model in a 2025 paper. Finding the beat in symbolic music matters for transcription and rhythm analysis, yet most methods target audio. Their model reads a MIDI stream and writes beat marks in one end to end pass.

New preprocessing steps, including varied augmentation and tuned token design, aid accuracy across data sets. Tests on four public sets, covering piano and guitar, compared the model with classic Markov tools and deep learning rivals. It beat prior symbolic methods and scored well across styles and instruments. The authors see a path to joining it with full transcription systems.

Catalog record

CREATORSebastian Murgul ( [email protected] ), Michael Heizmann ( [email protected] )
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-00466

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Beat and Downbeat Tracking in Performance MIDI Using an End-to-End Transformer Architecture Sebastian Murgul ( [email protected] ), Michael Heizmann ( [email protected] ) Abstract Beat tracking in musical performance MIDI is a challenging and important task for notation-level music transcription and rhythmical analysis, yet existing methods primarily focus on audio-based approaches. This paper proposes an end-to-end transformer-based model for beat and downbeat tracking in performance MIDI, leveraging an encoder-decoder architecture for sequence-to-sequence translation of MIDI input to beat annotations. Our approach introduces novel data preprocessing techniques, including dynamic augmentation and optimized tokenization strategies, to improve accuracy and generalizability across different datasets. We conduct extensive experiments using the A-MAPS, ASAP, GuitarSet, and Leduc datasets, comparing our model against state-of-the-art hidden Markov models (HMMs) and deep learning-based beat tracking methods. The results demonstrate that our model outperforms existing symbolic music beat tracking approaches, achieving competitive F1-scores across various musical styles and instruments. Our findings highlight the potential of transformer architectures for symbolic beat tracking and suggest future integration with automatic music transcription systems for enhanced music analysis and score generation. 1 Introduction Beat tracking aims to detect the underlying rhythmic grid within a musical performance [ 1 ] . This rhythmic grid consists of downbeats, beats, and tatum subdivisions. Downbeats refer to the first beat within a bar and therefore indicate the beginning of a new bar. In a notation-level music transcription system, we implicitly or explicitly need the rhythmical grid in order to be able to derive discrete note values from the detected tones [ 2 ] . In general, we have two input options in a transcription setting: We can either use the original input audio or the output performance MIDI data of a previous note tracking step. Audio-based beat tracking has seen substantial progress, particularly with the introduction of deep learning techniques. Early approaches relied on statistical models, but modern methods increasingly leverage neural networks to improve accuracy and robustness. One of the first notable deep learning approaches was proposed by Böck et al. (2011), who pioneered the use of bidirectional Long Short-Term Memory (LSTM) networks to classify beats from an audio spectrogram, smoothing predictions using autocorrelation [ 3 ] . This work evolved in 2016 when Böck et al. proposed an RNN-based beat tracking method that outputs beat and downbeat features directly from magnitude spectrograms. A Dynamic Bayesian Network (DBN) was then used to model variable-length bars and align the detected beats [ 4 ] . Davies et al. enhanced Böck’s approach in 2019 by replacing LSTMs with a Temporal Convolutional Network (TCN) featuring dilated convolutions along the temporal axis [ 5 ] . Zhao et al. introduced Beat Transformer in 2022, employing time-wise and instrument-wise attention mechanisms alongside dilated self-attention and demixed spectrograms to improve beat detection [ 6 ] . Foscarin et al. proposed Beat This! in 2024, an advanced transformer-based beat tracker demonstrating high accuracy and generality across diverse musical styles. Since Beat This! does not rely on DBN postprocessing, it is also suitable for pieces with time-signature changes or high tempo variations [ 7 ] . Unlike audio-based beat tracking, which has been extensively researched, MIDI-based methods have remained relatively scarce. Traditional approaches to MIDI-based beat tracking often relied on rule-based heuristics and statistical models, but recent research has begun incorporating deep learning techniques. Cambouropoulos et al. proposed a system for joint beat detection and rhythm quantization in 2000 [ 8 ] . Their approach clustered inter-onset intervals for beat detection, followed by assigning note onsets to the closest points on a metrical grid and assigning note values based on inter-onset intervals. Temperley’s 2007 book ‘Music and Probability’extended Bayesian probabilistic approaches to infer complete metrical grids rather than score positions relative to a bar [ 9 ] . Cogliati et al. presented an HMM-based system in 2016 for joint estimation of meter, harmony, and stream separation, combined with a distance-based quantization algorithm [ 10 ] . Foscarin et al. introduced a parse-based system in 2019 employing weighted context-free grammars (WCFGs) for joint rhythm quantization and music score production [ 11 ] . Shibata et al. proposed a piano transcription system in 2021 that incorporated HMMs and Markov Random Fields (MRFs) for rhythm quantization, leveraging non-local musical statistics to infer global parameters [ 12 ] . Liu et al. proposed a Convolutional Recurrent Neural Network (CRNN)-based system in 2022 for MIDI-to-score conversion, incorporating onset-based beat detection and rhythm quantization [ 13 ] . Kim et al. developed a transformer and Convolutional Neural Network (CNN)-based guitar transcription model in 2023 that produced note-level transcriptions from spectrograms using beat information [ 14 ] . Beyer et al. introduced a performance MIDI-to-score conversion approach in 2024 based on the Roformer architecture. Their encoder-decoder model directly generated MusicXML tokens while implicitly performing beat estimation and rhythm quantization on MIDI token sequences [ 15 ] . Despite these efforts, modern transformer architectures have seen limited application in symbolic beat tracking. Existing methods either rely on traditional probabilistic models or use deep learning approaches not specifically optimized for MIDI beat tracking. This gap presents an opportunity to explore more advanced techniques for symbolic music. In this work, we introduce a novel end-to-end transformer-based approach for beat tracking in MIDI performances, achieving state-of-the-art performance and surpassing existing symbolic beat tracking methods. By leveraging modern transformer architectures, our model effectively captures temporal dependencies and outperforms previous HMM-based and neural network-based systems. 2 Methodology Encoder Encoding Decoder ON \langle 55\rangle T \langle 0.01\rangle OFF \langle 55\rangle T \langle 0.44\rangle ON \langle 62\rangle T \langle 0.44\rangle … MIDI-like Input Tokens B \langle 1\rangle T \langle 0.01\rangle B \langle 2\rangle T \langle 0.44\rangle B \langle 3\rangle T \langle 0.89\rangle … Beat Output Tokens Autoregressive Sampling Figure 1: Our model is based on the T5 encoder-decoder transformer architecture [ 16 ] and uses MIDI-like tokens as input and outputs tokenized beat and downbeat information. During inference, autoregressive sampling with beam search is used. Our proposed approach for performance MIDI beat tracking is visualized in Figure 1 . The model follows an encoder-decoder transformer architecture, designed to translate an input MIDI segment into the corresponding beat sequence. Since transformers operate on text-based token sequences, MIDI data must first be preprocessed and tokenized before being fed into the model. The following sections detail the preprocessing pipeline, data augmentation strategies, encoding schemes, and model architecture. 2.1 Preprocessing The data processing is shown in the flow chart in Figure 2 . As input, we use the MIDI files of the A-MAPS dataset. Firstly, we extract the notes and the beat annotations from the MIDI files using the PrettyMIDI library [ 17 ] . The extracted annotations are then split into segments of 10\text{\,}\mathrm{s} with a hop size of 1\text{\,}\mathrm{s} . In the next step, the segments are cleaned, and all examples with less than one beat are dropped. Finally, the resulting examples are stored in a CSV file for access during training. A-MAPS MIDI Extract Notes and Beats Split Overlapping Segments Data Cleaning CSV-File Data Augmentation Quantize Note Times Encoding and Tokenization Dataset Figure 2: Flowchart of the data processing steps 2.2 Data Augmentation The data augmentation is done dynamically in the data loading process during the training. To enrich the diversity

Read the complete text