BMdataset: A Musicologically Curated LilyPond Dataset

Matteo Spanio ( [email protected] ), Ilay Guler ( [email protected] ), Antonio Roda ( [email protected] ) · 2026

Read BMdataset: A Musicologically Curated LilyPond Dataset by Matteo Spanio ( [email protected] ), Ilay Guler ( [email protected] ), Antonio Roda ( [email protected] ) (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10628.txt.

Introduction

Spanio, Guler, and Rodà build a music dataset from expert transcriptions of Baroque manuscripts. The scores use LilyPond, a text-based format for writing music, little used for music study. The set holds 393 scores with 2,646 movements, marked by composer, form, instruments, and section. They also train LilyBERT, an encoder tuned to LilyPond notation. Tuning on this small hand-picked set beats broad pretraining on a far larger corpus for composer and style sorting. Mixing both reaches 84.3 percent composer accuracy.

Catalog record

CREATORMatteo Spanio ( [email protected] ), Ilay Guler ( [email protected] ), Antonio Roda ( [email protected] )
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27360
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10628

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

BMdataset: A Musicologically Curated LilyPond Dataset Matteo Spanio ( [email protected] ), Ilay Guler ( [email protected] ), Antonio Roda ( [email protected] ) Abstract Symbolic music research has relied almost exclusively on MIDI-based datasets; text-based engraving formats such as LilyPond remain unexplored for music understanding. We present BMdataset , a musicologically curated dataset of 347 LilyPond scores (2,646 movements) transcribed by experts directly from original Baroque manuscripts, with metadata covering composer, musical form, instrumentation, and sectional attributes. Building on this resource, we introduce LilyBERT 1 1 1 https://huggingface.co/csc-unipd/lilybert , a CodeBERT-based encoder adapted to symbolic music through vocabulary extension with 115 LilyPond-specific tokens and masked language model pre-training. Linear probing on the out-of-domain Mutopia corpus shows that, despite its modest size ( ∼ {\sim} 90M tokens), fine-tuning on BMdataset alone outperforms continuous pre-training on the full PDMX corpus ( ∼ {\sim} 15B tokens) for both composer and style classification, demonstrating that small, expertly curated datasets can be more effective than large, noisy corpora for music understanding. Combining broad pre-training with domain-specific fine-tuning yields the best results overall (84.3% composer accuracy), confirming that the two data regimes are complementary. We release the dataset, tokenizer, and model to establish a baseline for representation learning on LilyPond. 1 Introduction Recent work in music information retrieval (MIR) and generative AI has renewed attention to symbolic music representations [ 1 , 2 , 3 ] . While audio-domain models have seen rapid commercial development, symbolic formats remain essential for musicological analysis, score generation, and artist-in-the-loop workflows where fine-grained control over musical structure is required. Among symbolic formats, MIDI dominates: large-scale datasets such as Lakh MIDI [ 4 ] and PDMX [ 5 ] support classification, generation, and retrieval tasks. However, MIDI discards much of the notational information present in written scores, including engraving directives, section boundaries, and the hierarchical structure of multi-movement works. MusicXML partially addresses this gap, but no comparable resources exist for LilyPond 2 2 2 https://lilypond.org , a widely used open-source music engraving system whose text-based markup language encodes both musical content and typographic structure in a human-readable, version-controllable format. LilyPond’s syntax shares structural properties with programming languages: it uses nested blocks, backslash commands, variable assignments, and macro definitions. This observation motivates the use of code pre-trained language models, which have been shown to capture hierarchical and syntactic patterns in structured text [ 6 ] , as a foundation for learning LilyPond representations. In this paper, we make four contributions: 1. BMdataset , a musicologically curated dataset of 347 compilable LilyPond scores (2,646 movements) sourced from BaroqueMusic.it 3 3 3 https://www.baroquemusic.it , last visited: november 2025 , transcribed directly from original manuscripts. The dataset is accompanied by structured metadata (composer, musical form, instrumentation, key, tempo, time signature, and historical period) and will be released with .ly , .midi , .pdf , and .json metadata files. 2. A LilyPond tokenizer that extends the CodeBERT vocabulary with 115 domain-specific tokens, ensuring that musically meaningful commands (e.g., \relative , \clef , \key ) are represented as atomic units rather than fragmented subwords. 3. LilyBERT , a CodeBERT-based encoder adapted to symbolic music through masked language model (MLM) pre-training on LilyPond corpora. 4. A systematic probing evaluation on the Mutopia corpus comparing four model variants that isolate the effects of pre-training data source, corpus size, and domain-specific fine-tuning on downstream composer and style classification. 2 Related Work 2.1 Symbolic Music Datasets Table 1: Comparison of symbolic music datasets. BMdataset is the first musicologically curated LilyPond-native dataset. Dataset Format Size License LMD [ 4 ] MIDI 174K CC-BY † SymphonyNet [ 7 ] MIDI 46K – MAESTRO [ 8 ] MIDI 1.3K CC-BY-NC-SA Wikifonia [ 9 ] MusicXML 6.4K – POP909 [ 10 ] MIDI 909 – Nottingham [ 11 ] ABC 1.2K PD ABC-Tune [ 12 ] ABC 165K CC-0 PDMX [ 5 ] MusicXML 254K CC-0 Mutopia LilyPond 2.1K PD BMdataset LilyPond 347 CC-BY-NC-ND \dagger Contains documented copyrighted works [ 3 ] . Table 1 summarises existing symbolic music datasets. Most large-scale datasets use MIDI [ 4 , 7 , 8 , 10 ] or MusicXML [ 5 , 9 ] formats. MIDI is compact and widely supported but discards notational details such as engraving directives, articulation placement, and the hierarchical structure of multi-movement works. MusicXML preserves richer score information but remains tied to XML-based tooling. LilyPond, by contrast, represents music as structured plaintext — making it version-controllable, human-readable, and directly processable by language models without format conversion. While the Mutopia Project provides a community-curated collection of public domain LilyPond scores, it lacks structured metadata beyond basic composer and style tags and has not been used as a basis for representation learning. To our knowledge, no musicologically curated LilyPond dataset with systematic metadata annotation currently exists. 2.2 Music Representation Learning Pre-trained Transformer models for symbolic music have improved both classification and generation benchmarks. MusicBERT [ 2 ] and MidiBERT-Piano [ 13 ] adapt BERT-style pre-training to MIDI representations and report strong downstream classification performance. Tokenisation-focused work such as Byte Pair Encoding for Symbolic Music [ 14 ] demonstrates that compression-aware vocabularies can improve both quality and efficiency in MIDI-based modelling. On the generative side, MIDI-GPT [ 15 ] shows controllable multitrack composition in MIDI, while NotaGen [ 16 ] scales symbolic generation with ABC notation pre-training. Most of these pipelines operate on MIDI. The principal text-based alternative, ABC notation 4 4 4 https://abcnotation.com , is more limited in structural and engraving expressivity, and parts of the ABC 2.1 specification are still marked as beta [ 17 ] . LilyPond provides a richer score representation (layout directives, reusable variables/macros, and hierarchical score structure) that has not yet been used for pre-training. 2.3 Code Pre-trained Models CodeBERT [ 6 ] is a bimodal pre-trained model for programming languages and natural language, based on the RoBERTa architecture [ 18 ] . Pre-trained on six programming languages using both MLM and replaced-token-detection (RTD) objectives, CodeBERT achieves strong results on code search, documentation generation, and clone detection. Subsequent work has incorporated richer structural signals: GraphCodeBERT [ 19 ] augments the input with data-flow graphs, while AST-aware approaches encode parse-tree structure directly. Because LilyPond files are compiled by a strict grammar, they could expose parse-tree or block-hierarchy information to such structure-aware encoders in future work. Our motivation for selecting CodeBERT as the backbone is that LilyPond behaves more like a compiled formal language than a simple markup format: files are parsed according to a strict grammar and compiled into engraved score outputs. Its syntax employs code-like constructs such as nested blocks ( { } ), backslash commands ( \relative ), variable bindings, includes, and macro definitions. We therefore hypothesise that a code-pretrained encoder is a better inductive fit than a model trained only on plain text token statistics. This parallels recent work showing that code-trained models can be effective on non-code structured languages such as chemical notation and mathematical expressions [ 20 ] . 3 BMdataset Dataset 3.1 Source and Compilation The BMdataset dataset originates from BaroqueMusic.it, a collection of 383 LilyPond projects transcribed by musicologists who worked directly from original manuscript sources. Each transcription is annotated with a reference to the original manuscript and its catalogue number, so that every score can

Read the complete text