Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training

Daniel Dratschuk, Paul Swoboda · 2026

Read Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training by Daniel Dratschuk, Paul Swoboda (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10835.txt.

Introduction

Daniel Dratschuk and Paul Swoboda introduce Transcoda, a 2026 system that reads sheet music from images and turns it into structured text, without ever training on hand-labelled real scans. Optical music recognition has stalled for lack of large annotated collections, and the usual text code for scores allows many spellings of the same page, which muddles learning.

Transcoda trains on richly varied synthetic scores, forces its code into one standard spelling, and decodes with grammar rules that keep the output well formed. A compact model trained in six hours on one graphics card beats far larger rivals on fresh synthetic tests and cuts errors sharply on old Polish scans. Archivists and musicians get a fast route from printed pages to searchable, playable notation.

Catalog record

CREATORDaniel Dratschuk, Paul Swoboda
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32045
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.10835

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training Daniel Dratschuk, Paul Swoboda Abstract Optical Music Recognition (OMR), the task of transcribing sheet music into a structured textual representation, is currently bottlenecked by a lack of large-scale, annotated datasets of real scans. This forces models to rely on either few-shot transfer or synthetic training pipelines that remain overly simplistic. A secondary challenge is encoding non-uniqueness: in the popular Humdrum **kern format for transcribing music, multiple different text encodings can render into the same visual sheet music. This one-to-many mapping creates a harder learning task and introduces high uncertainty during decoding. We propose Transcoda, an OMR system built on (i) an advanced synthetic data generation pipeline, (ii) a normalization of the **kern encoding to enforce a unique normal form and (iii) grammar-based decoding to ensure the syntactic correctness of the output. This approach allows us to train a compact 59M-parameter model in just 6 hours on a single GPU that outperforms billion-parameter baselines. Transcoda achieves the best score among state of the art baselines on a newly curated benchmark of synthetically rendered scores at 18.46% OMR-NED (compared to 43.91% for the next-best system, Legato) and reduces the error rate on historical Polish scans to 63.97% OMR-NED (down from 80.16% for SMT++). **kern **kern *clefF4 *clefG2 4.r 32eLLL . 32f## ... ... Synthetic (Verovio) Real Scans (Polish) 0 20 40 60 80 100 92.23 80.16 43.91 86.73 18.46 63.97 OMR-NED (%) \downarrow SMT++ LEGATO Transcoda (ours) Figure 1 : Transcoda maps score images to normalized **kern sequences. Trained entirely on synthetic data, it achieves the lowest error (OMR-NED) among compared systems on both clean synthetic benchmarks and zero-shot transfer to real historical scans. 1 Introduction Optical Music Recognition (OMR) translates images of sheet music into machine-readable textual formats. It is more complex than plain text recognition because music notation is a dense two-dimensional graph: noteheads, stems, beams, accidentals, staves, and rhythmic constraints interact across space and time. Still, like for OCR, modern end-to-end systems treat OMR as image-to-sequence generation, but unlike OCR they face a severe data problem. Annotated datasets of physical scans are not publicly available at the scale needed for current neural models, so systems are trained on synthetic renderings and evaluated zero-shot on real scans [ 32 , 25 ] . For synthetic data the main failure mode is not only visual noise. Real scores also have denser layouts and much longer textual targets than common synthetic training samples. Due to the interdependency of different music voices, rhythm etc. a small local visual error can push an autoregressive decoder into an invalid continuation, after which the whole transcription can become unusable. An additional challenge is non-uniqueness: The same visual score can map to many syntactically different but semantically equivalent transcription sequences. Flexible encoding conventions create a one-to-many training signal. The decoder must model source-specific formatting variation in addition to the musical structure. This makes training and decoding harder and distribution shift additionally compounds uncertainty. We introduce Transcoda , a compact 59M-parameter vision-encoder-decoder model trained only on synthetic data encoded in the **kern [ 9 ] format. Its core design choice is data-centric. Before training, a deterministic pipeline normalizes textual targets so that each rendered score maps to a unique grammar-aligned sequence. At inference time, an optional constrained decoder can enforce formal **kern validity whenever downstream rendering or playback needs it. Our experiments show that normalization, not model scale or decoding constraints, is the main stabilizer. Our contributions are: 1. We propose an elaborate synthetic data generation pipeline that better approaches the visual and structural complexity of real sheet music. We generate, to our knowledge, the largest openly available dataset for training OMR. 2. We identify non-uniqueness as a bottleneck in zero-shot OMR and show that deterministic target normalization stabilizes autoregressive generation. 3. We release a standardized synthetic evaluation protocol based on the Verovio rendering engine, and report ablations that disentangle sequence-modeling stability, visual domain shift, and formal validity. 4. Empirically our compact end-to-end Transcoda model achieves the best score among state of the art publicly available OMR systems on our synthetic benchmark and improves over larger baselines on real historical scans. We release code, training scripts, and model weights at https://github.com/btrkeks/transcoda . 2 Related work Traditional and modular OMR. Classical OMR pipelines decomposed the task into preprocessing, symbol detection, notation assembly, and encoding [ 23 ] . These hand-crafted stages were fragile on handwritten or degraded documents [ 27 ] , with errors cascading across stages [ 5 ] . Deep learning was initially adopted to strengthen individual stages rather than to replace the pipeline itself: Pacha and Eidenberger trained a universal symbol classifier across aggregated datasets [ 17 ] , and Yang et al. jointly trained a notation-assembly model on imperfect YOLOv8 detector outputs to mitigate cascade effects [ 33 ] . These systems improved component accuracy but inherited the staged structure and its failure modes. Shift to End-to-End Architectures. To avoid cascading errors, the field shifted towards end-to-end systems that treat OMR as a sequence recognition problem. Calvo-Zaragoza et al. proposed a convolutional RNN that directly transcribes preprocessed single-staff images into sequences of musical symbols [ 2 ] . The adoption of Transformer architectures improved spatial modeling; Li et al. proposed TrOMR to enhance contextual perception in polyphonic scores [ 11 ] , while Ríos-Vila et al. introduced the Sheet Music Transformer (SMT) [ 24 ] , which was the first system to successfully transcribe entire single systems of pianoform scores without any simplifications Recent Full-Page Baselines: SMT++ and Legato. Recent efforts have scaled end-to-end OMR from isolated systems to full pages and beyond, establishing our two primary baselines. By implementing curriculum learning, the SMT architecture was adapted into SMT++ [ 25 ] , achieving the first full-page, end-to-end OMR for pianoform sheet music while bypassing prior layout analysis or staff cropping. However, SMT++ relies on a relatively small dataset, is computationally expensive to train, and frequently generates syntactically invalid **kern notation. Expanding scale further, Yang et al. introduced Legato [ 32 ] , which processes multiple concatenated pages at a time. Legato uses a frozen Llama 3.2 11B Vision encoder [ 15 ] and a custom Byte-Pair Encoding (BPE) tokenizer. Trained on over 214,000 synthetic scores, Legato directly generates ABC notation [ 32 , 28 ] . Like **kern , ABC is a discrete text-based format for music, but it functions as a denser, more standardized alphanumeric shorthand. Compared to SMT++, we use a simpler training protocol and substantially more training data; compared to Legato, a much smaller model with increased training data. Our data generation pipeline improves over both. Nevertheless, our training and inference speed is still significantly higher. 3 Method Transcoda is a vision-encoder-decoder model that maps a fixed-size score image to a **kern token sequence. The method has three parts: a compact architecture, a target-normalization data engine, and an optional constrained decoder. 3.1 Architecture 1485{\times}1050 px (fixed) Score image ConvNeXt-V2 pretrained & finetuned 47{\times}33{\times}768 Vision encoder Projector MLP Lin. \to GELU \to Lin. 1551{\times}512 + 1551{\times}512 2D Sin. PE Self-attention RoPE Cross-attention Feed-forward \times 8 Transformer decoder d_{\mathrm{model}}{=}512 , 8 heads, d_{\mathrm{ff}}{=}1024 \otimes Constrained decoding **kern tokens vocab {=}3000 Output autoregressive Figure

Read the complete text