MusicSynth: An Automated Pipeline for Generating Violin Fingerboard Animations from Sheet Music Using Optical Music Recognition

Abhimanyu Kaushik, https://musicsynth.github.io/ · 2026

Read MusicSynth: An Automated Pipeline for Generating Violin Fingerboard Animations from Sheet Music Using Optical Music Recognition by Abhimanyu Kaushik, https://musicsynth.github.io/ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-17181.txt.

Introduction

Kaushik builds MusicSynth, a free browser tool that turns violin sheet music into animated fingerboard videos. Optical music recognition reads photographed scores, a parser handles digital files, and a renderer highlights each note's string and finger position in time. Across 110 public-domain scores it identified about 91 percent of notes in clean print and nearly all finger positions from digital input.

Catalog record

CREATORAbhimanyu Kaushik, https://musicsynth.github.io/
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23347
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2605.17181

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MusicSynth: An Automated Pipeline for Generating Violin Fingerboard Animations from Sheet Music Using Optical Music Recognition Abhimanyu Kaushik, https://musicsynth.github.io/ Abstract Learning the violin is harder than it looks. Unlike piano keys or guitar frets, the violin neck has no markings at all, so a beginner cannot tell by looking where to place each finger. MusicSynth is an open-source web tool that tries to fix that: user uploads a photo of any violin sheet music (or a digital score file), and the system automatically produces a video showing a violin fingerboard with each note highlighted at the right moment—no software to install, no manual note entry required. The system connects three existing open-source tools into one pipeline: an optical music recognition (OMR) library reads the notes from the uploaded image, a MusicXML parser extracts timing information from digital scores, and a video renderer draws the fingerboard frame by frame. The only part built from scratch is the lookup table that maps each musical note to a string and finger position on the violin. Tested across 110 public-domain violin scores, MusicSynth correctly identified 91.2 % of notes in clean printed music and assigned the right finger position 99.1 % of the time when given a digital score file. To the author’s knowledge, no freely available tool currently turns a sheet music image into an animated violin fingerboard tutorial automatically and in a single browser-based step. Keywords: optical music recognition, violin fingerboard, music education, string instrument pedagogy, MusicXML, Streamlit 1 Introduction Learning the violin is hard in a way that is easy to underestimate. Unlike fretted or keyed instruments, where physical markers guide finger placement, the violin neck is completely unmarked. Pitch depends on exactly where the finger contacts the string, and even a millimetre off-centre produces an out-of-tune note. Beginners must develop score-reading skill and physical spatial intuition at the same time. That dual load is one of the main reasons self-directed learners give up early, especially those without regular access to a teacher [ 4 ] . The tools available today do not fully solve this problem. Traditional method books such as Suzuki [ 15 ] and Wohlfahrt [ 5 ] provide the notes but no visual guide to where the hand should go. Apps such as Yousician can listen to live playing and give feedback, but they cannot take an arbitrary sheet music file and show how to finger it. The gap is straightforward: a user wants to upload any score and receive a clear, timed guide showing exactly where each finger belongs. MusicSynth is designed to fill that gap. The user uploads a photograph or scan of a violin score—or a MusicXML file if one is already available—and the system returns a video in which a fingerboard diagram highlights each note as it occurs, labelled with its finger number and string. The video plays directly in the browser and can be downloaded for offline practice. The core insight behind MusicSynth is not that any single component is new—OMR libraries, MusicXML parsers, and video renderers all exist—but that connecting them into one end-to-end, violin-specific pipeline that any user can use from a browser has not been done in a freely available form before. The contributions of this project are: • The first publicly available, browser-based tool to automatically produce animated violin fingerboard tutorials from uploaded sheet music images, without requiring any software installation or manual note entry. • A documented, easy-to-extend note-to-finger lookup table covering the standard violin range (G3–G6), based on established violin pedagogy. • A working integration of existing open-source OMR, MusicXML parsing, and video rendering libraries into a complete pipeline, deployed as a live web application with user authentication. • A benchmark evaluation across 110 public-domain violin scores measuring recognition accuracy and correct finger assignment across five difficulty levels and input types. 2 Background and Related Work 2.1 The Violin Fingerboard Problem The violin has four strings tuned in perfect fifths: G3, D4, A4, and E5. In first position—where beginners spend most of their time—the index finger on the G string produces A3, the middle finger B3, and so on. The same pattern repeats across all four strings, covering roughly G3 to B5 [ 5 ] . One complication is that many pitches can be played on more than one string: D4, for example, is both the open D string and the third finger on the G string. Standard violin pedagogy has established preferred fingerings for these cases, generally choosing whichever option requires fewer string crossings or hand shifts [ 15 ] . MusicSynth follows those conventions. 2.2 Optical Music Recognition Optical Music Recognition (OMR) is the task of reading a music score image and converting it into a list of notes that a computer can process [ 12 ] . Modern OMR systems use neural networks to detect note heads, staff lines, and other symbols, then assemble them into a structured format [ 16 , 2 ] . For single-voice violin music—no chords, not much polyphony—the problem is simpler than for full orchestral scores, which makes existing general-purpose OMR libraries good enough as a starting point. MusicSynth uses Oemer [ 1 ] , an open-source OMR library that takes a score image and returns the detected notes in MusicXML format. Oemer was chosen because it handles the full recognition process internally, returns a standard output format, and runs on a normal laptop CPU without a graphics card. 2.3 MusicXML as a Standard File Format MusicXML [ 6 ] is the standard way music notation software stores scores: it organises the music as measures containing notes, each with a pitch name, duration, and timing. Most professional notation programs—MuseScore, Sibelius, Finale, Dorico—can export MusicXML. When a user already has a digital score, they can upload the MusicXML file directly, skipping the image recognition step entirely and getting near-perfect results. 2.4 Existing Tools and What MusicSynth Adds Table 1 compares MusicSynth to the most relevant alternatives. No existing free tool satisfies all five criteria at once: image input, automatic OMR, fingerboard output, browser-based access, and no cost. MusicSynth is the first to do so. Table 1 : Capability comparison. MusicSynth is the only tool to satisfy all five criteria simultaneously. Tool Image Input OMR Fingerboard Browser Free MusicSynth (this work) Yes Yes Yes Yes Yes Audiveris Yes Yes No No Yes PhotoScore Yes Yes No No No ViolinOnline Fingering No No Yes (manual) Yes Yes MuseScore Online No No Conditional Yes Yes Yousician No No No Yes Partial 2.5 Animated Visualisations for Other Instruments Animated music visualisations are well established for piano. Synthesia-style “falling note” videos have been shown to help beginners understand what notes to play before they can read sheet music fluently [ 7 , 3 ] . Guitar tools such as Guitar Pro show fingerings on a fretboard, but guitar tablature already encodes finger positions, so no pitch-to-finger lookup is needed. For violin, no comparable open-source tool existed before MusicSynth. Ramirez et al. [ 11 ] built an augmented-reality system that projects finger positions onto a real violin using a depth camera. It works well but requires specialised hardware. MusicSynth takes a simpler approach: produce a downloadable video that any user can watch on a phone

Read the complete text