TokenSynth: A Token-based Neural Synthesizer for Instrument Cloning and Text-to-Instrument
Kyungsu Kim \musSharp {}^{\musSharp} Junghyun Koo \musSharp {}^{\musSharp} Sungho Lee \musSharp {}^{\musSharp} Haesun Joung \musSharp {}^{\musSharp} Kyogu Lee \musSharp \musNatural \musFlat {}^{\musSharp\musNatural\musFlat} · 2025
Read TokenSynth: A Token-based Neural Synthesizer for Instrument Cloning and Text-to-Instrument by Kyungsu Kim \musSharp {}^{\musSharp} Junghyun Koo \musSharp {}^{\musSharp} Sungho Lee \musSharp {}^{\musSharp} Haesun Joung \musSharp {}^{\musSharp} Kyogu Lee \musSharp \musNatural \musFlat {}^{\musSharp\musNatural\musFlat} (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-08939.txt.
Description
Cloning the sound of an instrument usually needs special training for each new target. Kyungsu Kim and co-authors built TokenSynth, a neural synthesiser that works from compact audio tokens instead. A decoder-only transformer takes in musical note data plus a sound embedding that carries timbre, and produces tokens for the desired audio. The same system can clone an instrument, build one from a text description, or reshape a sound under text guidance, all without retraining.
Tests measured how good the audio sounded, how close its tone sat to the target sound or text, and how faithfully it followed the input notes. The authors present the system as proof that modern audio codecs and transformers make flexible synthesisers. Code, weights, and demos are public.
Catalog record
| CREATOR | Kyungsu Kim \musSharp {}^{\musSharp} Junghyun Koo \musSharp {}^{\musSharp} Sungho Lee \musSharp {}^{\musSharp} Haesun Joung \musSharp {}^{\musSharp} Kyogu Lee \musSharp \musNatural \musFlat {}^{\musSharp\musNatural\musFlat} |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-08939