Amadeus: Autoregressive Model with Bidirectional Attribute Modelling for Symbolic Music

Hongju Su, Ke Li*, Lan Yang, Honggang Zhang, Yi-Zhe Song · 2025

Read Amadeus: Autoregressive Model with Bidirectional Attribute Modelling for Symbolic Music by Hongju Su, Ke Li*, Lan Yang, Honggang Zhang, Yi-Zhe Song (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-20665.txt.

Introduction

Hongju Su, Ke Li, Lan Yang, Honggang Zhang, and Yi-Zhe Song introduced Amadeus in 2025, a model that generates symbolic music. Existing top models treat a note's attributes as a strict sequence, yet changing which attribute starts gives similar results. The attributes of a note act more like an unordered set, they argue.

Amadeus pairs a step-by-step model for note sequences with a two-way diffusion model for attributes. Two added parts sharpen the learned music features and note decoding. In tests on plain and text-guided generation, Amadeus beat prior models on several measures and ran at least four times faster. The team also built the largest open symbolic music data set for training.

Catalog record

CREATORHongju Su, Ke Li*, Lan Yang, Honggang Zhang, Yi-Zhe Song
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-20665

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Amadeus: Autoregressive Model with Bidirectional Attribute Modelling for Symbolic Music Hongju Su, Ke Li*, Lan Yang, Honggang Zhang, Yi-Zhe Song Abstract Existing state-of-the-art symbolic music generation models predominantly adopt autoregressive or hierarchical autoregressive architectures, modelling symbolic music as a sequence of attribute tokens with unidirectional temporal dependencies, under the assumption of a fixed, strict dependency structure among these attributes. However, we observe that using different attributes as the initial token in these models leads to comparable performance. This suggests that the attributes of a musical note are, in essence, a concurrent and unordered set, rather than a temporally dependent sequence. Based on this insight, we introduce Amadeus, a novel symbolic music generation framework. Amadeus adopts a two-level architecture: an autoregressive model for note sequences and a bidirectional discrete diffusion model for attributes. To enhance performance, we propose Music Latent Space Discriminability Enhancement Strategy(MLSDES), incorporating contrastive learning constraints that amplify discriminability of intermediate music representations. The Conditional Information Enhancement Module (CIEM) simultaneously strengthens note latent vector representation via attention mechanisms, enabling more precise note decoding. We conduct extensive experiments on unconditional and text-conditioned generation tasks. Amadeus significantly outperforms SOTA models across multiple metrics while achieving at least 4 × \times speed-up. Furthermore, we demonstrate training-free, fine-grained note attribute control feasibility using our model. To explore the upper performance bound of the Amadeus architecture, we compile the largest open-source symbolic music dataset to date, AMD (Amadeus MIDI Dataset), supporting both pre-training and fine-tuning. 1 Introduction Fig. 1: Overview of the Amadeus framework. The framework consists of two main components: a note generator that auto regressively generates note-level latent variables, and a bidirectional discrete diffusion note decoder that decodes these latent variables into multi-dimensional attributes iteratively. Algorithmic composition has long been a core challenge in music theory, intersecting technical hurdles with music aesthetics ( Xenakis 1992 ) , creativity research ( Hiller and Isaacson 1957 ) , and human-computer interaction ( Cope 2000 ) . In recent years, deep learning-based music generation methods ( Dhariwal et al. 2020 ; Agostinelli et al. 2023 ; Huang et al. 2018 ; Huang and Yang 2020 ; Hsiao et al. 2021 ) have driven rapid advancements in this field. From a modelling perspective, existing approaches primarily fall into two categories: audio-based generation ( Dhariwal et al. 2020 ; Agostinelli et al. 2023 ) and symbolic music-based generation ( Huang et al. 2018 ; Huang and Yang 2020 ; Dong et al. 2023 ; Qu et al. 2024 ) . Compared to audio-based methods, symbolic music modelling—which represents music as sequences of performance signals ( e.g. , MIDI)—offers significant advantages in lossless editing capability and compact representation. In sequential generation tasks, auto-regressive modelling ( Radford et al. 2018 ) has garnered widespread attention due to its high performance and scalability. Symbolic music can be characterized as a time series of notes, each composed of multiple attributes ( e.g. , pitch, instrument). Consequently, auto-regressive modelling is widely used for symbolic music generation ( Roy, Puri, and Herremans 2025 ; Lu et al. 2023 ; Huang et al. 2018 ; Huang and Yang 2020 ) and has achieved notable success. However, ( Roy, Puri, and Herremans 2025 ) directly applying auto-regressive modelling to generate attribute sequences step-by-step (treating intra-note attributes as independent tokens) results in excessively long sequences (max sequence length = number of notes × number of attributes), making it difficult to model long-term musical structures. Some studies ( Jiwoo Ryu and Jeong 2024 ) decouple the generation process into two steps: first auto regressively generating note-level latent variables, then decoding them into multi-dimensional attributes. While this approach effectively reduces note sequence length, it compromises the quality of the generated symbolic music. To address these limitations, we argue that while treating notes as time series for auto regressive modelling is valid, intra-note attributes ( e.g. , pitch, type, velocity, instrument and duration) are inherently concurrent and temporally independent sets (unordered), rather than sequences with intrinsic temporal dependencies. Thus, the theoretical basis for auto regressively modelling intra-note attributes is questionable—for instance, existing studies ( e.g. , CPWord ( Hsiao et al. 2021 ) using type to guide generation, NMT ( Jiwoo Ryu and Jeong 2024 ) starting with metric or pitch as the initial attribute) achieve comparable generation quality, suggesting that strict temporal dependency assumptions may be unnecessary. Imposing sequential order and generating attributes one-by-one not only introduces efficiency bottlenecks but also restricts the potential for controllable symbolic music generation. To overcome these constraints, we propose Amadeus—a novel framework for symbolic music generation. Amadeus adopts a two-level architecture: an autoregressive model for the note sequence and a bidirectional discrete diffusion model at the attribute level. To enhance performance, we propose a Music Latent Space Discriminability Enhancement Strategy(MLSDES)—incorporating enhanced contrastive learning constraints to amplify the discriminability of intermediate music representations in the model. Simultaneously, to enhance note decoding accuracy, we propose the Conditional Information Enhancement Module (CIEM), which amplifies discriminative features in auto regressively generated note latent vectors through attention mechanisms and incorporates global contextual information, thereby guiding note decoding. To validate Amadeus’s modelling capability and controllability, we designed experiments across three dimensions: i) Unconditional symbolic music generation to verify foundational modelling capabilities; ii) Training-free fine-grained attribute control by leveraging attribute independence and specifiable decoding steps (specifying attribute values during decoding in the unconditional Amadeus model); iii) Text-controlled symbolic music generation. All experiments demonstrate that Amadeus significantly outperforms existing state-of-the-art methods in generation quality, condition fidelity, attribute controllability, and inference speed. Notably, the method allows flexible trade-offs between quality and speed via hyperparameter tuning. Finally, we compiled, curated, and open-sourced the largest symbolic music dataset to date: AMD (Amadeus MIDI Dataset). This dataset includes a 1.9-million-sample pre-training set and a 320,000-sample high-quality fine-tuning set with textual annotations, enabling in-depth exploration of data scale and model parameter size impacts on Amadeus’s performance. Our core contributions are summarized as follows: • We propose the Amadeus architecture—performing autoregressive modelling at the note level and bidirectional modelling at the attribute level via discrete diffusion—enhanced by the MLSDES and CIEM modules to boost representational capacity. This approach delivers robust symbolic music modelling and exceptional controllability. • Through comprehensive and fair comparisons in unconditional and controllable symbolic music generation tasks, we empirically demonstrate Amadeus’s superiority over existing methods in music quality, condition adherence, attribute controllability, and inference speed. • We compile and open-source the largest public symbolic music dataset, AMD, comprising 1.9 million pre-training samples and 320,000 annotated fine-tuning samples. 2 Related Work 2.1 Computer-Aided Composition The earliest attempt at computer-aided music composition was the Illiac Suite (later known as String Quartet No. 4 ), created using the ILLIAC I computer ( Hiller and Isaacson 1957 ) . Its fourth movement employed probabilistic models and Markov chains to generate rhythm and melody. Subsequent research ( Cope 2000 ) explored rule-based and probabilistic systems, with methods like transition matrices built on existing melodies ( Collins et al. 2011 ) . These approaches introduced mathematical formalism into music composition but suffered from poor musical quality due to limited expressive modelling capacity. Later efforts turned to machine learning and neural networks ( Nakamura et al. 2015 ; Mogren 2016 ; Raffel 2016 ) , which improved fluency but still

Read the complete text