Adaptive Accompaniment with ReaLchords

Yusong Wu, Tim Cooijmans, Kyle Kastner, Adam Roberts, Ian Simon, Alexander Scarlatos, Chris Donahue, Cassie Tarakajian, Shayegan Omidshafiei, Aaron Courville, Pablo Samuel Castro, Natasha Jaques, Cheng-Zhi Anna Huang · 2025

Read Adaptive Accompaniment with ReaLchords by Yusong Wu, Tim Cooijmans, Kyle Kastner, Adam Roberts, Ian Simon, Alexander Scarlatos, Chris Donahue, Cassie Tarakajian, Shayegan Omidshafiei, Aaron Courville, Pablo Samuel Castro, Natasha Jaques, Cheng-Zhi Anna Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-14723.txt.

Introduction

Yusong Wu and a team of co-authors present a study from twenty twenty-five on a music model that plays chords live with a human. Jamming needs timing, foresight, and shared flair. Most music models write a full piece offline and cannot play in the moment. Their model, ReaLchords, builds chord backing for a user's tune as the tune unfolds.

They first train the model in the standard way on past music. Reinforcement learning then tunes it for live play. A reward model grades how well chords fit the tune in pitch and in time. A teacher that can see the tune ahead guides the tuning too. Tests and listener trials show the model adapts to new tunes and plays fitting chords.

Catalog record

CREATORYusong Wu, Tim Cooijmans, Kyle Kastner, Adam Roberts, Ian Simon, Alexander Scarlatos, Chris Donahue, Cassie Tarakajian, Shayegan Omidshafiei, Aaron Courville, Pablo Samuel Castro, Natasha Jaques, Cheng-Zhi Anna Huang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-14723

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adaptive Accompaniment with ReaLchords Yusong Wu, Tim Cooijmans, Kyle Kastner, Adam Roberts, Ian Simon, Alexander Scarlatos, Chris Donahue, Cassie Tarakajian, Shayegan Omidshafiei, Aaron Courville, Pablo Samuel Castro, Natasha Jaques, Cheng-Zhi Anna Huang Abstract Jamming requires coordination, anticipation, and collaborative creativity between musicians. Current generative models of music produce expressive output but are not able to generate in an online manner, meaning simultaneously with other musicians (human or otherwise). We propose ReaLchords , an online generative model for improvising chord accompaniment to user melody. We start with an online model pretrained by maximum likelihood, and use reinforcement learning to finetune the model for online use. The finetuning objective leverages both a novel reward model that provides feedback on both harmonic and temporal coherency between melody and chord, and a divergence term that implements a novel type of distillation from a teacher model that can see the future melody. Through quantitative experiments and listening tests, we demonstrate that the resulting model adapts well to unfamiliar input and produce fitting accompaniment. ReaLchords opens the door to live jamming, as well as simultaneous co-creation in other modalities. Figure 1 : Online models finetuned with RL are able to recover from mistakes, while models trained with MLE alone do not. We take a melody from the test set and midway introduce an abrupt transposition designed to disrupt the accompaniment model (top row). The Online MLE model predicts a bad chord (B7) and fails to adapt. ReaLchords also predicts a bad chord (F \sharp m), but adapts quickly. Wrong chords highlighted in orange are our own judgment informed by music theory, but the overall pattern is corroborated by an objective measure of harmonic quality, averaged over many trials of this experiment (bottom row). 1 Introduction Deep generative models produce realistic, high-quality content, and are seeing increasing integration into the creative processes of artists. However, such models tend not to be designed for the demands of live scenarios such as interactive improvisation, which requires anticipation of others’ intentions and adaptation to mistakes, stylistic choices and deliberate exploration. We focus on music in particular, which is inherently interactive and dynamic and revolves around anticipatory collaboration in ensemble settings. Most existing models in this space, while capable of creating expressive compositions or accompaniments, are not suited for simultaneous creation, where adaptation to and alignment with the ongoing musical structure are crucial. This paper introduces ReaLchords , a generative model tailored for online adaptive musical accompaniment. Emulating the spontaneity of live music jamming, ReaLchords generates chord accompaniments in response to a stream of monophonic melody notes, adapting on-the-fly to the unfolding musical narrative. Each chord must be generated without knowing in advance which melody note it will accompany. This simultaneous interaction imposes a conditional independence assumption on the joint generative process, that an online model must respect. Moreover, a model must be able to gracefully handle unfamiliar situations and unexpected changes. Likelihood models, however, suffer from exposure bias due to being trained entirely on ground truth data, and transfer poorly to the online settings where mistakes, imperfections and stylistic differences are common (see Figure 1 for an example). To address this, we use RL finetuning to improve the model with respect to reward models that consider musical coherence (§ 3.2 ). These reward models see the entire composition and evaluate its musical coherence from various perspectives (§ 3.3 ). Our setup bears similarities to RLHF ( Ouyang et al. 2022 ; Jaques et al. 2019 ) and RLAIF ( Bai et al. 2022 ; Lee et al. 2023 ) , however our reward models are trained through self-supervision rather than human labeling. Finally, we combine RL finetuning with knowledge distillation ( Agarwal et al. 2023 ; Zhou et al. 2023 ) in a novel way, distilling from a teacher that can see the future into a student that cannot, hence forcing anticipation (§ 3.4 ). We develop key algorithmic components (Figure 2 ) needed to produce an online adaptive accompaniment model that is amenable to interactive use. Our contributions and findings are as follows: 1 1 1 Listen to audio examples here: https://storage.googleapis.com/realchords/index.html . \sharp We propose ReaLchords , an online accompaniment generation model trained by RL finetuning. Figure 1 shows how ReaLchords adapts to out-of-distribution input, a necessary skill for live jamming. \sharp We leverage knowledge distillation to learn from a non-causal teacher that can see the future (§ 3.4 ). Distillation greatly improves the quality of the model, as evidenced by the human evaluation shown in Figure 3 . \sharp We further employ a novel set of self-supervised reward models to encourage musical coherence and perceptual quality (§ 3.3 ). Based on a human listening test, we show that our reward models align closely with human preferences (Figure 3 ), despite being trained without human feedback (§ 3.3 ). \sharp We demonstrate through a series of controlled experiments that without RL finetuning, models fail to adapt to mistakes and perturbations (Figure 4 , § 5.4 ). \sharp Finally, we analyze the behavior of our models in terms of domain-specific metrics (Table 1 , § 5.3 ). We find that each component in our RL finetuning methods improves the rhythmic and harmonic quality of generated accompaniments. 2 Related Work Adaptive music accompaniment systems In contrast to automatic music generative systems, accompaniment systems often take input (such as melody) from a user, and generate output that is meant to be played in synchrony to complement what the user is playing. Some of these systems are asynchronous, where the user first provides the full melody, and the system generates an accompaniment offline. Examples include MySong ( Simon et al. 2008 ) , where a user sings a melody and the system generates chords to accompany them. Most recently, SingSong ( Donahue et al. 2023 ) supports a very similar interaction, but generates full-band backing tracks. Both are offline systems. In contrast, online accompaniment systems need to synchronize with user actions in real-time. Score-following is a special case where the system has the score, the full context of the content of what the musician will play, but still needs to follow along and infer when to play their own part. Music Plus One ( Raphael 2010 ) adapts its playback speed of an orchestral recording (without the soloist) to a soloist’s expressive performance. Similarly, Antescofo ( Cont 2008 ) follows where a soloist is in a score and triggers live electronics accordingly. Generative accompaniment systems or more generally co-creative music systems, not only have to anticipate user actions, they need to learn how to respond. Voyager ( Lewis 2003 ) takes a rule-based approach in how to listen, respond and generate musical material on the fly, while Omax Brothers ( Assayag et al. 2006 ) recombines what a musician plays on-the-fly as an accompaniment but often requires another computer musician to control when it comes in and what section of material to draw from. ImproteK and later DJazz ( Nika & Chemillier 2012 ; Nika et al. 2017 ) leverages a shared predefined chord progressions (such as a Jazz Standard) to coordinate the human-machine improvisation. Instead of tight synchronization, Spire Muse ( Thelle & Pasquier 2021 ) serves as a brainstorming partner

Read the complete text