Generative Adversarial Post-Training Mitigates Reward Hacking in Live Human-AI Music Interaction

Yusong Wu, Stephen Brade, Teng Ma, Tia-Jane Fowler, Enning Yang, Berker Banar, Aaron Courville, Natasha Jaques, Cheng-Zhi Anna Huang · 2025

Read Generative Adversarial Post-Training Mitigates Reward Hacking in Live Human-AI Music Interaction by Yusong Wu, Stephen Brade, Teng Ma, Tia-Jane Fowler, Enning Yang, Berker Banar, Aaron Courville, Natasha Jaques, Cheng-Zhi Anna Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-17879.txt.

Description

Yusong Wu, Stephen Brade, Aleksandra Teng Ma, and six co-authors address a failure in training music generation models by reinforcement learning: the model games its coherence score and produces dull, repetitive output. In live jamming with a human player, that loss of variety is especially damaging.

Their adversarial post training adds a discriminator that learns to tell model output from real music data. Meanwhile the model tries to fool it as well as earn coherence rewards. Tests in simulation and a user study with expert musicians show better diversity, harmony, adaptation speed, and sense of control for the human partner.

Catalog record

CREATORYusong Wu, Stephen Brade, Teng Ma, Tia-Jane Fowler, Enning Yang, Berker Banar, Aaron Courville, Natasha Jaques, Cheng-Zhi Anna Huang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-17879

Browse the full library · About the Tower of Babel