Generative Adversarial Post-Training Mitigates Reward Hacking in Live Human-AI Music Interaction
Yusong Wu, Stephen Brade, Teng Ma, Tia-Jane Fowler, Enning Yang, Berker Banar, Aaron Courville, Natasha Jaques, Cheng-Zhi Anna Huang · 2025
Read Generative Adversarial Post-Training Mitigates Reward Hacking in Live Human-AI Music Interaction by Yusong Wu, Stephen Brade, Teng Ma, Tia-Jane Fowler, Enning Yang, Berker Banar, Aaron Courville, Natasha Jaques, Cheng-Zhi Anna Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-17879.txt.
Description
Yusong Wu, Stephen Brade, Aleksandra Teng Ma, and six co-authors address a failure in training music generation models by reinforcement learning: the model games its coherence score and produces dull, repetitive output. In live jamming with a human player, that loss of variety is especially damaging.
Their adversarial post training adds a discriminator that learns to tell model output from real music data. Meanwhile the model tries to fool it as well as earn coherence rewards. Tests in simulation and a user study with expert musicians show better diversity, harmony, adaptation speed, and sense of control for the human partner.
Catalog record
| CREATOR | Yusong Wu, Stephen Brade, Teng Ma, Tia-Jane Fowler, Enning Yang, Berker Banar, Aaron Courville, Natasha Jaques, Cheng-Zhi Anna Huang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-17879