Amuse : Human-AI Collaborative Songwriting with Multimodal Inspirations

Yewon Kim, Sung-Ju Lee, Chris Donahue · 2024

Read Amuse : Human-AI Collaborative Songwriting with Multimodal Inspirations by Yewon Kim, Sung-Ju Lee, Chris Donahue (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-18940.txt.

Introduction

Kim, Lee, and Donahue built Amuse, a tool that helps songwriters turn pictures, text, or sounds into chord patterns. Song ideas often start from such mixed sources, the authors note, yet most music software cannot use them. Their 2024 system takes in an image, a passage, or an audio clip and returns chords that fit the mood of the source.

No large set of paired examples exists for this task, so the team uses a large multimodal model to suggest rough chords first. A smaller chord-only model then filters those guesses into a smooth, usable pattern. In a study with songwriters, Amuse helped users turn mixed ideas into music they could build on. Users also felt more control and more creative with the tool.

Catalog record

CREATORYewon Kim, Sung-Ju Lee, Chris Donahue
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE133 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-18940

Browse the full library · About the Tower of Babel