AI-Assisted Music Production: A User Study on Text-to-Music Models
Francesca Ronchini, Luca Comanducci, Simone Marcucci, Fabio Antonacci · 2025
Read AI-Assisted Music Production: A User Study on Text-to-Music Models by Francesca Ronchini, Luca Comanducci, Simone Marcucci, Fabio Antonacci (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23364.txt.
Introduction
Text-to-music tools promise fast creation, but how they change real producer workflows is little studied. Francesca Ronchini and 3 co-authors ran a user study on that question. Producers made tracks with a custom tool that joins text-to-music and source separation models. Semi-structured interviews followed the sessions. Thematic analysis sorted what producers said into patterns.
Challenges, opportunities, and ethical concerns all surfaced. The findings show where these models help and where they grate in practice. Integration into daily workflows lags the raw capability. The study is a case study, with a small producer pool. It offers grounded insight for tool builders and musicians.
Catalog record
| CREATOR | Francesca Ronchini, Luca Comanducci, Simone Marcucci, Fabio Antonacci |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-23364
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AI-Assisted Music Production: A User Study on Text-to-Music Models Francesca Ronchini, Luca Comanducci, Simone Marcucci, Fabio Antonacci Abstract Text-to-music models have revolutionized the creative landscape, offering new possibilities for music creation. Yet their integration into musicians’ workflows remains underexplored. This paper presents a case study on how TTM models impact music production, based on a user study of their effect on producers’ creative workflows. Participants produce tracks using a custom tool combining TTM and source separation models. Semi-structured interviews and thematic analysis reveal key challenges, opportunities, and ethical considerations. The findings offer insights into the transformative potential of TTMs in music production, as well as challenges in their real-world integration. 0 0 footnotetext: All rights remain with the authors under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Proc. of the 17th Int. Symposium on Computer Music Multidisciplinary Research, London, United Kingdom, 2025 1 Introduction Deep learning is rapidly advancing music generation, particularly through Text-to-Music (TTM) models that generate audio from text prompts [ 47 , 31 ] . These models are transforming music composition and production [ 16 , 22 ] , enabling creative experimentation even for those without formal training, as seen with Suno [ 35 ] and Udio [ 38 ] . Despite advancements in AI music generation, TTM models still struggle to interpret musicians’ controls [ 44 , 30 ] , underscoring the need for focused research on creator interaction and collaboration. Ronchini et al. [ 30 ] conducted a user experience study exploring how musicians interact with a TTM model enhanced by fine-tuning techniques to adapt to users’ specific preferences. However, the study was constrained by a homogeneous participant pool and a broad focus. Parallel to this paper, Fu et al. [ 12 ] conducted a similar investigation focused on music production, using a limited sample of university students with little to no production experience, where pressure due to deadlines and assignments may have influenced the results. Figure 1 : Workflow of the experimental procedure. The red lines mean optional actions, while the green lines mean the reiteration process. This paper overcomes previous limitations, proposing a study to better investigate the interpretation gap [ 44 ] by conducting a user case study on AI-assisted music production. The research involves 17 music producers with varying levels of experience, and evaluate how TTM models influence productivity and effectiveness, the usability of AI tools as creative collaborators, and the overall perspective on incorporating TTM models into creative workflow. Participants were invited to use their preferred setups to produce a musical track with the help of a custom-developed tool that integrates a TTM model and an AI-based source separation model, providing producers with greater control and flexibility over the generated composition [ 33 ] . Fig. 1 illustrates the workflow of the experiment. We believe these results offer insightful perspectives on user experiences and ethical implications of TTM models. Supplementary material, including videos of interaction and code for replication, are available online . 2 Background Text-to-music Models . Several text-based generative models, based on different approaches, have been proposed for the raw audio and music generation task [ 47 ] . The first model proposed is AudioLM [ 2 ] , which was further developed in MusicLM [ 1 ] . Subsequently, transformer-based models have been proposed [ 17 , 7 , 49 ] , and diffusion-based models have gained attention [ 43 , 19 , 10 ] . Recently, commercial solutions like Suno AI [ 35 ] and Udio [ 38 ] have been introduced. However, we opted not to consider them in order to adhere to the principles of open science, enabling fellow researchers to reproduce our experiments. Moreover, the use of closed-source models would have prevented us from integrating the source separation technique into a single interface. In recent research, control inputs such as tempo, rhythm, chords, and melody have been integrated alongside text [ 41 , 18 , 37 ] . As text remains the primary control in these models, this study focuses on text-based approaches. For a more detailed survey, the reader is referred to [ 47 , 22 ] . Music Source Separation Models . Data-driven models for Music Source Separation are categorized into spectrogram-based [ 36 ] and waveform-based models [ 9 ] . In [ 8 ] , the authors propose hybrid approaches that combine both temporal and spectral domains. Spleeter [ 14 ] leverages pre-trained models. Recently, Hybrid Transformer Demucs (HT-Demucs) [ 32 ] , inspired by Hybrid Demucs [ 8 ] , and Banquet [ 39 ] have been introduced. Most models separate audio into four stems: vocals, drums, bass, and other (VDBO). Spleeter adds piano for five stems, HT-Demucs includes guitar for six, and Banquet generates multiple stems [ 39 ] . Co-creativity and Human-AI interaction . Several studies explore Human-AI interaction in music generative models [ 24 , 20 ] , the perception of AI-generated music [ 4 , 34 ] , and opportunities and challenges of AI for music [ 4 , 26 ] . In [ 21 ] , the authors evaluate generative models and AI-steering interfaces in downstream creative tasks. Zhou et al. [ 48 ] explore ways to enhance user interaction by allowing direct control over a model’s sampling behavior, while Huang et al. [ 26 ] study human-AI co-creation, identifying challenges musicians face when composing with AI. Several interfaces have been proposed to improve interaction between generative models and users [ 3 , 20 , 29 , 46 , 48 , 42 ] , AI-assisted music composition [ 28 ] , along with creative supporting tools designed to enhance collaborative creativity [ 5 , 11 , 16 ] . However, research of TTM integration into music creation workflows remains limited, but crucial [ 30 , 12 , 44 ] . 3 Study design and method Models and framework selection . We chose MusicGen [ 7 ] for music generation and HT Demucs [ 32 ] for source separation due to their performance trade-offs and easy integration into a unified framework, ensuring reproducibility. MusicGen is a text- or melody-conditioned autoregressive model available in three sizes, while HT Demucs [ 32 ] uses dual U-Nets and a Transformer to separate instruments. We use the htdemucs_6s model, which separates VDBO, guitar and piano. Combining music separation with generation enhances composition allows independent manipulation of individual sources. MusicGen-Stem [ 33 ] , which integrates both functions, has been recently released. However, it was released during our study and supports only three stems. While recent studies provide additional controls for TTM models [ 41 , 37 ] , text instructions continue to be the dominant approach. Given this and the study’s limited duration, we decided to focus the study on models that use text as input. Hence, we do not use the melody conditioning of MusicGen. Interface design . (a) (b) Figure 2 : User interface: (a) music generation model; (b) source separation model. We designed the interface based on the AI controllability exploration principles proposed in [ 40 ] . Inspired by [ 30 , 16 ] , we implemented multiple output principles together with general and domain-specific controls, allowing users to explore a