Echoes of Humanity: Exploring the Perceived Humanness of AI Music

Flavio Figueiredo, Giovanni Martinelli, Henrique Sousa, Pedro Rodrigues, Frederico Pedrosa, Lucas N. Ferreira, Universidade Federal de Minas Gerais (UFMG) · 2025

Read Echoes of Humanity: Exploring the Perceived Humanness of AI Music by Flavio Figueiredo, Giovanni Martinelli, Henrique Sousa, Pedro Rodrigues, Frederico Pedrosa, Lucas N. Ferreira, Universidade Federal de Minas Gerais (UFMG) (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-25601.txt.

Introduction

Flavio Figueiredo and co-authors tested in 2025 whether listeners can tell AI-made songs from human ones. AI music services are reshaping the industry, so perception matters twice: spotting machine songs, and improving the machines. Their experiment was a blind, Turing-style trial. Each listener heard a pair of songs and picked which one the AI wrote.

The design crossed listeners over randomized pairs, with similarity between the paired songs controlled. That control allowed a causal reading, rare in this literature. Reliability rose when the two songs in a pair were alike, forcing finer judgment. The AI songs came from real commercial use, not from the authors' own prompts. Written feedback showed listeners leaning on vocal and technical cues. The study gives educators and model builders a cleaner baseline.

Catalog record

CREATORFlavio Figueiredo, Giovanni Martinelli, Henrique Sousa, Pedro Rodrigues, Frederico Pedrosa, Lucas N. Ferreira, Universidade Federal de Minas Gerais (UFMG)
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-25601

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Echoes of Humanity: Exploring the Perceived Humanness of AI Music Flavio Figueiredo, Giovanni Martinelli, Henrique Sousa, Pedro Rodrigues, Frederico Pedrosa, Lucas N. Ferreira, Universidade Federal de Minas Gerais (UFMG) Abstract Recent advances in AI music (AIM) generation services are currently transforming the music industry. Given these advances, understanding how humans perceive AIM is crucial both to educate users on identifying AIM songs, and, conversely, to improve current models. We present results from a listener-focused experiment aimed at understanding how humans perceive AIM. In a blind, Turing-like test, participants were asked to distinguish, from a pair, the AIM and human-made song. We contrast with other studies by utilizing a randomized controlled crossover trial that controls for pairwise similarity and allows for a causal interpretation. We are also the first study to employ a novel, author-uncontrolled dataset of AIM songs from real-world usage of commercial models (i.e., Suno). We establish that listeners’ reliability in distinguishing AIM causally increases when pairs are similar. Lastly, we conduct a mixed-methods content analysis of listeners’ free-form feedback, revealing a focus on vocal and technical cues in their judgments. 1 Introduction In the book I, Robot (1950), Isaac Asimov wrote a passage stating that: “you just can’t differentiate between a robot and the very best of humans.”. This quote offers a provocative view on the evolution of AI. In a way, Asimov challenges us to consider that AI could become comparable to the very best of humanity. Over the last few decades, we’ve indeed seen AI outperform humans in several objective tasks with a clear score (e.g., image recognition or playing Go) [ 17 , 29 , 8 , 32 , 33 ] . However, when we shift our focus to artistic content, the subjectivity of aesthetic assessment makes it such that we no longer have a single score of correctness [ 25 ] . In these settings, Asimov’s statement becomes even more grounded, as we shift from improvements in scores to subjective differentiation. This poses the techno-philosophical question of what differentiates authentic (human) from artificial (AI). From a musical perspective, this question can be rephrased as: What Sounds Like an AI? To understand its relevance, consider the recent advances in AI Music (AIM). Nowadays, services such as Suno 1 1 1 https://suno.ai and Udio 2 2 2 https://udio.com base their business models on generating songs from textual prompts. These services are changing several aspects of music composition [ 25 ] , production [ 21 ] , and consumption [ 13 ] . In fact, synthetic music projects that release exclusively AIM, e.g., The Velvet Sundown [ 5 ] , are now reaching millions of streams. Within this context, understanding how humans perceive AIM is essential not only for educating users on how to identify AIM, but also, conversely, to improve AI models. Thus, motivated by these industry disruptive advances, we focus on investigating the hypothesis below: Human listeners rely on contextually grounded cues (e.g., as repetitive structure or synthetic-sounding vocals) that help discern whether a piece of music is AIM or human-made. We study this hypothesis based on a listener-focused Turing [ 34 ] -like blind test, in which listeners are exposed to a human-made song and an AIM song. While similar listening experiments may be employed to assess new AIM models [ 11 , 18 ] , in this case, our object of study is not a model or algorithm, but the listener themself. Specifically, consider that confusion between AIM and human-made songs is commonly regarded as a positive outcome for a new model. Our study does not seek this result. Our focus lies in a deeper examination of when and how listeners differentiate. To tackle the when , we make use of a randomized controlled crossover trial [ 20 ] . In this trial, some pairs of songs are highly similar. This is the treatment group. Other pairs are uniformly random pairs of songs. The control group. Listeners also evaluate multiple pairs of songs — the longitudinal aspect — wherein the order of pairs is randomized for each listener. The initial audio in each pair is also randomized. To address how listeners differentiate, we explore a mixed-methods [ 12 ] grounded-theory content analysis [ 10 ] on the text feedback provided on why they made their choices. Unlike other studies [ 16 , 31 , 18 ] , and to reduce bias, we explore a novel in-the-wild-style dataset of AIM songs, sourced from users sharing AIM on Reddit 3 3 3 https://reddit.com/r/SunoAI/ . In this sense, AIM songs are derived from real-world usage without any prior connection to our research. Human-made songs were obtained from independent artists on Jamendo 4 4 4 https://jamendo.com via the MTG-Jamendo dataset [ 4 ] . This dataset was gathered in 2019 and predates the commercial rise of AIM (Suno was released in 2023). To gather a diverse set of feedback, our study sampled from two distinct listener populations: one composed of volunteers initially recruited from (but not limited to) the Computer Science and Music Departments of a major Brazilian university, and another composed of crowd-workers hired through Prolific 5 5 5 https://prolific.com/ . Our study shows that when pairs are random listeners cannot differentiate AIM from human-made songs, i.e., they are no better than random guessing. Nevertheless, when pairs are highly similar , listeners are able to make this distinction. Our results also show that longer exposure to practical musical experience (e.g., playing an instrument) and prior knowledge of AIM also increase the chance of AIM identification. Our content analysis indicates that a focus on vocal and technical aspects again leads to an increases on AIM identification. These results have implications on how to make AIM sound more human-like and on how we may be used to educate users on how to identify AIM. We now discuss related work. Section 3 describes our experiment, Section 4 results, and Section 5 conclusions. Appendices A to E present an ethical statement, limitations, and complementary results. 2 Related Work With the rise of generative models in recent years, the intersection of AI and art in general has attracted significant attention [ 25 , 7 , 15 , 38 , 19 , 23 ] . As our work focuses on musical content, the discussion in this section will primarily address music-related studies. Before doing so, we note that perception studies for images have existed since at least the 1960s. In particular, the seminal work of Michael Noll [ 27 ] who closely mimicked one of Mondrian’s work, Composition with Lines (1917) , using a computer. In this very particular setting, participants often could not tell what as human-made versus computer-generated. Recent studies reach similar conclusions when examining generative AI for images [ 14 , 28 , 37 ] and poetry [ 22 ] . More focused on human perception rather than accuracy, as we are , Candello et al. [ 6 ] found that humans perceive some typefaces as more machine-like. These efforts, however, steer the participant toward focusing on particular characteristics, such as beauty or novelty . The authors also do not employ a randomized controlled crossover trial (RCCT) [ 20 ] as we do. Using the RCCT here causally shows

Read the complete text