From Audio Deepfake Detection to AI-Generated Music Detection – A Pathway and Overview
Yupei Li, Manuel Milling, Lucia Specia, Björn W. Schuller · 2024
Read From Audio Deepfake Detection to AI-Generated Music Detection – A Pathway and Overview by Yupei Li, Manuel Milling, Lucia Specia, Björn W. Schuller (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-00571.txt.
Introduction
Yupei Li and co-authors survey how to detect music made by artificial intelligence, borrowing lessons from the older field of audio deepfake detection. Their 2024 paper notes that AI music tools raise worries for the music industry, for copyright, and for artistic integrity. Good detection is the first step to guard those values.
The authors sort methods into four levels: signal level, feature level, watermark, and semantic consistency, based on the trace each method uses. They then ask which parts of deepfake detection carry over to music, and under what conditions. A second sorting covers input type, model type, target, robustness, and how easy each method is to explain. Future work is mapped at the close.
Catalog record
| CREATOR | Yupei Li, Manuel Milling, Lucia Specia, Björn W. Schuller |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-00571
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Audio Deepfake Detection to AI-Generated Music Detection – A Pathway and Overview Yupei Li, Manuel Milling, Lucia Specia, Björn W. Schuller Abstract As Artificial Intelligence (AI) technologies continue to evolve, their use in generating realistic, contextually appropriate content has expanded into various domains. Music, an art form and medium for entertainment deeply rooted in human culture, is seeing an increased involvement of AI into its production. However, the unregulated use of AI music generation (AIGM) tools raises concerns about potential negative impacts on the music industry, copyright, and artistic integrity, underscoring the importance of effective AIGM detection. This paper provides a systematic overview of existing AIGM detection methods. We first establish a four-level detection taxonomy: signal-level, feature-level, watermark, and semantic consistency, organising methods according to the type of trace they exploit. Drawing on the more mature field of audio deepfake detection, we then present a stratified transferability analysis that examines which components may or may not transfer to AIGM detection, and under what conditions. A multi-dimensional classification further organises representative methods along input modality, detection granularity, feature type, model type, detection target, robustness setting, and interpretability. We conclude by discussing implications and proposing directions for future research to address ongoing challenges in the field. PubID: pubid: © 2024 The Authors. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ I Introduction As multimodal large language models (MLLMs) like GPT-4 [ 1 ] and Whisper [ 2 ] continue to evolve rapidly and are increasingly adopted by the public, artificial intelligence-generated content (AIGC) has gained considerable popularity. Among these advancements, audio deepfake has become particularly widespread, creating challenges in the form of potential spam through voice conversion (VC) that mimics intimate voices, as well as public misinformation via text-to-speech (TTS) models that can impersonate political figures. AI-generated music (AIGM), which has overlap with audio deepfake generation, has seen rapid adoption with platforms like OpenAI’s MuseNet 1 1 1 https://openai.com/research/musenet , Jukedeck 2 2 2 https://jukedeck.com , and AIVA 3 3 3 https://www.aiva.ai . This technology carries profound implications for the music industry, sparking debates around originality, copyright, and artistic value [ 3 ] within the music community. For musicians, AI’s rapid music production capabilities risk overshadowing artist-created works, which demand significant time, effort, and personal experience [ 4 ] . Moreover, the pervasive influence of AI-generated patterns may diminish originality, as artists might unconsciously replicate repetitive AI-driven motifs, hindering innovation [ 5 ] . Additionally, the prevalence of music plagiarism has led to numerous legal disputes, highlighting the need for standardised tools for AIGM detection [ 6 ] . For audiences, there is concern that the aesthetic landscape of music could shift in favour of styles driven by AI, potentially altering public taste [ 7 ] . Given the subjective nature of musical judgment, there is a collective reluctance to let AIGM, which lacks authentic emotional depth and creativity, replace human artistry. Therefore, robust AIGM detection is essential to monitor and manage the influence of AI music production, safeguarding the creative integrity of the music community and preventing potential disruptions to both creators and consumers [ 8 ] . Among the efforts to detect AI-generated audio, the majority of attention has been directed towards audio and multimodal deepfake detection. Existing audio deepfake surveys offer fragmented perspectives: Mubarak et al. [ 9 ] focus on model performance under ASVspoof 2021 [ 10 ] and ADD 2023 [ 11 ] with limited methodological comparison; Masood et al. [ 12 ] and Lin et al. [ 13 ] treat audio as one modality among text, image, and video, limiting depth on audio-specific techniques; Dixit et al. [ 14 ] propose a high-level pipeline over a narrow set of approaches; Patel [ 15 ] examines generation mechanisms without comprehensive detection analysis; and Almutairi and Elgibreen [ 16 ] provide a structured review but only up to 2022. Meanwhile, multimodal deepfake surveys [ 17 , 18 ] address AI-generated content broadly without focusing on AIGM, and music-domain surveys [ 19 , 20 ] concentrate on generation techniques rather than detection. As summarised in Table I , to the best of our knowledge, no existing survey explicitly focuses on the detection of AIGM, highlighting a significant gap in the current literature. Although AIGM detection shares its binary classification structure with audio deepfake detection, the two tasks differ along several fundamental dimensions that warrant a dedicated treatment. First, music is an art form governed by musicological constraints such as melodic contour, harmonic progressions, cadential resolution, and long-range form [ 21 ] , which have no direct speech counterpart and require domain-specific features to exploit. Second, speech signals carry prosodic cues (formant trajectories, phonotactics, speaking rate) tied to a small phoneme inventory, whereas music signals span dense harmonic overtones, polyphonic overlap, and a wide dynamic range, making the statistical patterns that signal-level detectors rely upon fundamentally different. Third, music often integrates lyrics with poetic qualities [ 22 ] , introducing a multimodal dimension that is absent from most speech-deepfake tasks: inconsistencies between AI-generated melody and lyrics constitute a detection signal that single-modality approaches cannot access. These structural, musicological, and signal-level distinctions mean that methods and representations developed for audio deepfake detection cannot be directly transferred to AIGM, motivating the domain-specific taxonomy and transferability analysis presented in this work. TABLE I: Comparison of existing surveys related to audio deepfake detection and AI in music. Survey Year Main Focus Coverage of Audio Deepfake Detection Limitations Almutairi and Elgibreen [ 16 ] 2022 Audio deepfake detection overview General techniques and datasets Limited to studies before 2022; lacks discussion of recent methods Mubarak et al. [ 9 ] 2023 audio deepfake detection models Evaluation of models using ASVspoof 2021 and ADD 2023 Dataset scope is limited; lacks broader methodological comparison Masood et al. [ 12 ] 2023 Multimodal deepfake detection Audio considered as one modality among others Focuses on multimodal approaches; limited analysis of audio-specific techniques Lin et al. [ 13 ] 2024 Multimodal deepfake detection Includes audio alongside image and video modalities Audio detection methods are not examined in depth Dixit et al. [ 14 ] 2023 Detection pipeline for audio deepfakes High-level detection workflow Covers a relatively narrow range of techniques Patel [ 15 ] 2023 Deepfake generation and case studies Brief overview of detection methods Lacks a comprehensive analysis of detection strategies Mitra et al. [ 19 ] 2025 AI techniques in music generation Not focused on detection Primarily reviews music generation rather than detection Wei et al. [ 20 ] 2025 AI tools for music production Not focused on detection Focuses on generation technologies This work 2026 AIGM detection survey Comprehensive analysis of detection methodologies for AIGM First survey dedicated to AIGM detection Fig. 1: Five steps of music production with AIGM detection granularity mapping. Composition , where scattered melodies, harmonies, and rhythms are created to form basic music components; Arrangement , where these components are organised into complete pieces with selection of instruments and harmonic structures. These two stages (in green) generate the core musical content and leave characteristic AI traces such as statistically regular note distributions, structurally shallow harmonic progressions, and formulaic arrangement patterns that are targeted by semantic-level,