From Discord to Harmony: Decomposed Consonance-based Training for Improved Audio Chord Estimation

2025

Read From Discord to Harmony: Decomposed Consonance-based Training for Improved Audio Chord Estimation (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01588.txt.

Introduction

Naming the chords in a recording is a core task in music analysis, yet systems have hit a ceiling that better models alone have not broken. Andrea Poltronieri, Xavier Serra, and Martin Rocamora trace part of that ceiling to the labels themselves. Annotators disagree about chords, and chord data sets over-represent some chord classes while starving others.

Their first contribution measures agreement between annotators with richer tools than simple match counts. They also propose a distance metric informed by consonance, how pleasant two chords sound together, so near misses count for less than wild ones. Analysis suggests that metric tracks musical meaning better. On that base the team builds consonance-based training, using the perceptual distance to teach models. The work aims to unstick chord estimation by fixing how harmony is judged, not only how it is predicted.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-01588

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Discord to Harmony: Decomposed Consonance-based Training for Improved Audio Chord Estimation Abstract Audio Chord Estimation (ACE) holds a pivotal role in music information research, having garnered attention for over two decades due to its relevance for music transcription and analysis. Despite notable advancements, challenges persist in the task, particularly concerning unique characteristics of harmonic content, which have resulted in existing systems’ performances reaching a glass ceiling. These challenges include annotator subjectivity, where varying interpretations among annotators lead to inconsistencies, and class imbalance within chord datasets, where certain chord classes are over-represented compared to others, posing difficulties in model training and evaluation. As a first contribution, this paper presents an evaluation of inter-annotator agreement in chord annotations, using metrics that extend beyond traditional binary measures. In addition, we propose a consonance-informed distance metric that reflects the perceptual similarity between harmonic annotations. Our analysis suggests that consonance-based distance metrics more effectively capture musically meaningful agreement between annotations. Expanding on these findings, we introduce a novel ACE conformer-based model that integrates consonance concepts into the model through consonance-based label smoothing. The proposed model also addresses class imbalance by separately estimating root, bass, and all note activations, enabling the reconstruction of chord labels from decomposed outputs. 1 Introduction In Western music theory, chords denote simultaneous combinations of three or more notes, forming harmonic structures integral to musical composition and analysis [ 1 , 2 , 3 , 4 ] . However, manually annotating chords from audio recordings is a labour-intensive task requiring music professionals’ expertise. Consequently, Audio Chord Estimation (ACE) emerged as a crucial task in Music Information Retrieval/Research (MIR) to automate chord transcription from audio due to its relevance for its numerous applications in music transcription and analysis. The research in ACE has witnessed more than two decades of exploration, but despite the important advancements achieved [ 5 ] , performance results have stagnated in recent years, leading some researchers to suggest that the task has hit a glass ceiling [ 6 ] . These challenges stem from several significant open problems [ 5 ] , which are fundamentally linked to the complex nature of harmonic content and its representation within audio signals. One such challenge is the chord vocabulary imbalance, stemming from the unequal frequency of occurrence among chord labels. For instance, in ChoCo [ 7 ] , the most extensive corpus of chord annotations to date, approximately 74.9\% of the distribution of the 8064 distinct chord classes is dominated by just major, minor, major seventh, minor seventh and dominant seventh chord types. Another critical challenge is inter-annotator agreement, which arises from the inherent ambiguity in what constitutes a chord from a musical perspective and the subjective nature of human annotation processes. For example, a clear distinction between a chord sequence and a melodic line can be subject to individual interpretation. Moreover, there is significant variance among annotators regarding the level of detail in annotating chord sequences [ 8 ] . Various studies have investigated inter-annotator agreement in chord annotation, reporting agreement rates for the root note ranging from 76\% [ 8 ] to 92\% [ 9 ] , using different datasets and numbers of annotators. Such evaluations typically use binary metrics to compare labels, but penalising the agreement evaluation equally for every discrepancy can be inappropriate [ 10 ] . Indeed, binary evaluation risks overlooking harmonic aspects that might be shared among chord sequences, although annotated differently. As a preliminary contribution of this paper, we analyse patterns of inter-annotator disagreement in chord annotation. Our analysis reveals that when annotators disagree, their chord labels tend to be harmonically related rather than randomly different. Specifically, we find that disagreements commonly occur between chords that share significant harmonic content (c.f. Section 3.1 ). Building upon these insights, we propose a method for incorporating such information into the supervised training of ACE systems. Hence, we introduce a novel model integrating consonance-based label smoothing [ 11 ] (c.f. Section 3.2.2 ). To tackle the class imbalance issue, instead of mapping audio features to a predetermined vocabulary of chord labels, we adopt an approach inspired by [ 12 ] , in which the chord root, bass, and all note activations are classified separately. The final predicted chord label is derived from decoding these three sets of information without explicitly imposing any vocabulary on it (c.f. Section 3.2.1 ). The proposed model leverages the Conformer architecture [ 13 ] , which has recently been explored in several music audio applications [ 14 , 15 , 16 ] . We demonstrate that the proposed model performs better than the state-of-the-art approaches, especially when evaluated using non-binary and consonance-based distance metrics (c.f. Section 4 ). 2 Related Work Since Fujishima’s early work [ 17 ] , chord recognition has followed knowledge-driven approaches [ 18 ] , typically extracting chroma [ 19 ] or Tonnetz features [ 20 ] , and classifying them via HMMs, DBNs [ 19 ] , or CRFs [ 21 ] . With the emergence of deep learning, various architectures have been explored for the task, including Convolutional Neural Networks (CNNs) [ 12 , 21 ] , Recurrent architectures (RNN) [ 22 ] , Convolutional Recurrent Neural Networks (CRNNs) [ 23 ] , and Transformers [ 24 ] . While deep-learning approaches have surpassed traditional knowledge-driven ones, several challenges must be tackled. Most of the proposed approaches to addressing the chord class imbalance challenge can be divided into two categories: chord simplification and chord decomposition. The former reduces the size of the chord vocabulary by converting complex chord labels into simpler representations. Notably, the vast majority of studies have adopted restricted vocabularies of approximately 25 symbols, encompassing major-minor chords [ 17 , 18 ] . Chord decomposition strategies focus on predicting the chord constituting components separately, and then map them to templates to predict the final chord [ 12 , 25 , 23 ] . Some additional approaches do not fall into these two categories, like addressing the unequal distribution of chords through a balanced learning process [ 26 ] , or using a curriculum learning training scheme to begin with simple chord qualities and then move to more complex and less common ones [ 27 ] . The inter-annotator agreement in chord annotation continues to pose a significant challenge. Despite existing diagnoses and quantification of this phenomenon in the literature [ 8 , 9 ] , definitive solutions have yet to emerge. Clercq et al. [ 9 ] observe an inter-annotator agreement rate of 94\% for the root note between two different annotations of the top 20 tracks from Rolling Stone magazine’s list of the 500 Greatest Songs of All Time . In contrast, Koops et al. [ 8 ] report an inter-annotator agreement rate of 76\% for the root note on four different annotations of a 50-song subset of the Billboard dataset [ 28 ] . To address annotation subjectivity, Koops et al. [ 29 , 8 ] propose a personalised chord estimation framework that adapts labels to individual annotator vocabularies. Their method computes Shared Harmonic Interval Profiles (SHIPs) from multiple reference annotations aligned with CQT frames and trains a neural network to predict user-specific chord labels, offering an alternative to fixed-vocabulary systems. While

Read the complete text