A Hierarchical Deep Learning Approach for Minority Instrument Detection
2025
Read A Hierarchical Deep Learning Approach for Minority Instrument Detection (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-21167.txt.
Introduction
Dylan Sechet and colleagues present a hierarchical deep learning approach for detecting rare instruments in audio. Their 2025 arXiv paper serves music information retrieval. Knowing which instruments play in a clip aids cataloging and discovery. Past models focused on common instruments with plenty of training data. Rare instruments have few labeled examples to learn from.
The method builds on the Hornbostel-Sachs classification of instruments. A hierarchy lets the model predict broad instrument groups as well as fine classes. The team tested ways to build that structure into the model. They used the MedleyDB dataset, known for its range of instruments and genres. Results showed more reliable detection at the coarse level. The approach narrows the gap between fine identification and group recognition.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-21167
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Hierarchical Deep Learning Approach for Minority Instrument Detection Abstract Identifying instrument activities within audio excerpts is vital in music information retrieval, with significant implications for music cataloging and discovery. Prior deep learning endeavors in musical instrument recognition have predominantly emphasized instrument classes with ample data availability. Recent studies have demonstrated the applicability of hierarchical classification in detecting instrument activities in orchestral music, even with limited fine-grained annotations at the instrument level. Based on the Hornbostel-Sachs classification, such a hierarchical classification system is evaluated using the MedleyDB dataset, renowned for its diversity and richness concerning various instruments and music genres. This work presents various strategies to integrate hierarchical structures into models and tests a new class of models for hierarchical music prediction. This study showcases more reliable coarse-level instrument detection by bridging the gap between detailed instrument identification and group-level recognition, paving the way for further advancements in this domain. 1 Introduction The identification of instruments within an audio excerpt poses an enduring challenge in the field of Music Information Retrieval (MIR). This task is inherently intricate due to the poly-instrumental nature of real-world music, where the pitches of multiple instruments often intertwine. Furthermore, the task is complicated by substantial variations in timbre and performance style among instruments, further hindering recognition endeavors. Even trained musicians may encounter perceptual similarities among specific instruments, adding another layer of complexity to the recognition process. Instrument identification bears significant implications across various domains, including music cataloging and discovery. It aids in tasks such as song retrieval [ 1 ] , facilitates genre recognition systems [ 2 ] , and contributes to recommendation systems [ 3 ] . While the recognition of more common instruments benefits from abundant available data, challenges arise in genres such as orchestral or opera, as well as with rare or non-western instruments, for which data is much more scarce. Hierarchical classification systems have been proposed to address the complexities of instrument recognition. These systems enable the prediction of instruments at various levels of specificity, demonstrating particular promise in handling imbalanced datasets and scenarios involving few-shot learning [ 4 ] . However, existing work in this domain has been confined to specific genres and a restricted set of instruments. This study assesses the scalability of hierarchical approaches on a more complex dataset, like MedleyDB, and proposes different formulations of the hierarchical problem. 1.1 Instrument detection The field of instrument detection incorporates a diverse array of methodologies, spanning from signal processing techniques to contemporary deep learning approaches. Meanwhile, multi-label classification for audio signals has attracted considerable interest across various domains [ 5 ] . In the 2000s, Marques et al. [ 6 ] conducted instrument classification on brief 0.2s music excerpts utilizing Gaussian Mixture Models (GMM) and Support Vector Machines (SVM), with features extracted through Mel-Frequency Cepstral Coefficients (MFCC). In a similar vein, Essid et al. [ 7 ] showcased the advantages of GMMs over SVMs by employing MFCC features preprocessed with Principal Component Analysis (PCA), even in the context of longer mono-instrument samples. More recently, Deep learning models, which have seen successful applications across various domains [ 8 ] , have demonstrated promise in mono-instrument detection as well. Initially designed for image recognition tasks, Convolutional Neural Networks (CNNs) have been effectively repurposed to handle spectrogram-like features such as MFCCs or Constant-Q Transforms. Solanky et al. highlighted in [ 9 ] the efficiency of an AlexNet-inspired CNN model in predominant instrument recognition, while in [ 10 ] , Avramidis et al. introduced performance enhancements in instrument recognition by integrating recurrent components into CNN architectures. Attention-based models have also emerged as an promising alternative to CNNs in the field of audio classification: transformers, initially introduced for text classification [ 11 ] , have been adapted to image recognition tasks [ 12 ] . Jamil et al. [ 13 ] used a vision transformer architecture for audio classification to distinguish harmless from malicious drones. In the domain of Music Information Retrieval, Regunath et al. [ 5 ] were able to outperform a CNN architecture using a vision transformer for predominant instrument recognition in polyphonic settings. 1.2 Hierarchical classification for audio Exploration of hierarchical structures for classifying audio segments has been a subject of prior research across diverse domains. For instance, hierarchical methods have been employed in classifying bird songs, with a class tree rooted in biological taxonomy [ 14 ] . Notably, these methods operated on more extended audio excerpts than the frame-level analysis. In the realm of Music Information Retrieval, Fu et al. introduced in [ 15 ] a hierarchical approach tailored for singing voice classification and transcription. On the other hand, Essid et al. [ 7 ] explored hierarchical classification using GMMs, focusing primarily on synthetic music extracts rather than actual recordings and not at the frame level. In a recent study in 2023, Krause et al. [ 16 ] delved into hierarchical classification methods explicitly designed for orchestral and opera pieces. Their research demonstrated performance improvements, particularly in scenarios with limited fine-grained annotations. Furthermore, Garcia et al. [ 4 ] investigated hierarchical classification for few-shot learning situations, aiming to enable model adaptation to unseen classes, contrasting with the study’s utilization of predefined classes. 1.3 Work on rare instrument detection In the domain of rare audio source detection, where annotated data is scarce or nonexistent, previous research efforts have aimed to address this challenging task. Various strategies have been explored in the domain of few-shot learning, including hierarchical methodologies akin to those proposed by Garcia et al. [ 4 ] , along with approaches centered on continual learning [ 17 ] . Continual learning techniques focus on easily incorporating new instruments into a model as additional data becomes available. Moreover, attempts have been made to capitalize on weakly annotated data, where instrument presence is identified but precise activation times are not specified, yielding only incremental enhancements [ 18 ] . The utilization of pre-training strategies has emerged as a pivotal area of interest. In 2023, Zong et al. [ 19 ] employed isolated notes for pre-training before transitioning to training on polyphonic data, albeit with a specific emphasis on predominant instrument recognition exclusively. Another explored avenue involves synthetic data generation achieved through layering mono-instrumental excerpts with tempo and pitch shifting to produce realistic artificial multi-instrument tracks [ 20 ] . Furthermore, model reprogramming has been proposed, involving training a smaller model to map inputs to the input space of a larger pre-trained model. This technique, akin to transfer learning, harnesses the generalization capabilities of the larger model to mitigate data imbalance, consequently significantly reducing training time requirements [ 21 ] . 1.4 Contributions and outline of the paper In this study, we introduce several methodologies aimed at efficiently incorporating hierarchical instrument structures into our predictive models, and evaluate this novel class of models tailored for hierarchical music prediction. Importantly, our evaluations are conducted on the MedleyDB dataset. This dataset is renowned for its expansive and varied content, which allows us to overcome constraints related to particular music genres and a restricted instrument set. As far as we know, this is the first work on polyphonic instrument recognition using the MedleyDB dataset, providing crucial baseline performances in this domain. The paper is organized