Generative AI-based data augmentation for improved bioacoustic classification in noisy environments

Anthony Gibbons · 2024

Read Generative AI-based data augmentation for improved bioacoustic classification in noisy environments by Anthony Gibbons (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01530.txt.

Introduction

Anthony Gibbons and co-authors use fake bird song spectrograms to teach sound classifiers in noisy fields. Wind farms are harsh spots for bird calls, with turbine hum behind every note. Labeled calls are scarce, more so for rare birds. Their 2024 study grows extra training data with two generative models. It also shares a new set of 640 hours of calls from Irish wind farm sites.

Experts labeled about 800 clips from that trove. Of the two fake-data engines, the diffusion model made the most lifelike spectrograms. Every classifier gained when fake clips joined real ones, and gains grew with dose. A mixed ensemble stood up well next to the trusted BirdNET tool. The plan could stretch to more species and land types.

Catalog record

CREATORAnthony Gibbons
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-01530

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Generative AI-based data augmentation for improved bioacoustic classification in noisy environments Anthony Gibbons Abstract Abstract 1. Obtaining data to train robust artificial intelligence (AI)-based models for species classification can be challenging, particularly for rare species. Data augmentation can boost classification accuracy by increasing the diversity of training data and is cheaper to obtain than expert-labelled data. However, many classic image-based augmentation techniques are not suitable for audio spectrograms. 2. We investigate two generative AI models as data augmentation tools to synthesise spectrograms and supplement audio data: Auxiliary Classifier Generative Adversarial Networks (ACGAN) and Denoising Diffusion Probabilistic Models (DDPMs). The latter performed particularly well in terms of both realism of generated spectrograms and accuracy in a resulting classification task. 3. Alongside these new approaches, we present a new audio data set of 640 hours of bird calls from wind farm sites in Ireland, approximately 800 samples of which have been labelled by experts. Wind farm data are particularly challenging for classification models given the background wind and turbine noise. 4. Training an ensemble of classification models on real and synthetic data combined gave 92.6% accuracy (and 90.5% with just the real data) when compared with highly confident BirdNET predictions. 5. Our approach can be used to augment acoustic signals for more species and other land-use types, and has the potential to bring about a step-change in our capacity to develop reliable AI-based detection of rare species. Our code is available at https://github.com/gibbona1/SpectrogramGenAI . Keywords: Bioacoustics, Data Augmentation, Deep Learning, Generative Models, Generative Adversarial Networks, Passive Acoustic Monitoring, Species Recognition, Stable Diffusion 1 Introduction Species monitoring is a major component of assessing biodiversity and ecosystem health, serving as a barometer of environmental change and conservation needs. The dynamics and diversity of bird communities are key indicators of ecosystem health and environmental change [ 37 ] . Birds provide valuable ecosystem services, which include limiting pest populations [ 16 ] , seed dispersion [ 46 ] and pollination [ 76 ] . The fields of bioacoustics and ecoacoustics have long studied the behaviour and diversity of bird species activity via sound. Acoustic monitoring has become increasingly popular due to increased efficiency, reduced cost, and time saving, compared to traditional methods [ 10 ] . However, many concerns remain surrounding the accuracy and ability of software to identify bird species from acoustic recordings [ 18 ] . Computer vision developments have carried over to the field of bioacoustics to detect species via sound. Audio is represented as spectrograms in order to use image classification methods [ 72 ] . Labelled data is required to train such classifiers, necessitating audio clips annotated to the degree of detail desired with, for example, content such as species [ 73 ] , call type [ 4 ] , individual animals [ 41 ] , or other audio events present. However, manual labelling of these datasets is both costly and time-consuming to obtain, requiring expert knowledge and skills. Bioacoustics is still a nascent field and lacks standardised labelled datasets [ 72 ] . To expand on the limited data available, data augmentation is often applied. However, not all data augmentation methods designed for images apply readily to spectrograms due to their inherent time-frequency structure [ 91 ] . Here, we investigate both Auxiliary Classifier Generative Adversarial Networks (ACGAN) [ 55 ] and Stable Diffusion models [ 61 ] as data augmentation methods for enhancing bird call spectrogram classification models, with the aim of reducing the amount of domain-specific labelled data required to train species classifiers. Our key contributions include: • Application of Stable Diffusion to bird species monitoring. The model generates high-quality and diverse spectrograms of bird sound. As far as we are aware, this is the first application of stable diffusion to bioacoustic classification. • Improved classification. Each classifier we used found some improvement by including synthetic data. Classification metrics generally improved in line with the amount of synthetic data added. • Ecoacoustic pipelines. We suggest generative models to enhance bioacoustic classification pipelines. Our Stable Diffusion model produces high-quality diverse spectrograms, improving bird song classification accuracy and reducing the need for expensive expert-labelled data. First, we give background on acoustic species classification and data augmentation and place our techniques in the context of existing data augmentation methods. After describing the details of our acoustic dataset, we then detail the audio pre-processing and the generative models we employed. We use the terms stable diffusion, diffusion models, Denoising Diffusion Probabilistic Models (DDPMs), and latent diffusion interchangeably once the stable diffusion model is introduced. We then provide a detailed description of our experimental design and results, and follow this up with in-depth discussion of our results, limitations and paths for future work. 2 Existing methods Passive Acoustic Monitoring (PAM) is a non-invasive technique where sensors are deployed in the field to record audio, usually on a pre-defined schedule, for weeks, months or years at a time without intervention. The ever-decreasing cost of recording equipment and data storage has allowed much larger PAM datasets to be collected [ 35 , 48 , 62 ] . The advancement of deep learning methods has continued to grow, allowing automatic species detection methods to catch up with the growing volume of data emerging via PAM. The scale and sophistication of these classifiers has advanced rapidly from presence/absence [ 44 ] to classifying single digit [ 60 ] to tens and hundreds [ 1 , 90 ] of species. BirdNET [ 32 ] and Perch [ 17 ] , both considered among the current state of the art in terms of bird classification capabilities, were trained on over 6k and 10k bird species globally, respectively. The typical bird species classifier’s input format will be as spectrograms, a visual representation of sound. The spectrograms are coloured by sound energy levels across time and frequency. These may be standard (linear-frequency), mel spectrograms (to better align with human sound perception), or log-frequency spectrograms. Other audio representations used as input for classification include Scalogram [ 66 ] , MFCC [ 78 ] , Gammatone Filterbank [ 57 ] , Constant-Q Transform [ 81 ] and the raw waveform itself [ 39 ] . While generalised solutions for bird species classification such as BirdNET and Perch are widely applicable, fine-tuned approaches are often desired for specific tasks [ 38 ] . The benefits include optimising hyperparameters, improving performance and robustness or reducing the number of parameters, which, in turn, reduces the size and computational cost of deploying such models [ 40 , 30 ] . Training custom classifiers is commonly done through techniques such as transfer learning [ 88 ] and knowledge distillation [ 21 ] . These training processes necessarily require some amount of labelled training data from the specified task of interest. This can, in the case of acoustic data containing bird species, be time- and labour-intensive to obtain. Semi-supervised learning is a strategy to remedy class imbalance or issues with low volumes of high quality data, where labelled datasets are supplemented with unlabelled data [ 22 ] . It is common, for example, to train a classifier on a small about of labelled (human-verified) data, run through a large amount of unlabelled data, and take the highly confident

Read the complete text