Multispecies Animal Re-ID Using a Large Community-Curated Dataset

Lasha Otarashvili, Tamilselvan Subramanian, Jason Holmberg, J.J. Levenson, Charles V. Stewart · 2024

Read Multispecies Animal Re-ID Using a Large Community-Curated Dataset by Lasha Otarashvili, Tamilselvan Subramanian, Jason Holmberg, J.J. Levenson, Charles V. Stewart (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-05602.txt.

Introduction

Lasha Otarashvili, Tamilselvan Subramanian, Jason Holmberg, J. J. Levenson, and Charles Stewart build one system to recognize individual animals across many species in a 2024 study. Ecologists identify animals from photos to track populations, but training a separate model per species is costly and starves rare species of data. Shared looks across species also go unused.

The authors assemble a large community-curated data set spanning many species. A single re-identification model trains on all of it at once. Pooling data should help species with few photos while cutting repeated engineering work. The approach lowers the barrier for conservation groups that need individual tracking.

Catalog record

CREATORLasha Otarashvili, Tamilselvan Subramanian, Jason Holmberg, J.J. Levenson, Charles V. Stewart
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-05602

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multispecies Animal Re-ID Using a Large Community-Curated Dataset Lasha Otarashvili, Tamilselvan Subramanian, Jason Holmberg, J.J. Levenson, Charles V. Stewart Abstract Recent work has established the ecological importance of developing algorithms for identifying animals individually from images. Typically, a separate algorithm is trained for each species, a natural step but one that creates significant barriers to wide-spread use: (1) each effort is expensive, requiring data collection, data curation, and model training, deployment, and maintenance, (2) there is little training data for many species, and (3) commonalities in appearance across species are not exploited. We propose an alternative approach focused on training multi-species individual identification (re-id) models. We construct a dataset that includes 49 species, 37K individual animals, and 225K images, using this data to train a single embedding network for all species. Our model employs an EfficientNetV2 backbone and a sub-center ArcFace loss function with dynamic margins. We evaluate the performance of this multi-species model in several ways. Most notably, we demonstrate that it consistently outperforms models trained separately on each species, achieving an average gain of 12.5% in top-1 accuracy. Furthermore, the model demonstrates strong zero-shot performance and fine-tuning capabilities for new species with limited training data, enabling effective curation of new species through both incremental addition of data to the training set and fine-tuning without the original data. Additionally, our model surpasses the recent MegaDescriptor on unseen species, averaging an 19.2% top-1 improvement per species and showing gains across all 33 species tested. The fully-featured code repository is publicly available on GitHub, and the feature extractor model can be accessed on HuggingFace for seamless integration with wildlife re-identification pipelines. The model is already in production use for 60+ species in a large-scale wildlife monitoring system. [ 2 ] , 1 Introduction Individual animal identification has significant challenges that are not typically encountered in related problems such as human face recognition [ 7 , 32 ] and human re-id [ 38 ] : • Ground truth training datasets tend to be small. This is because individual animal identity is difficult and time-consuming to curate: it generally relies on humans searching for matching images, an effort that grows quadratically with the size of the dataset. • Some species have large volumes of training data while others have relatively little. This is partially caused by difficult in situ imaging conditions that make it hard to obtain good quality photos. It is also a result of the aforementioned challenges of manual curation, where the resources necessary to build large training sets are available for only the most charismatic species. • Standard practice is to develop and train animal re-id algorithms on a species-by-species basis, sometimes with per-species algorithmic features. This practice is reasonable in research settings and when just a few species are considered, but it is increasing impractical when supporting dozens of species, each with frequent, ongoing additions of new images and individuals. Our goal in this paper is to address these issues through training and testing a single model for identifying individual animals from many different species. We go beyond the ground breaking work of [ 39 ] and explore a dataset that is expert-curated from ongoing use of animal re-id in scientific and conservation practice. Our study focuses on two broad issues: • How does the overall performance of models trained on multiple species compare to those trained on individual species? When performance is equivalent then individual species models are unnecessary, and when performance is better on the multiple species model we also see a performance advantage to combined training on multiple species. • How well do multispecies models perform when applied to species unseen during training? This is crucial to addressing the shortage of annotated training data for many species. Ideally, a model would work as well as if training data were available, but practically if a multi-species model can significantly reduce the amount of training data needed obtain a given level of performance for a new species it is an important development. As a result of our study, we make the following main contributions: 1. We train a multispecies animal re-id model MiewID on a community-curated dataset involving 49 species from 59 datasets, 37,138 individual animals and 225,374 total annotations. We publicly release the code as a GitHub repository and model weights on Huggingface . We show that our model consistently outperforms analogous single-species models, exhibiting an overall performance gain of 12.5% top-1 accuracy and per-species gains ranging from a minimum of -0.2% (negative for 1 species) to a maximum of 77.3%. 2. We demonstrate the effectiveness of our model on previously species unseen during training, and show that it substantially outperforms MegaDescriptor [ 39 ] on all 33 tested species, with an average top-1 improvement of 19.2%. 3. We show the potential of our model for learning to effectively identify species that have only have a small set of data available during training. This is seen both when the new data are incorporated into the overall training set and when fine-tuning the published model. 2 Background Historically, a wide range of methods have been used for animal re-identification, ranging from manual feature extraction to more recent advances in deep learning. In recent years, global descriptor-based methods leveraging deep learning [ 8 , 21 , 39 , 3 , 11 , 22 ] have gained prominence over traditional manual feature-based approaches [ 33 , 24 , 30 ] and local feature algorithms [ 5 , 24 , 18 , 28 ] . These deep learning models are becoming increasingly popular due to their ability to produce more robust, generalizable descriptors compared to hand-crafted or localized feature techniques. A major advantage of deep learning methods is their scalability and capability to streamline the re-identification pipeline. While they require larger datasets and greater computational resources to train, they minimize the need for extensive preprocessing and can run end-to-end, making them highly efficient for large-scale animal ID tasks. In this work, we focus on end-to-end deep learning models due to their ability to handle complex identification tasks across multiple species with minimal manual intervention, thus improving scalability and applicability to real-world scenarios. The release of several large animal re-identification datasets has accelerated research in this field [ 4 , 12 , 9 , 23 , 31 , 37 , 25 , 1 , 8 , 17 , 36 , 35 ] . These datasets cover a broad spectrum of species, providing the necessary data to train more effective and generalizable models. Recent studies have explored multi-species re-identification, which focuses on applying individual identification methods across diverse sets of species [ 4 , 39 , 11 ] . Our work reported here goes beyond these methods in several important ways, including the quality and diversity of species used in training and the extensive experiments in zero-shot and few-shot fine-tuning. 3 Data We construct our multi-species dataset from a mixture of 1) publicly available datasets [ 4 , 12 , 9 , 23 , 31 , 37 , 25 , 1 , 8 ] and 2) ecological data and images contributed directly and privately to the Wild Me Lab’s self-hosted Wildbook platforms [ 2 ] , which support dynamic and secure data curation

Read the complete text