A multi-modal dataset for insect biodiversity with imagery and DNA at the trap and individual level

Johanna Orsholm, John Quinto, Hannu Autto, Gaia Banelyte, Nicolas Chazot, Jeremy deWaard, Stephanie deWaard, Arielle Farrell, Brendan Furneaux, Bess Hardwick, Nao Ito, Amlan Kar, Oula Kalttopää, Deirdre Kerdraon, Erik Kristensen, Jaclyn McKeown, Tommi Mononen, Ellen Nein, Hanna Rogers, Tomas Roslin, · 2025

Read A multi-modal dataset for insect biodiversity with imagery and DNA at the trap and individual level by Johanna Orsholm, John Quinto, Hannu Autto, Gaia Banelyte, Nicolas Chazot, Jeremy deWaard, Stephanie deWaard, Arielle Farrell, Brendan Furneaux, Bess Hardwick, Nao Ito, Amlan Kar, Oula Kalttopää, Deirdre Kerdraon, Erik Kristensen, Jaclyn McKeown, Tommi Mononen, Ellen Nein, Hanna Rogers, Tomas Roslin, (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06972.txt.

Introduction

Johanna Orsholm and co-authors present a data set paper from 2025 on insect diversity. Insects span millions of species, and many are in steep decline. Fast tools are needed to census them at scale. DNA barcodes and sharp photos can each name specimens, but most photo tools train on single insects, not the mixed bulk that traps collect.

Their data set pairs photos and DNA at both levels: the whole trap sample and each single specimen. Human labelers, aided by a smart tool, drew outlines around each creature in bulk photos. They named more than seventeen thousand specimens in all. The set tests the hard task of finding tiny objects in a crowd. The team hopes it speeds both ecology field work and machine learning study.

Catalog record

CREATORJohanna Orsholm, John Quinto, Hannu Autto, Gaia Banelyte, Nicolas Chazot, Jeremy deWaard, Stephanie deWaard, Arielle Farrell, Brendan Furneaux, Bess Hardwick, Nao Ito, Amlan Kar, Oula Kalttopää, Deirdre Kerdraon, Erik Kristensen, Jaclyn McKeown, Tommi Mononen, Ellen Nein, Hanna Rogers, Tomas Roslin,
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-06972

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A multi-modal dataset for insect biodiversity with imagery and DNA at the trap and individual level Johanna Orsholm, John Quinto, Hannu Autto, Gaia Banelyte, Nicolas Chazot, Jeremy deWaard, Stephanie deWaard, Arielle Farrell, Brendan Furneaux, Bess Hardwick, Nao Ito, Amlan Kar, Oula Kalttopää, Deirdre Kerdraon, Erik Kristensen, Jaclyn McKeown, Tommi Mononen, Ellen Nein, Hanna Rogers, Tomas Roslin, Paula Schmitz, Jayme Sones, Maija Sujala, Amy Thompson, Evgeny V. Zakharov, Iuliia Zarubiieva, Akshita Gupta, Scott C. Lowe, Graham W. Taylor Abstract Insects comprise millions of species, many experiencing severe population declines under environmental and habitat changes. High-throughput approaches are crucial for accelerating our understanding of insect diversity, with DNA barcoding and high-resolution imaging showing strong potential for automatic taxonomic classification. However, most image-based approaches rely on individual specimen data, unlike the unsorted bulk samples collected in large-scale ecological surveys. We present the Mixed Arthropod Sample Segmentation and Identification (MassID45) dataset for training automatic classifiers of bulk insect samples. It uniquely combines molecular and imaging data at both the unsorted sample level and the full set of individual specimens. Human annotators, supported by an AI-assisted tool, performed two tasks on bulk images: creating segmentation masks around each individual arthropod and assigning taxonomic labels to over 17 000 17\,000 specimens. Combining the taxonomic resolution of DNA barcodes with precise abundance estimates of bulk images holds great potential for rapid, large-scale characterization of insect communities. This dataset pushes the boundaries of tiny object detection and instance segmentation, fostering innovation in both ecological and machine learning research. 1 Background Insects are the most diverse organism group on Earth, with more than one million described species [ 1 ] and an estimated four million species yet to be discovered [ 2 ] . Climate change and anthropogenic activities are causing rapid declines in insect populations [ 3 ] , and many species likely face extinction in the coming decades [ 4 ] . Yet our understanding of insect diversity is seriously data-limited and incomplete. In particular, data generation is hampered by a severe lack of available taxonomic expertise [ 5 , 6 ] . Consequently, there is an urgent need for high-throughput methods to study and monitor insect communities, with machine learning-based image classification and molecular methods quickly advancing as promising tools for the taxonomic characterization of samples. To taxonomically classify diverse insect samples, substantial amounts of training data are needed. This requires taxonomically labelled examples — images for visual classification and DNA sequences for molecular approaches. At present, the most abundant insect DNA sequence data consists of short fragments (typically 658 bp) from a standardized gene region, the cytochrome c oxidase subunit 1 (COI), commonly known as DNA barcodes [ 7 ] . Comprehensive datasets containing millions of images and DNA barcodes from individual insect specimens are now available for training classifiers [ 8 , 9 , 10 ] . However, many large-scale ecological studies or monitoring efforts collect insects in bulk samples, which contain multiple specimens with mixed taxonomic composition. Sorting bulk samples into individual specimens requires substantial effort. Thus, for many ecological applications, there is currently a discrepancy between the available specimen-level training data and the bulk-level samples in need of classification. To bridge this gap, taxonomically annotated bulk-level training images are needed. Capturing images of unsorted bulk samples is a straightforward procedure, yielding a single image depicting all specimens from the sample, hereafter referred to as a bulk image . Classifying insects from such bulk images presents two key challenges. First, given insects’ small size and the high specimen density in bulk samples, individual specimens appear as tiny objects within the image, with restricted morphological detail. Second, insects’ high taxonomic diversity requires large training datasets to achieve adequate coverage of different taxa by the classifier. Together, these pose a significant challenge for the taxonomic annotation of images. As a consequence, previous studies attempting insect classification from bulk images have worked with a substantially reduced number of classes [ 11 , 12 ] , resulting in more coarse information than what is often needed for community ecology. DNA barcode data can also be generated from bulk samples, a process called DNA metabarcoding [ 13 ] . The resulting DNA barcodes can then be classified into taxa by comparison with extensive reference databases, such as the Barcode of Life Data Systems (BOLD) [ 14 ] . Assuming that the target group is well-represented in the reference data, DNA metabarcoding produces detailed information about the taxonomic composition of the sample. It also yields data on the relative abundances of different sequence variants. However, one of the main drawbacks of DNA metabarcoding is that it is challenging to infer absolute abundances across samples and taxa [ 15 ] . To address this challenge, we propose to integrate DNA and image data, leveraging the strength of both modalities. Combining detailed taxonomic classifications from DNA metabarcoding with bulk images of the same samples, where counting of taxa is relatively straightforward, could enable absolute, taxon-specific abundance estimates. Using both images and barcodes has been shown to enhance classifier performance compared to using either data source in isolation [ 16 , 17 ] . Further, when using individual-level data of both modalities, the genus of unknown species was predicted with more than 80% success [ 16 ] . This could allow researchers to explore the taxonomic composition at higher ranks even in samples which contain a large number of unknown species, thus enabling studies of particularly diverse ecosystems and organism groups. An additional benefit of combining molecular and image data is the potential to extract trait data, such as the body size distribution across individuals in a sample [ 11 ] . Here, we present the Mixed Arthropod Sample Segmentation and Identification (MassID45) dataset, centred on 45 bulk arthropod samples (primarily insects) collected using Malaise traps deployed in Sweden and Finland in 2021. Harnessing both imaging and molecular information, each sample features DNA metabarcoding data and one or more unsorted bulk images. We further provide sample-level biomass measurements, which may support trait-based analyses. To facilitate the training of machine learning-based classifiers, we additionally supply individual-level images and DNA barcode sequences obtained after sorting each sample into individual specimens ( 36\,402 in total). Leveraging AI-assisted annotation and sample-specific DNA-based classifications, we provide a detailed segmentation mask and taxonomic annotation for each arthropod in the bulk images. Using the bulk-level data, we benchmark instance segmentation on tiny objects, drawing on related methods for object detection. Our dataset and experiments provide a valuable perspective on the domain-specific challenges of capturing tiny, densely arranged, and sometimes overlapping objects — an area less emphasized in standard object detection datasets. Recent advances in tiny object detection [ 18 , 19 ] , open-world detection [ 20 ] , multi-modal learning [ 17 , 16 ] , and AI-assisted annotation [ 21 ] collectively allow researchers to better detect, classify, and analyze numerous minuscule specimens in a single frame. This dataset is poised to support a wide range of ecological applications, such as training automated classifiers for bulk samples, accurately counting specimens in large collections, and enabling large-scale morphological analyses. From a machine learning standpoint, it also opens up avenues in developing instance segmentation methods for tiny objects, exploiting

Read the complete text