Self-Supervised Pretraining for Fine-Grained Plankton Recognition
Joona Kareinen, Tuomas Eerola, Kaisa Kraft, Lasse Lensu, Sanna Suikkanen, Heikki Kälviäinen · 2025
Read Self-Supervised Pretraining for Fine-Grained Plankton Recognition by Joona Kareinen, Tuomas Eerola, Kaisa Kraft, Lasse Lensu, Sanna Suikkanen, Heikki Kälviäinen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-11341.txt.
Introduction
Plankton hold up ocean food webs and draw down carbon, so tracking their species matters, but naming them from images is hard and labeled examples are scarce. Joona Kareinen and five co-authors train a general plankton recognizer without labels. Using masked autoencoding on a large, varied collection of plankton images, they pretrain an image encoder that can then be fine-tuned for new datasets with very few labeled pictures.
In their experiments, the self-supervised start beat the practice of starting from a network pretrained on everyday photographs, especially when training data were limited. Accuracy rose further when unlabeled images from the target dataset joined the pretraining. The authors present the approach as a way to keep up with the flood of new plankton datasets while asking little of the experts who would label them.
Catalog record
| CREATOR | Joona Kareinen, Tuomas Eerola, Kaisa Kraft, Lasse Lensu, Sanna Suikkanen, Heikki Kälviäinen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-11341
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Self-Supervised Pretraining for Fine-Grained Plankton Recognition Joona Kareinen, Tuomas Eerola, Kaisa Kraft, Lasse Lensu, Sanna Suikkanen, Heikki Kälviäinen Abstract Plankton recognition is an important computer vision problem due to plankton’s essential role in ocean food webs and carbon capture, highlighting the need for species-level monitoring. However, this task is challenging due to its fine-grained nature and dataset shifts caused by different imaging instruments and varying species distributions. As new plankton image datasets are collected at an increasing pace, there is a need for general plankton recognition models that require minimal expert effort for data labeling. In this work, we study large-scale self-supervised pretraining for fine-grained plankton recognition. We first employ masked autoencoding and a large volume of diverse plankton image data to pretrain a general-purpose plankton image encoder. Then, we utilize fine-tuning to obtain accurate plankton recognition models for new datasets with a very limited number of labeled training images. Our experiments show that self-supervised pretraining with diverse plankton data clearly increases plankton recognition accuracy compared to standard ImageNet pretraining when the amount of training data is limited. Moreover, the accuracy can be further improved when unlabeled target data is available and utilized during the pretraining. 1 Introduction Despite the advancements in vision foundation models, fine-grained recognition in the presence of dataset shift remains a challenging task [ 63 ] . The subtle differences between classes, combined with distribution shifts in class appearance and composition between training and target datasets, necessitate image representations that are both general enough to handle dataset shifts and specific enough to distinguish visually similar classes. These challenges are difficult to tackle with a general-purpose vision foundation model and call for tailored solutions. Plankton recognition offers an interesting and environmentally relevant application for studying and developing such methods. Recognizing taxonomically close plankton species is challenging and is further complicated by domain and class distribution shifts across the existing datasets. Figure 1 : We evaluate three different self-supervised pretraining strategies: (a) pretraining on ImageNet-1k and fine-tuning on plankton data, (b) pretraining on a diverse plankton dataset and fine-tuning on unseen plankton data, and (c) pretraining on a diverse plankton dataset and fine-tuning on a subset of the same dataset. Plankton are a collection of microscopic organisms that serve as the foundation of aquatic ecosystems. These organisms are broadly classified into two primary groups: phytoplankton and zooplankton. Phytoplankton are critical as primary producers at the base of the food chain, contributing approximately 50% of the world’s oxygen production and around 40% of global carbon fixation [ 17 , 18 ] . Similarly, zooplankton are critical in matter and nutrient cycling from the primary producers to higher trophic levels, such as commercially significant fish species [ 5 ] . Furthermore, zooplankton transport carbon from surface to deep waters through feeding, daily migrations, and fecal pellets, playing a key role in global carbon cycles [ 2 , 67 ] . In addition to being vital to the marine food web, plankton also serve as an indicator of ocean health. Plankton are sensitive to environmental changes: they respond rapidly to changes in temperature, nutrient availability, and water flows [ 22 ] . This makes them valuable for understanding the aquatic ecosystem dynamics, which in turn contributes to the prediction of environmental changes such as pollution and climate change [ 22 ] . Understanding plankton species distributions provides valuable information on the oceans’ health, but monitoring plankton communities and composition requires sustained observations and is challenging due to their diversity and dynamic changes. To address this, various plankton imaging and analysis instruments have been developed for both in-situ and laboratory use [ 40 ] . Recent technological advancements have led to the development of both automated and semi-automated imaging systems that can be used to capture massive plankton datasets. However, the use of plankton imagery faces various challenges, such as the fine-grained nature of the identification task, the rarity of species, and the uncertainty of category labeling [ 15 ] . There is a growing need for automated plankton species identification systems, as they significantly reduce the need for manual identification and improve the efficiency of processing large amounts of data. As plankton imaging instruments become more accessible, an increasing number of plankton image datasets are being collected. These datasets vary in both the imaging instruments used and the plankton species they contain. This causes dataset shifts between them, and therefore, a plankton recognition model trained on one dataset often fails when applied to another [ 4 ] . Manually labeling sufficient training data for each dataset to create dataset-specific models is impractical. Therefore, a more general model that minimizes the need for expert labeling would be highly beneficial. However, training a single general model using multiple datasets in a supervised manner is challenging due to the lack of a universally agreed-upon method for categorizing plankton, although connecting the classes to the World Register of Marine Species is routinely done by many users. The set of class labels in a plankton dataset depends on various factors, such as the environmental application for which the dataset was collected, the range of particle sizes the imaging instrument can capture, and the geographic region from which the water samples were collected. Moreover, the full taxonomical hierarchy of plankton is undergoing continuous revision as recent studies have deepened the understanding of their biodiversity [ 61 ] . Therefore, the class labels between datasets are not always entirely comparable, making it difficult to use them simultaneously for supervised training. This issue is especially notable in classes that do not correspond to a specific species, as they are particularly affected by taxonomic revisions. In this paper, we address the issue of incomparable class labels between datasets through self-supervised pretraining. First, we collect a large pool of public plankton image datasets and use unsupervised learning to obtain general-purpose plankton representations without the need to harmonize class labels across datasets. For this, we employ Masked Autoencoders (MAE) [ 24 ] , which guide the encoder to learn plankton image representations by masking a large portion of input images and then learning to generate the missing parts. This masking technique allows the model to learn features that effectively describe the subtle visual differences between fine-grained plankton classes by forcing it to focus on small regions in the images. Next, we fine-tune the encoder with an incorporated classification head on plankton datasets with a limited amount of labeled training images. In the experimental part of the work, we compare different pretraining methods on plankton data, as shown in Figure 1 . We evaluate our models pretrained on plankton image data against standard ImageNet pretraining in varying labeled data scenarios. Our results show that domain-specific pretraining can obtain results comparable to ImageNet pretraining with significantly fewer images. Additionally, we show that domain-specific pretraining can surpass ImageNet pretraining in accuracy, particularly when labeled data is limited. The main contributions of this study are as follows: (1) The first application of Masked Autoencoders for self-supervised pretraining on plankton image data, (2) Extensive evaluation of different pretraining strategies under varying labeled data conditions, and (3) Analysis of the benefits of domain-specific pretraining on fine-grained plankton recognition. 2 Related works 2.1 Plankton recognition A large number of