Transfer Learning and Mixup for Fine-Grained Few-Shot Fungi Classification
Jason Kahei Tam, Murilo Gustineli, Anthony Miyaguchi · 2025
Read Transfer Learning and Mixup for Fine-Grained Few-Shot Fungi Classification by Jason Kahei Tam, Murilo Gustineli, Anthony Miyaguchi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08248.txt.
Introduction
Jason Kahei Tam, Murilo Gustineli, and Anthony Miyaguchi report their entry to a fungi naming contest that gives very few sample photos per species. Fungi are hard for vision models: species look nearly alike, while one species varies a lot. The contest dataset was built for that few-shot test.
The team tried several transformer vision models, image augmentation, and balanced sampling. They also fed in text data about the species. Language models asked to name fungi cold did far worse than vision models. Pretraining on fungi images and careful sampling drove their best score. Their entry placed mid-pack in the final private ranking. The report is candid that headroom remains. It maps what helped for other teams facing rare-species data.
Catalog record
| CREATOR | Jason Kahei Tam, Murilo Gustineli, Anthony Miyaguchi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-08248
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Transfer Learning and Mixup for Fine-Grained Few-Shot Fungi Classification Jason Kahei Tam, Murilo Gustineli, Anthony Miyaguchi Abstract Accurate identification of fungi species presents a unique challenge in computer vision due to fine-grained inter-species variation and high intra-species variation. This paper presents our approach for the FungiCLEF 2025 competition, which focuses on few-shot fine-grained visual categorization (FGVC) using the FungiTastic Few-Shot dataset. Our team (DS@GT) experimented with multiple vision transformer models, data augmentation, weighted sampling, and incorporating textual information. We also explored generative AI models for zero-shot classification using structured prompting but found them to significantly underperform relative to vision-based models. Our final model outperformed both competition baselines and highlighted the effectiveness of domain-specific pretraining and balanced sampling strategies. Our approach ranked 35/74 on the private test set in post-completion evaluation, this suggests additional work can be done on metadata selection and domain-adapted multi-modal learning. Our code is available at https://github.com/dsgt-arc/fungiclef-2025 . [orcid=0009-0005-0853-0414, [email protected], ] *1 [orcid=0009-0003-9818-496X, [email protected], ] *1 [orcid=0000-0002-9165-8718, [email protected], ] *1 keywords LifeCLEF ,FungiCLEF ,Fine-Grained Visual Categorization (FGVC) ,Vision Transformers ,fungi ,species identification ,machine learning ,computer vision ,CEUR-WS † † copyrightyear: 2025 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: CLEF 2025: Working Notes, 9 – 12 September 2025, Madrid, Spain † † address: Georgia Institute of Technology, North Ave NW, Atlanta, GA 30332 † † corresp: Corresponding author. 1 Introduction Accurate identification of fungi species is challenging and often requires microscopic examination Lücking et al. 2020 , which places this task beyond simple image classification and falls into fine-grained visual categorization (FGVC). This paper documents and evaluates our approaches as part of the FungiCLEF 2025 Klarka and picekl 2025 @ CVPR-FGVC & LifeCLEF 2025 Joly et al. 2025 competition. The goal of this competition is fungi species recognition and the evaluation metric is top-5 accuracy. Multiple aspects make this task challenging. First, there is intra-species variation due to factors such as genotype, local conditions, time of year, and age (Figure 1 ) Picek et al. 2024 . Second, there is subtle inter-class variation even at higher order taxonomic ranks, which makes it difficult for pattern learning algorithms to differentiate (Figure 2 ) Picek et al. 2024 . Figure 1: Example of intra-species variation - Species: Comatricha alta Figure 2: Example of inter-species similarity and variation from three distinct families. Species left to right: Psathyrella citerinii, Inocybe assimilata, and Entoloma favrei Figure 3: Distribution of classes in training and validation datasets 1.1 Dataset Overview The dataset provided for the competition is the few-shot subset of the FungiTastic dataset, a collection of fungal records continuously collected over a twenty-year span Picek et al. 2025 . Each observation in the dataset contains associated images, metadata, and vision language model (VLM), Molmo Deitke et al. 2024 , generated caption. The metadata contains information such as date, location, substrate, and full taxonomic ranks. It is important to note that the task is to classify images based on category_id, which has a slightly different count than species. The training dataset contains 7,819 images with 2,413 unique species and 2,427 unique category_id. The validation dataset contains 2,285 images with 569 species and 570 unique category_id. The test dataset contains 1,911 images with no taxonomic ranks. The provided image dataset contains the training, validation, and testing sub-datasets. Each sub-dataset contains images in different maximum pixel sizes, ranging from 300p to full-size images. The datasets do not have the same category_id distribution (Figure 3 ). In the chart, the category_id is mapped to class ID and then sorted by frequency, with category_id 2383 appearing most frequently. Both datasets exhibit class imbalance, with the most common class having approximately 30 images and multiple classes having only 1 image. 2 Related Work Previous work by the DS@GT group for FungiCLEF 2024 demonstrated the strong performance of using DINOv2 vision transformers in image classification Chiu et al. 2024 . Last year’s winner, Team IES, combined image embeddings from Swin Transformer V2 Liu et al. 2022 with metadata features from multi-layer-perception for species classification Wolf et al. 2024 . 3 Methodology Our benchmark approach uses PlantCLEF 2024 Goëau et al. 2024 embeddings, weighted sampling Hughes 2024 , and Mixup Zhang et al. 2018 . Using off-the-shelf generative AI models, multi-modal approach of combining text embeddings with image embeddings, and multi-objective loss were also explored. The competition evaluation metric is top-k accuracy, with k = 5. \text{Top-k Accuracy}=\frac{1}{N}\sum_{i=1}^{N}1\left[y_{i}\in\hat{Y}_{i}^{(k)}\right] (1) The cloud computing resources were funded by the Data Science at Georgia Tech (DS@GT). Data and computing was hosted by the Partnership for an Advanced Computing Environment (PACE) at Georgia Tech PACE 2017 . 3.1 Benchmark Methodology Our benchmark methodology can be summarized in a few steps: 1. Image embeddings from PlantCLEF 2024 model 2. Weighted sampling to balance the training dataset 3. Mixup on batches during training 4. Linear classifier 5. Mixup loss with cross-entropy PyTorch 2025 Figure 4: Pipeline of the benchmark methodology 3.1.1 Dataset Preparation We pre-computed both the image and text embeddings and stored them in parquet files for a modular experimentation workflow. We encountered ”Premature End of JPEG file” error when reading images because some images did not end with the default hex code. This may result in unintended side effects during training. This error was solved by loading the image with OpenCV and then saving it again Poulinakis 2021 . There was one corrupted image in the validation 720p set; we did not use this image since we only used the full-size images for our pipeline. All approaches except Generative AI involved using the pytorch lightning library in training the classifier. The hyperparameters used are as follows: batch size of 256, 50 maximum epochs with early stopping of 3, Adam Kingma and Ba 2017 optimizer, learning rate of 5\cdot 10^{-4} and no learning rate scheduler. 3.1.2 Image Embeddings We experimented with multiple transformer models: Facebook DINOv2 Oquab et al. 2024 , PlantCLEF 2024 pre-trained model Goëau et al. 2024 , FungiTastic BEiT Picek et al. 2025 , and FungiTastic ViT Picek et al. 2025 . A summary of the models used is shown in Table 1 and Table 2 . DINOv2 was selected for its state-of-the-art performance in computer vision tasks and its strong results in FungiCLEF 2024 Chiu et al. 2024 . The PlantCLEF model was selected due to its foundation in DINOv2 and its pre-training on 1.4 million plant images from the Pl@ntNet database, offering the potential benefits of transfer learning. The two FungiTastic models were selected because they were pre-trained on the fungi images. The image embeddings from the PlantCLEF 2024 model used in our benchmark methodology have a size of 768. Table 1: Summary of models explored for image embeddings generation Shorthand Model Name DINOv2 dinov2-base Oquab et al. 2024 PlantCLEF 2024 vit_base_patch14_reg4_dinov2_lvd142m_pc24_onlyclassifier_then_all Goëau et al. 2024 FungiTastic BEiT beit_base_patch16_224.in1k_ft_fungitastic_224 Picek et al. 2025 FungiTastic ViT vit_base_patch16_224.in1k_ft_fungitastic_224 Picek et al. 2025 Table 2: Details of models explored for image embeddings generation Shorthand Architecture Hidden Size Pretraining/Finetuning DINOv2 ViT-B/14 Oquab et al. 2024 768 LVD-142M/None PlantCLEF 2024 ViT-B/14 Oquab et al. 2024 768 DINOv2 on LVD-142M/Pl@ntNet FungiTastic BEiT BEiT-B/16 Bao et al. 2022 768 ImageNet-1k/FungiTastic FungiTastic