iNatAg: Multi-Class Classification Models Enabled by a Large-Scale Benchmark Dataset with 4.7M Images of 2,959 Crop and Weed Species

Naitik Jain, Amogh Joshi, Mason Earles · 2025

Read iNatAg: Multi-Class Classification Models Enabled by a Large-Scale Benchmark Dataset with 4.7M Images of 2,959 Crop and Weed Species by Naitik Jain, Amogh Joshi, Mason Earles (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-20068.txt.

Introduction

Naitik Jain, Amogh Joshi, and Mason Earles built a 2025 photo set to teach computers to name crop and weed plants. Naming plants right helps farmers spray less and farm with care. Look alike leaves, shifting light, and a lack of large farm photo sets have held the task back.

Their set holds over four million photos of close to three thousand species, drawn from a vast nature photo bank. Photos come from every continent and show real field clutter. Models trained on the set beat past best scores on crop versus weed and species tasks. The authors also study the errors the models still make.

Catalog record

CREATORNaitik Jain, Amogh Joshi, Mason Earles
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-20068

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

iNatAg: Multi-Class Classification Models Enabled by a Large-Scale Benchmark Dataset with 4.7M Images of 2,959 Crop and Weed Species Naitik Jain, Amogh Joshi, Mason Earles Abstract Accurate identification of crop and weed species is critical for precision agriculture and sustainable farming. However, it remains a challenging task due to a variety of factors -- a high degree of visual similarity among species, environmental variability, and a continued lack of large, agriculture-specific image data. We introduce iNatAg, a large-scale image dataset which contains over 4.7 million images of 2,959 distinct crop and weed species, with precise annotations along the taxonomic hierarchy from binary crop/weed labels to specific species labels. Curated from the broader iNaturalist database, iNatAg contains data from every continent and accurately reflects the variability of natural image captures and environments. Enabled by this data, we train benchmark models built upon the Swin Transformer architecture and evaluate the impact of various modifications such as the incorporation of geospatial data and LoRA finetuning. Our best models achieve state-of-the-art performance across all taxonomic classification tasks, achieving 92.38% on crop and weed classification. Furthermore, the scale of our dataset enables us to explore incorrect misclassifications and unlock new analytic possiblities for plant species. By combining large-scale species coverage, multi-task labels, and geographic diversity, iNatAg provides a new foundation for building robust, geolocation-aware agricultural classification systems. We release the iNatAg dataset publicly through AgML 1 1 1 https://github.com/Project-AgML/AgML , enabling direct access and integration into agricultural machine learning workflows. 1 Introduction Figure 1 : Examples of 40 crop species from the iNatAg dataset. This visual snapshot highlights variation in leaf shape, color, and background due to real-world, user-generated data. Fine-grained plant species classification is critical for applications in agriculture, ecology, and biodiversity monitoring. Specifically, accurate recognition supports crop health assessment, invasive species control, and precision farming workflows Wäldchen and Mäder 2018 ; Aodha et al. 2019 ; Milioto et al. 2018 ; Sa et al. 2018 . Recent advances in deep learning and the availability of large-scale annotated datasets have enabled promising results in automated species identification Mohanty et al. 2016 ; Ferentinos 2018 . However, real-world field conditions introduce significant challenges: images are captured under natural lighting, with occlusions, cluttered backgrounds, and varying viewpoints Ubbens and Stavness 2017 ; Milioto et al. 2018 . Moreover, many species – particularly weeds – exhibit especially subtle morphological differences from crops, making early-growth-stage classification especially difficult Yang et al. 2020 ; Milioto et al. 2018 . While deep learning models have achieved state-of-the-art results on general-purpose plant datasets, existing benchmarks often fall short in agricultural settings. PlantDoc Singh et al. 2020 focuses on plant disease identification but is limited in species diversity, containing only 2,598 images across 13 species and 27 classes, and comprising only 17 disease and 10 healthy categories. DeepWeeds Olsen et al. 2019 provides high-quality imagery of eight weed species from Australia (17,509 images), but lacks broader taxonomic coverage and does not include crop species. In order to enable broader experiments and accurate, generalized classification models, datasets of larger scales of magnitude are necessary – particularly for generalization across classes and to distinct environments. In addition, most existing datasets support single-task classification only — typically predicting species labels without considering whether the species is agriculturally relevant, or whether it behaves as a crop or a weed Steininger et al. 2023 ; Ilyas et al. 2023 ; Goncalves et al. 2023 . In practical farming scenarios, both types of predictions are necessary: one needs to know what species is present, and whether it should be preserved or removed. To address data scaling, biodiversity datasets such as iNaturalist Horn et al. 2017 and PlantCLEF Joly et al. 2018 offer large-scale species coverage and rich metadata such as geolocation and taxonomy, but are not tailored for agricultural applications: they lack task-specific annotations like crop/weed labels. To this end, AgriNet Al Sahili and Awad 2022 offers a sizable collection of agricultural images (160K across 423 classes) collected from diverse geographic sources, but does not provide per-image location metadata (e.g., latitude/longitude) or multi-task labels for tasks such as crop/weed distinction. The AgML Joshi et al. 2023 database centralizes over 50 datasets across various agricultural tasks and environments, including geographic diversity and various classification labels, but its compilation of data from distinct sources leads to the over-representation of certain species and the under-representation of others, which would lead to an unbalanced trained model. In addition, none of the data sources mentioned account for the taxonomic hierarchy present in plant species. This can lead to inherent errors in trained models, and in fact, species classification errors often occur among closely related taxa. For example, two species from the same genus may be visually similar but have vastly different implications in an agricultural field. Current evaluation metrics rarely account for taxonomic hierarchy, such as genus or family-level correctness Elhamod et al. 2021 ; Chen et al. 2019 . Figure 2 : Examples of 40 weed species from the iNatAg dataset. This snapshot demonstrates the visual complexity introduced by growth stage variation, occlusion, and environmental background. To address these gaps, we introduce iNatAg, a large-scale image dataset specifically designed for agricultural plant species classification. iNatAg is derived from publicly available data on iNaturalist iNaturalist ; iNaturalist 2025 ; Horn et al. 2017 , but is curated through a targeted filtering process that selects only species relevant to agriculture. By cross-referencing species names with two authoritative sources – the FAO Ecocrop database FAO and the Weed Science Society of America (WSSA) Weed Science Society of America composite list – we identify and retain species known to be either crops or weeds. The resulting dataset contains 4.72 million images across 2,959 species, including 1,986 crop species and 973 weed species, covering diverse geographical regions and growing conditions. Each image in iNatAg is annotated for both species identity and crop/weed status, enabling multi-task learning. Unlike controlled lab datasets, the images in iNatAg reflect the variability of real-world, crowd-sourced data: changes in background, lighting, plant angle, and resolution. Moreover, nearly all images include latitude and longitude metadata, which we use as input features during model training. Prior work in ecological modeling has shown that incorporating spatial context can help disambiguate species with overlapping visual traits but distinct geographic distributions – a scenario common in both invasive weed mapping and regional crop monitoring. For instance, Aodha et al. 2019 introduced presence-only geographical priors to improve fine-grained image classification, while Cole et al. 2023 proposed spatial implicit neural representations to jointly estimate species distributions at global scale. To benchmark iNatAg, we evaluate nine configurations of Swin Transformer models Liu et al. 2021 , across different scales (Tiny, Small, Base, Large), with and without Low-Rank Adaptation (LoRA) fine-tuning Hu et al. 2021 , and using both image-only and image + location inputs. We assess performance across species, genus, family, and crop/weed levels using standard metrics including accuracy, precision, recall, and F1 score. We also analyze confusion matrices for commonly occurring crops and weeds, providing insight into common failure modes and the role of geolocation in improving classification outcomes. We summarize our contributions as follows: • We introduce iNatAg, a large-scale agricultural dataset with 4.7M images spanning 2,959 crop and

Read the complete text