Lightweight Multispectral Crop-Weed Segmentation for Precision Agriculture

Zeynep Galymzhankyzy, Eric Martinson · 2025

Read Lightweight Multispectral Crop-Weed Segmentation for Precision Agriculture by Zeynep Galymzhankyzy, Eric Martinson (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07444.txt.

Introduction

Zeynep Galymzhankyzy and Eric Martinson build a light model to tell crops from weeds in field images in 2025. Exact weed maps can guide spot spraying, but older image methods often fail under tough field conditions.

Their hybrid model joins transformer and CNN parts and uses more than plain color images. It also takes in near-infrared and red-edge bands, each through its own path, then blends them as needed. On the WeedsGalore data set it beat color-only models by a wide margin in overlap score. With few parameters, it may run in real time on drones and field devices.

Catalog record

CREATORZeynep Galymzhankyzy, Eric Martinson
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE14 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-07444

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Lightweight Multispectral Crop-Weed Segmentation for Precision Agriculture Zeynep Galymzhankyzy, Eric Martinson Abstract Efficient crop-weed segmentation is critical for site-specific weed control in precision agriculture. Conventional CNN-based methods struggle to generalize and rely on RGB imagery, limiting performance under complex field conditions. To address these challenges, we propose a lightweight transformer-CNN hybrid. It processes RGB, Near-Infrared (NIR), and Red-Edge (RE) bands using specialized encoders and dynamic modality integration. Evaluated on the WeedsGalore dataset [ 5 ] , the model achieves a segmentation accuracy (mean IoU) of 78.88%, outperforming RGB-only models by 15.8 percentage points. With only 8.7 million parameters, the model offers high accuracy, computational efficiency, and potential for real-time deployment on Unmanned Aerial Vehicles (UAVs) and edge devices, advancing precision weed management. I Introduction Weed management is a critical challenge in precision agriculture, where accurate and timely identification of weeds enables site-specific control, reducing herbicide use and boosting crop yields. Recent advances in deep learning, particularly convolutional neural networks (CNNs), have improved crop–weed segmentation by analyzing field imagery. However, existing methods face significant limitations. Many rely solely on RGB imagery, which struggles to generalize across diverse vegetation types and lighting conditions, such as shadows or overcast skies. Additionally, static fusion strategies for combining multiple data types, like RGB and infrared, lack robustness to sensor noise or missing data, limiting their reliability in real-world settings. Transformer models, while effective at capturing complex patterns, are often too computationally intensive for deployment on resource-constrained Unmanned Aerial Vehicles (UAVs). Real-time performance, essential for practical UAV-based applications, remains underexplored in models using multispectral data, such as Near-Infrared (NIR) and Red-Edge (RE) bands. To address these challenges, we propose a lightweight transformer–CNN hybrid model for efficient crop–weed segmentation. Our approach uses multispectral imagery (RGB, NIR, RE) through specialized encoders and a dynamic fusion mechanism that adapts to varying field conditions. With only 8.7 million parameters, the model is optimized for real-time inference on UAVs, offering high accuracy and computational efficiency. II Related Work II-A Deep Learning Methods for Weed Segmentation Early approaches relied on handcrafted features and classifiers like Random Forests [ 1 ] . While computationally efficient, these methods lacked robustness to field variability and required manual feature engineering. Convolutional Neural Networks (CNNs) significantly improved segmentation accuracy. For WeedMap, Sa et al. [ 2 ] used a modified SegNet, achieving strong performance for background and crop classes but lower accuracy for weeds. Celikkan et al. [ 5 ] evaluated DeepLabv3+ and MaskFormer on WeedsGalore, achieving mean Intersection over Union (mIoU) scores above 82% with multispectral inputs. However, CNNs often require deep architectures to model long-range dependencies, increasing computational demands and limiting efficiency on UAV platforms [ 13 ] . Vision Transformers (ViTs) have recently gained traction for weed segmentation. Castellano et al. [ 6 ] proposed lightweight variants, Lawin and DoubleLawin, outperforming DeepLabv3 on WeedMap. Reedha et al. [ 7 ] and Wang et al. [ 9 ] showed that transformers excel at capturing global shape and context, crucial for distinguishing overlapping or visually similar vegetation. However, many ViTs remain computationally intensive, hindering real-time deployment. II-B Multispectral Fusion Strategies Multispectral imagery provides spectral cues beyond RGB. NIR reflects plant health by capturing canopy structure, while RE detects chlorophyll variations, aiding crop–weed differentiation. Common fusion methods, such as channel stacking or concatenation, treat modalities equally, ignoring variations in quality or relevance. WeedsGalore [ 5 ] demonstrated that combining RGB, NIR, and RE improves segmentation over RGB-only models, particularly for rare weed classes. However, most fusion methods are static and cannot adapt to modality noise or dropouts. There is a need for adaptive fusion mechanisms that dynamically prioritize modalities based on their relevance or quality, enhancing robustness to environmental variability. II-C UAVs and Real-Time Constraints UAVs are increasingly used to collect high-resolution, georeferenced multispectral data. Orthomosaic generation and pixel-level annotations enable large-scale field analysis, but real-time deployment remains challenging. CNNs and transformers often incur high memory and latency costs, limiting their use on resource-constrained UAV platforms. Lightweight transformer–CNN hybrids offer a promising solution, balancing efficiency and performance with multispectral inputs, as demonstrated in tasks like image super-resolution [ 17 ] . Our proposed architecture addresses these challenges through modular spectral encoding, adaptive gated fusion, and efficient pyramid-based decoding, optimized for UAV-based precision agriculture. III Multispectral and Multitemporal Data The WeedsGalore dataset [ 5 ] is a publicly available benchmark for semantic and instance-level segmentation in maize fields, collected using an Unmanned Aerial Vehicle (UAV). Data was acquired over a 1{,}840\;\mathrm{m}^{2} agricultural plot in Marquardt, Germany, with a DJI Phantom P4 Multispectral UAV. Four flights, conducted between May and June 2023, captured high-resolution imagery at a 5 m altitude, achieving a ground sampling distance (GSD) of 2.5 mm. Each flight produced approximately 1,150 raw frames, from which 156 image tiles of 600{\times}600 pixels were manually cropped and annotated. The tiles are labeled across five semantic categories: maize, amaranth, barnyard grass, quickweed, and other weeds, representing common crops and weed species in Central European maize farming. The dataset is spatially split into 109 training (70%), 23 validation (15%), and 24 test (15%) tiles, ensuring no spatial overlap between splits. Fig. 1: WeedsGalore dataset: acquisition and annotation workflow. Reprinted from [ 5 ] . WeedsGalore stands out due to several key features: • Multispectral coverage : RGB, Red-Edge (730 nm), and Near-Infrared (NIR, 840 nm) bands provide rich spectral information, enhancing crop–weed separability compared to RGB-only datasets. • Multitemporal sampling : Four flights capture distinct plant growth stages, enabling models to learn robustly across temporal variations. • High annotation density : Over 10,081 plant polygons, with an average of 78 instances per tile, offer a challenging benchmark for small-object segmentation, surpassing datasets like CWFID. • Public baselines : Performance scores for DeepLabv3+ and MaskFormer, using both RGB and Multispectral Input (MSI), provide robust baselines for comparison. For multimodal segmentation, each tile is processed into a 5-channel array (RGB, NIR, Red-Edge) and resized to 600{\times}600 pixels. Semantic masks are remapped into three categories: background, crop, and weed. On-the-fly augmentations, including random flips and rotations, are applied during training to improve generalization. This combination of spectral diversity, dense annotations, and temporal variation makes WeedsGalore ideal for evaluating dynamic modality weighting and Transformer-based segmentation models. Fig. 2: WeedsGalore dataset: example tile with RGB, NIR, and Red-Edge channels, alongside the remapped 3-class segmentation mask. With its high-resolution 2.5 mm GSD, multispectral richness, and multitemporal coverage, WeedsGalore is well-suited for developing site-specific weed management algorithms. By benchmarking on this dataset, we demonstrate that Transformer-driven multimodal fusion enhances segmentation accuracy while remaining efficient for real-time UAV inference. IV Proposed Methodology Fig. 3: Architecture of the lightweight transformer–CNN hybrid, showing modality-specific encoding, Transformer refinement, gated fusion, and pyramid-based decoding. IV-A Model Architecture The proposed lightweight transformer–CNN hybrid is designed for multispectral crop–weed segmentation using Unmanned Aerial Vehicle (UAV) imagery. It processes five-channel inputs—RGB, Near-Infrared (NIR), and Red-Edge (RE)—through modality-specific convolutional encoders that extract low-level spatial features. Each modality stream is refined by dedicated Transformer blocks to capture long-range spatial dependencies. A key component, the Gated Fusion Module, dynamically weighs each spectral modality using learned weights, prioritizing the most informative inputs under varying field conditions, such as sensor noise or lighting changes. The fused representation is processed by

Read the complete text