From Semantic To Instance: A Semi-Self-Supervised Learning Approach

Keyhan Najafian 1 , Farhad Maleki 2 , Lingling Jin 1 , Ian Stavness 1 1 Department of Computer Science, University of Saskatchewan, Saskatoon, Saskatchewan, Canada 2 Department of Computer Science, University of Calgary, Calgary, Alberta, Canada {keyhan.najafian, lingling.jin, ian.stavness}@usask.ca · 2025

Read From Semantic To Instance: A Semi-Self-Supervised Learning Approach by Keyhan Najafian 1 , Farhad Maleki 2 , Lingling Jin 1 , Ian Stavness 1 1 Department of Computer Science, University of Saskatchewan, Saskatoon, Saskatchewan, Canada 2 Department of Computer Science, University of Calgary, Calgary, Alberta, Canada {keyhan.najafian, lingling.jin, ian.stavness}@usask.ca (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-16563.txt.

Introduction

Keyhan Najafian, Farhad Maleki, Lingling Jin, and Ian Stavness segment single objects with very little hand labeling in a 2025 paper. Drawing each object's outline in dense scenes, like wheat heads packed in a field, takes huge effort. Their method starts from rough class maps and turns them into per object masks.

A new image code makes the model attend to shape, texture, and pattern rather than color. The pipeline reached a score of 98.5 percent on wheat head masks, a new best. On a general photo data set it beat standard models by over 12.6 points on the same score, showing reach beyond farm images.

Catalog record

CREATORKeyhan Najafian 1 , Farhad Maleki 2 , Lingling Jin 1 , Ian Stavness 1 1 Department of Computer Science, University of Saskatchewan, Saskatoon, Saskatchewan, Canada 2 Department of Computer Science, University of Calgary, Calgary, Alberta, Canada {keyhan.najafian, lingling.jin, ian.stavness}@usask.ca
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-16563

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Semantic To Instance: A Semi-Self-Supervised Learning Approach Keyhan Najafian 1 , Farhad Maleki 2 , Lingling Jin 1 , Ian Stavness 1 1 Department of Computer Science, University of Saskatchewan, Saskatoon, Saskatchewan, Canada 2 Department of Computer Science, University of Calgary, Calgary, Alberta, Canada {keyhan.najafian, lingling.jin, ian.stavness}@usask.ca, [email protected] Abstract Instance segmentation is essential for applications such as automated monitoring of plant health, growth, and yield. However, extensive effort is required to create large-scale datasets with pixel-level annotations of each object instance for developing instance segmentation models that restrict the use of deep learning in these areas. This challenge is more significant in images with densely packed, self-occluded objects, which are common in agriculture. To address this challenge, we propose a semi-self-supervised learning approach that requires minimal manual annotation to develop a high-performing instance segmentation model. We design GLMask, an image-mask representation for the model to focus on shape, texture, and pattern while minimizing its dependence on color features. We develop a pipeline to generate semantic segmentation and then transform it into instance-level segmentation. The proposed approach substantially outperforms the conventional instance segmentation models, establishing a state-of-the-art wheat head instance segmentation model with mAP@50 of 98.5 % 98.5\% . Additionally, we assessed the proposed methodology on the general-purpose Microsoft COCO dataset, achieving significant performance improvement of over 12.6 % 12.6\% mAP@50. This highlights that the utility of our proposed approach extends beyond precision agriculture and applies to other domains, specifically those with similar data characteristics. Keywords: Instance Segmentation, Semi-Supervised Learning, Self-Supervised Learning, Wheat Head Instance Segmentation, Precision Agriculture, Data Synthesis 1 Introduction Semantic segmentation [ 32 , 36 ] is the pixel-level classification task in which each pixel within an image is assigned to a predefined semantic category, such as sky, tree, or vehicle. In contrast, instance segmentation [ 14 , 16 ] involves identifying and outlining individual objects with well-defined boundaries, even if they belong to the same semantic category. Instance segmentation poses distinct challenges compared to semantic segmentation, as it demands precise identification and separation of individual objects, which is crucial for applications such as autonomous driving [ 41 ] , medical imaging [ 8 ] , and precision agriculture [ 11 ] . Instance segmentation offers a significant advantage in precision agriculture by providing precise identification of individual objects within an agricultural setting [ 3 ] . In contrast to semantic segmentation, it enables the differentiation of individual plants, fruits, or other agricultural product elements. This distinction is critical for tasks such as tracking and counting crops [ 10 , 34 ] , assessing plant health [ 42 , 27 ] , and monitoring growth patterns [ 42 ] , resulting in more targeted interventions, better resource allocation, and improved yield predictions. However, training deep instance segmentation models is challenging due to the need for large-scale datasets with pixel-level annotations of each object instance and the significant computational demand involved. Creating large-scale annotated datasets across various domains is often expensive, time-consuming, and requires rare and highly specialized expertise for data collection and annotation. Moreover, supervised learning introduces challenges such as annotation bias [ 4 ] , lack of generalizability [ 40 ] , and robustness [ 15 ] . Semi-supervised and self-supervised learning leverage partially labeled and unlabeled data, respectively, for model training. They present successful alternatives to supervised learning in addressing its inherent challenges. By reducing the reliance on manual data annotation, semi- and self-supervised learning approaches are greatly expanding the applicability of deep learning in domains where creating annotated datasets is impractical. In precision agriculture [ 21 ] , a data-driven approach to optimizing the monitoring and management of agricultural products, the application of semi- and self-supervised learning can improve the performance of deep learning techniques [ 26 , 5 , 35 , 25 ] . However, the considerable variability in agricultural products and environments, influenced by factors such as weather conditions, lighting, and crop diversity, necessitates the creation of large and diverse annotated databases for effective model training, posing challenges even for semi- and self-supervised learning approaches [ 35 ] . In contrast to general-purpose datasets [ 33 , 19 , 16 ] , agricultural data are often characterized by small, repetitive, and overlapping objects with hidden and fine boundaries [ 25 , 6 ] , making the manual pixel-accurate and object-specific annotations of such large-scale datasets infeasible. This paper proposes a semi-self-supervised learning approach for instance segmentation of images featuring dense, self-similar, and overlapping objects resembling those found in precision agriculture. We experiment and evaluate the effectiveness of our proposed methodology on the wheat head instance segmentation task. In addition, we experimented with the well-known COCO dataset [ 18 ] to showcase the applicability of our proposed method in the general-purpose domains. We first use a limited number of manually annotated images to generate large-scale synthetic datasets with pixel-accurate annotations for each instance. These datasets are then used to develop a instance segmentation model for segmenting images of wheat fields taken in uncontrolled environments. We also apply domain adaptation to address the domain shift between synthetic and real data. Moreover, we propose to use the input RGB image and its corresponding semantic segmentation mask to create an image-mask representation, referred to as GLMask , ultimately enabling the deep model to transform the semantic segmentation mask into an instance mask. The GLMask is constructed by combining multiple color spaces derived from the RGB format and a computationally generated segmentation map. More specifically, GLMask represents the G rayscale from the RGB channels, L channel of LAB, and a semantic segmentation M ask. Models that are overly reliant on color features are problematic in a diverse wheat dataset, where wheat plants change color due to growth stage and variable outdoor lighting. Therefore, the G-L components in GLMask are designed to encourage the model to focus on shape, texture, and pattern while minimizing its dependence on color features. Specifically, the proposed approach follows a semi-self-supervised learning paradigm, wherein only a small number of pixel-accurate and instance-specific human-annotated images and video frames are used to computationally generate two large-scale datasets. Then a high-performing YOLOv9 [ 39 ] model is trained in two consecutive pretraining and domain adaptation phases by leveraging the synthetic datasets. Additionally, we augment the supervisory signals, generated by a pretrained semantic segmentation model within the input data by incorporating GLMask as the model input. Furthermore, we experiment with pseudo-labeling, as a self-supervised approach, utilizing the labels generated by our pretrained model. This work establishes a state-of-the-art top-view wheat head instance segmentation benchmark through the following contributions: (1) introducing a semi-self-supervised learning methodology for high-performance instance segmentation model development; (2) incorporating a shape/pattern emphasized image-mask representation for semantic to instance mask conversion, replacing conventional RGB images, which substantially enhances model performance by providing a strong self-supervisory signal; (3) creating a large-scale computationally annotated synthetic data for deep model training; and (4) a comparison of two domain adaptation methods, rotation augmentation and pseudo-labeling; (5) An evaluation of our proposed methodology on a general-purpose dataset for the instance segmentation task, demonstrating its applicability beyond domain-specific applications. Figure 1 provides an overview of our proposed semi-self-supervised learning approach. Nevertheless, the utility of our proposed approach extends beyond wheat crops

Read the complete text