WS 2 \text{WS}^{2} : Weakly Supervised Segmentation using Before-After Supervision in Waste Sorting

Andrea Marelli, Alberto Foresti, Leonardo Pesce, Giacomo Boracchi, Mario Grosso · 2025

Read WS 2 \text{WS}^{2} : Weakly Supervised Segmentation using Before-After Supervision in Waste Sorting by Andrea Marelli, Alberto Foresti, Leonardo Pesce, Giacomo Boracchi, Mario Grosso (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06485.txt.

Introduction

Sorting waste on a moving belt is the idea behind a 2025 computer vision paper led by Andrea Marelli, with four co-authors. The workers who already pick items off the belt supply the help. Factories still rely on people to spot unwanted objects in mixed streams. Full labeling for training software is too costly across many sorting jobs.

The authors define before-after supervision: a camera sees the belt before a worker removes an item and again afterward, and the difference itself teaches the system. They release a dataset of more than eleven thousand high-resolution frames from several camera views, the first of its kind for this task. A full processing pipeline is included, and several weakly supervised methods, which learn from rough hints rather than exact outlines, are benchmarked on it.

Catalog record

CREATORAndrea Marelli, Alberto Foresti, Leonardo Pesce, Giacomo Boracchi, Mario Grosso
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-06485

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

WS 2 \text{WS}^{2} : Weakly Supervised Segmentation using Before-After Supervision in Waste Sorting Andrea Marelli, Alberto Foresti, Leonardo Pesce, Giacomo Boracchi, Mario Grosso Abstract In industrial quality control, to visually recognize unwanted items within a moving heterogeneous stream, human operators are often still indispensable. Waste-sorting stands as a significant example, where operators on multiple conveyor belts manually remove unwanted objects to select specific materials. To automate this recognition problem, computer vision systems offer great potential in accurately identifying and segmenting unwanted items in such settings. Unfortunately, considering the multitude and the variety of sorting tasks, fully supervised approaches are not a viable option to address this challange, as they require extensive labeling efforts. Surprisingly, weakly supervised alternatives that leverage the implicit supervision naturally provided by the operator in his removal action are relatively unexplored. In this paper, we define the concept of Before-After Supervision, illustrating how to train a segmentation network by leveraging only the visual differences between images acquired before and after the operator. To promote research in this direction, we introduce WS 2 \text{WS}^{2} (Weakly Supervised segmentation for Waste-Sorting), the first multiview dataset consisting of more than 11 000 high-resolution video frames captured on top of a conveyor belt, including ”before” and ”after” images. We also present a robust end-to-end pipeline, used to benchmark several state-of-the-art weakly supervised segmentation methods on WS 2 \text{WS}^{2} . 1 1 1 The WS 2 \text{WS}^{2} dataset is publicly available for download at https://zenodo.org/records/14793518 , all the details are reported in the supplementary material. Figure 1 : In a waste sorting plant, two cameras, C_{1} and C_{2} , are placed along a conveyor belt where a human operator manually removes unwanted objects ( red ) from a mixed waste stream, leaving on the belt only wanted ones ( green ). C_{1} captures the belt section before the HO’s intervention, while C_{2} captures the section after , where only wanted objects remain. We aim to train, without any pixel-wise annotation nor additional supervision, a binary segmentation model for wanted / unwanted objects. (a) (b) (c) Figure 2 : (a) “ Before ” frame showing both wanted and unwanted waste items on the conveyor belt. (b) Annotation overlay: green outlines mark the wanted items to keep (semi-transparent PET bottles), while red outlines indicate unwanted waste to be removed (all other plastic materials). (c) “ After ” frame in which only the wanted items remain. The absence of the unwanted waste, present only in the “ before ” frames, serves as supervision in our WS segmentation approach, enabling the training of a network able to segment the unwanted items. 1 Introduction Despite the increasing automation in multiple industrial sectors, Human Operators (HOs) are still strictly necessary for numerous tasks. Many quality control processes, ranging from food to pharmaceutical lines, rely on HOs meticulously monitoring a continuous flow of items, identifying and manually removing anomalies or imperfections based on each item’s visual appearance. This principle is reflected in modern recycling plants. Here, waste is first coarsely sorted by NIR-based automatic machines, then ends up on multiple separate conveyor belts where HOs manually remove undesired items from the stream, leaving on each dedicated belt only the objects of a specific material based on visual cues. Figure 1 illustrates this process: the HO discards the unwanted objects, leaving on the belt only the wanted ones. Supporting this activity with automatic tools would improve both the efficacy and efficiency of sorting plants, reducing the risks of injuries and alleviating stressful working conditions for HOs [ 24 ] . With the advent of deep learning, computer vision systems have emerged as promising solutions to automating sorting tasks. In fact, a semantic segmentation network can be trained to analyze images or videos capturing the stream running on a belt and segment the unwanted items, supporting humans in their operation and paving the way for the system’s automation. In spite of substantial research on industrial waste sorting segmentation, existing efforts have been mainly based on datasets designed for Fully Supervised (FS) learning (Table 1 ). Unfortunately, gathering extensive manual annotations in the form of pixel-level annotated segmentation masks can be very time-consuming. Considering the fine-grained sorting demands of modern recycling plants, a dedicated detection model would be required for each specific line of work. Therefore, training each specific model in a FS fashion would require a large amount of data to be annotated for every belt, resulting in prohibitive costs. In this paper, we investigate an alternative learning paradigm for segmenting unwanted items in sorting tasks, without the need to manually annotate the large quantity of ground-truth masks required to train a distinct segmentation network for each individual line. Our key insight lies in the fact that, comparing images of the belt sections captured before and after the HO’s removal operation (Figure 2 ), it is possible to learn the supervisory signals for the segmentation task that is already implicit in the HO’s activity. In fact, by removing the unwanted objects from the stream, the operator effectively indicates which items should be segmented without any manual annotations. We define this special supervision signal as ” Before-After ” supervision. Nonetheless, considering that the HO’s operation alters the relative position also of the wanted elements on the belt, straightforward solutions like subtracting (or directly comparing) pairs of corresponding before and after images is not effective to recover the desired segmentation masks. Thus, inspired by Zerowaste [ 2 ] , we detailed a self-supervised learning solution in which an auxiliary classifier \mathcal{K}_{\theta} is trained to distinguish between before and after images, and Saliency Maps (SMs) of \mathcal{K}_{\theta} are then computed to leverage the implicit supervision provided by the HO. Specifically, the SMs for the before class on the corresponding images highlight the most representative regions of the before class, which essentially correspond to the unwanted objects, as these are never or rarely seen in the after images. Training a neural network to identify unwanted items via this Weakly Supervised (WS) framework would enable a vision system to automatically assess the HO’s performance and provide corrective feedback, effectively learning the selection task performed by the HO directly from a video stream. This novel self-supervised paradigm not only learns a specific sorting criterion but, for each new conveyor belt, it requires only collections of before/after intervention frames, with no need for manual annotations. To boost research in this area, our first contribution is \text{WS}^{2} (Weakly Supervised segmentation for Waste Sorting), a comprehensive, open-access “Before-After” dataset collected in Seruso S.p.A., a real-world plastic sorting facility in Verderio (LC), Italy. Specifically, the \text{WS}^{2} dataset comprises a sequence of more than 11 000 video frames acquired before and after a HO selecting semi-transparent colored PET objects from a conveyor line where only plastic items are present (Figure 2 ). This corresponds to a valuable benchmark in a waste sorting modern scenario. Unlike existing datasets (see Section 2.2 ), we specifically collected a large number of video frames to test WS segmentation methods exploiting temporal correlation in videos like [ 14 ] . In our second contribution, we design a comprehensive pipeline for general SM-based WS segmentation methods to exploit the Before-After supervision paradigm. The pipeline, detailed in Section 3 ,

Read the complete text