Leveraging generative adversarial networks and SE-UNet for high-precision tobacco leaf image segmentation and blend uniformity detection Authors: Meizhou Ding, Xianhui Peng, Xiaonan Ji, Song Chen, Quansheng Li, Helan Zou, Zhiwei Fan, Shucai Wang Published: 2026 ABSTRACT AbstractThe uniformity of tobacco blend mixing is a critical factor influencing cigarette quality. However, due to the dynamic nature of the blending process and the complex characteristics of the materials involved, current detection methods primarily rely on manual sampling or offline analysis. These approaches suffer from poor real-time performance and lack representativeness. To address these limitations, this study explores methods for generating tobacco-leaf image data and proposes a real-time semantic segmentation technique for tobacco-leaf images based on U-Net. First, a small set of tobacco leaf images was acquired to establish a raw dataset of composite tobacco leaf images. Three generative adversarial networks—CycleGAN, WGAN, and DCGAN—were employed to generate tobacco leaf images from this dataset. Based on image quality, the optimal image generation network was selected. Experimental results showed that CycleGAN produced the highest-quality tobacco leaf images. Next, the Squeeze Excitation Attention (SE) mechanism was integrated into the VGG16 backbone network. This enhancement improved the network’s ability to extract features from various types of tobacco leaf images while suppressing interference from irrelevant pixels. The results demonstrated that, compared to mainstream models such as Segformer, DeepLabV3, PSPNet, and the unmodified U-Net, the proposed SE-UNet model achieved superior segmentation accuracy, excellent real-time performance, and overall optimal segmentation capabilities. Compared to Segformer, DeepLabV3, PSPNet, and the original model, the MIOU in the evaluation metrics improved by 17.46, 8.7, 13.39, and 2.77 percentage points, respectively. Finally, the pixel areas of different tobacco leaf types were extracted and calculated from the segmented images to determine the proportion of each leaf type within the images. This research offers a novel approach for practical tobacco production and quality inspection, while also providing a new pathway for real-time online detection of other agricultural products.Keywords: Generative adversarial networks, Semantic segmentation, Comparative experiments, Tobacco flake uniformity, Pixel area, SE attention mechanismSubject terms: Computational biology and bioinformatics, Mathematics and computing, Plant sciences Introduction China is a leading producer and consumer of tobacco and related products. In 2019, the total import and export value of tobacco and related products reached US$3.325 billion, with cigarette exports amounting to US$705.4 million, accounting for 49.68% of the total trade in tobacco and related products1. The World Health Organization’s Framework Convention on Tobacco Control (FCTC) has established guidelines under Article 9 (Regulation of the Contents of Tobacco Products) and Article 10 (Disclosure of Tobacco Product Information). These guidelines require tobacco product manufacturers and importers to disclose the composition of tobacco products to governments, including the types of tobacco leaf used and the proportion of each type2. Tobacco manufacturers are required to possess equipment and methods for testing and measuring the composition of tobacco shreds. The blending ratios of tobacco flakes, expanded flakes, reconstituted tobacco, and shredded tobacco in cigarettes significantly influence their characteristics, physical properties, and sensory quality. Therefore, achieving high-precision and high-efficiency identification of tobacco flake types is essential for authenticating tobacco products, exploring formulation design, ensuring quality in tobacco blending processes, and maintaining consistency across similar products3. In recent years, deep learning has provided advanced and efficient solutions for image processing tasks, including image classification, segmentation, and object detection4,5. Its exceptional feature extraction capabilities have significantly reduced the workload associated with these tasks. Due to variations in application objectives, researchers often modify network architectures to address specific practical challenges6,7. Deep learning, integrated with machine vision, has been widely applied in various agricultural inspection tasks, including plant pest and disease identification (e.g., wheat blast disease image classification and plant leaf disease classification), plant variety identification (e.g., vegetable, fruit, and rapeseed variety classification), crop quality inspection (e.g., corn seed defect detection), and research on rapid fruit tree sorting systems (e.g., online citrus sorting systems)8,9. With the advancement and maturation of deep learning technology in agricultural inspection tasks, research on the integration of deep learning and machine vision for tobacco product identification and quality inspection has steadily increased. Gao et al. developed four convolutional-neural-network-based models for tobacco leaf component recognition and reported 100% accuracy on the training set and 84.95% accuracy on the test set 10. Although the training performance appears high, the substantial gap between training and test accuracy suggests that the model may have suffered from overfitting or insufficient generalization to unseen samples. This discrepancy may be related to the model’s reliance on local visual features of tobacco leaves, which can reduce robustness when sample characteristics vary. Wei et al. first refined the contours of tobacco shreds to extract their physical characteristics and then developed an image recognition model with an average relative error below 5%11. This approach relies on physical features such as length, area, and width. However, when sample characteristics become complex, these features are insufficient to describe the high-dimensional nature of the data. Zhong et al. developed a method for identifying individual tobacco shreds using residual neural networks. The test set achieved accuracy and recall rates exceeding 96%, offering a new approach for tobacco shred classification. However, this method struggles to accurately identify tobacco shreds with subtle macroscopic differences and minor distinctions, such as certain reconstituted tobacco shreds and leaf shreds12. Niu et al. improved the VGG16 network by reducing the number of convolutional kernels and adding residual modules. The modified Light-VGG16 model achieved a recognition accuracy of 95.5% on a self-constructed tobacco shred dataset. Initially, the images underwent grayscale conversion. The background threshold unrelated to tobacco shreds was determined using the maximum interclass variance method. The grayscale image was then converted into a binary image, and OpenCV was used to extract the minimum bounding rectangle of the tobacco shred region13. Research on tobacco leaf grading is well-established and relatively advanced. However, due to the difficulty in obtaining tobacco shred samples, their small size, and minimal morphological variation, studies on image recognition remain limited. Near-infrared spectroscopy and computer vision are two different but potentially complementary approaches for tobacco blending analysis. Near-infrared spectroscopy mainly reflects spectral and chemical composition information and is suitable for rapid non-destructive detection of material properties. However, its discrimination ability may decrease when different tobacco components have similar spectral responses. In contrast, computer vision focuses on external visual features, including color, morphology, texture, contour, and spatial distribution, which are directly related to the surface distribution and mixing uniformity of tobacco shreds. Therefore, this study adopts a machine-vision-based semantic segmentation method to identify different tobacco components and quantify their visual area proportions, providing an image-based approach for tobacco blend uniformity evaluation14,15. Generative adversarial networks (GANs) have opened new avenues for advancing computer vision technology. Designed based on the principles of zero-sum games and adversarial training, they enable high-resolution image generation and image segmentation transfer. As a result, GANs have found widespread applications in fields such as medicine, biology, and agriculture. Hu et al. improved the effectiveness of lung cancer image analysis using the SAGAN method, thereby providing more robust decision support for clinical practice16. Chen et al. proposed a conditional generative adversarial network with a multi-scale discriminator to address the challenges of limited white blood cell samples and unclear cell images in the dataset. Compared to training solely on real data, increasing cell sample diversity improved the average accuracy of cell classification from 90.4% to 94.7% 17. Several researchers have developed novel GAN architectures, such as PCGAN, SinGAN, BigGAN, and SRGAN. By employing high-resolution image generation networks to simulate the distribution of tobacco leaf image data and generate images that preserve original semantic information, the issue of insufficient samples for real-time semantic segmentation of tobacco leaf images using deep learning-based computer vision methods can be effectively mitigated. Convolutional neural networks are highly effective at learning both shallow and deep features in images, allowing researchers to apply them successfully to various image semantic segmentation tasks. Du et al. improved the Mask R-CNN model to address the challenge of detecting small targets, such as grape clusters and stems in fresh grapes, thereby enabling mechanization and automation in flower thinning18. Zhao et al. tackled issues such as low accuracy, poor real-time performance, and challenges in interpretability for navigational path recognition between tea garden ridges. By optimizing the UNet model and integrating the strengths of both UNet and ResNet architectures, the improved model achieved an average intersection-over-union (IoU) of 91.89% for navigational path segmentation under varying light conditions and weed coverage19. Yu et al. replaced the backbone network Xception with MobileNetV2 and introduced the CBAM attention mechanism. The improved model achieved detection accuracy and mean intersection-over-union (mIoU) of 96.27% and 93.18%, respectively, for detecting uncultivated field ridges20. Zhu et al. proposed the SwinT-YOLOACT corn ear segmentation model based on YOLOACT to enable high-throughput, automated acquisition of corn phenotypic parameters for unmanned inspection robots. Experimental results demonstrated that SwinT-YOLOACT achieved an average mask accuracy of 79.43% and an inference speed of 35.44 frames per second21. Wang et al. employed Cascade Mask R-CNN as the baseline network, integrating HrNetV2 and FPN modules to construct a pig body detection and segmentation model. This approach addresses issues such as mutual occlusion and adhesion among pigs, enabling high-precision detection and segmentation of group pig images in complex environments22. This study initially employed generative adversarial networks (GANs) to analyze existing tobacco shred image datasets, generating images with complete semantic information. Limited training samples may reduce the diversity of visual features and increase the risk of overfitting in deep semantic segmentation models. Therefore, data augmentation is necessary to improve the representativeness of the training set. Compared with conventional augmentation methods such as rotation, cropping, and brightness adjustment, GAN-based image generation can learn the distribution characteristics of real tobacco shred images and generate new samples with similar visual and semantic features. In this study, GAN-based augmentation was used as an auxiliary data expansion strategy to enrich the training samples for subsequent semantic segmentation, rather than as a replacement for real image acquisition.Through a comparative evaluation of three networks—CycleGAN, WGAN, and DCGAN—the optimal GAN architecture was selected. A combined dataset was then created using both the generated and original images. An improved UNet network was applied to segment the tobacco images, yielding segmentation accuracy, mean intersection-over-union (mIoU), and single-image processing time metrics. These results were compared with those from three traditional semantic segmentation methods. The experiments demonstrated that the proposed method achieved high segmentation accuracy, excellent real-time performance, and overall superior segmentation capabilities. Materials and methods Test materials and image acquisition The four tobacco samples used in this experiment were provided by Henan Tobacco Industry Co., Ltd. They were divided into two distinct brands, comprising 1423.8 g of leaf tobacco, 890.1 g of stem tobacco, 977 g of expanded tobacco, and 635.9 g of reconstituted tobacco. The experimental image acquisition system employed an industrial camera and a Hikvision area-scan light source, with the image acquisition platform shown in Fig. 1. The camera used was a Dahen MARS-900-120GTC industrial model, equipped with a Global Shutter CMOS sensor chip. It has a resolution of 4192 × 2160 pixels, with 12 million effective pixels, paired with a 12-mm zoom lens. The camera was powered via a POE interface and used a Gigabit Ethernet cable with a GIGE interface for data transmission between the camera and the computer. Images were captured at a distance of 38 cm from the conveyor belt, with a 10 cm height between the camera and the area-array light source. Image acquisition took place in Room 425 of the Engineering Building at Huazhong Agricultural University. Camera parameters were configured, and image capture was controlled using MV Viewer software, with the captured images stored locally. To maximize sample variability and ensure model generalization during training, 3,000 tobacco shred images were randomly collected. The specific distribution is detailed in Table 1 below, and the dataset construction workflow is illustrated in Fig. 2. Schematic Diagram of the Tobacco Shreds Image Acquisition Platform. Table 1Proportion of Components in Self-Blended Tobacco Mixtures.TS(g)CS(g)ETS(g)RTS(g)1001934100034100004100193010019001002634100260411026301102681290260129020812 Proportion of Components in Self-Blended Tobacco Mixtures. Schematic Diagram of the Cigarette Tobacco Data Set Construction Process. Tobacco leaf image generative adversarial network Generative adversarial networks GAN typically consist of two components: a generator and a discriminator. The generator learns the data distribution of real samples and generates data that progressively resembles real data. The discriminator determines whether input data is real, outputting a probability value23,24. The GAN network architecture is shown in Fig. 3. The random noise (z) is typically a Gaussian variable, and the generator (G) imposes no restrictions on its dimension. After the random noise is input into the generator, it produces data G(z) (generated images), which are then fed into the discriminator (D). The essence of GAN networks continuously optimizing the generator and discriminator lies in minimizing the generator and maximizing the discriminator, with their objective function defined as: Schematic Diagram of the GAN Network Architecture. In the Eq. (1), D(X) represents the probability that the discriminator outputs a “true” judgment when the input data is genuine; D(G(Z)) represents the probability that the discriminator classifies the input data as genuine when the input is synthetic data G(Z); Pdata(x) denotes the distribution of genuine data; pz​(z) denotes the distribution of synthetic data; and E represents the expectation. The generator aims to make the generated data indistinguishable from real data, i.e., D(G(Z)) approaches 1, while D(G(Z)) approaches 0, thus increasing V(D,G). max(D)V(D,G) represents the discriminator’s objective function, which maximizes the negative log-expectation of the discriminator’s prediction probability25,26. Tobacco shreds image generation network DCGAN, WGAN, and CycleGAN were selected because they represent three typical GAN generation strategies: basic convolutional generation, stability-improved generation, and semantic-preserving image translation. Comparing these models allowed us to evaluate which strategy was more suitable for generating tobacco shred images with complex morphology and overlapping structures. DCGAN combines convolutional neural networks (CNNs) with generative adversarial networks (GANs) to improve the quality and diversity of generated images23. The DCGAN generator (G) consists of three deconvolution layers and three convolutional layers, while the discriminator (D) is composed of four convolutional layers and one fully connected layer. The architecture of the DCGAN network is shown in Fig. 4. DCGAN Network Architecture Diagram. First, a random noise matrix of size 64 × 64 × 128 is input into the deconvolution layer. Then, a feature tensor of size 128 × 128 × 128 is passed to the convolutional layer, where a 3 × 3 convolutional kernel is applied. After batch normalization and ReLU activation, the output is fed into the next layer. Finally, the last convolutional operation, with a Tanh activation function, produces a generated tobacco leaf image of size 512 × 512 × 3. The discriminator takes both generated and original images as input, aiming to determine whether an image is “real” or "fake." The input image undergoes four layers of convolutional operations using a 5 × 5 kernel. After batch normalization and the Leaky ReLU activation function, the image is passed to a fully connected layer. Finally, the fully connected layer outputs a binary value ("1" or "0") via the sigmoid function. The expressions for the ReLU, Tanh, and Leaky ReLU activation functions are shown in Eqs. (2), (3), and (4). WGAN uses the Wasserstein distance to measure the difference between the generated and real data distributions, which can improve training stability compared with the original GAN. The critic in WGAN is required to satisfy the Lipschitz continuity constraint. In this study, WGAN was used as a representative stability-improved GAN model for tobacco shred image generation, rather than as a newly proposed GAN variant27. Therefore, the analysis mainly focuses on its application-level comparison with DCGAN and CycleGAN. More detailed theoretical issues, including gradient penalty implementation in WGAN-GP, dataset-specific convergence analysis, and computational complexity, will be investigated in future work. The WGAN architecture used in this study is composed of fully convolutional layers, as shown in Fig. 5. WGAN Architecture Diagram. Unlike GANs, WGANs use the Wasserstein distance to measure the minimum cost required to transfer the generated distribution ​ to the real distribution ​ along a planned path. Real and generated samples are randomly sampled, and the distance between the two samples is calculated under the joint distribution expectation. A lower bound for this expectation is then determined by searching across all joint distributions. In the Eq. (5), m and k denote samples from the true distribution and the generated distribution, respectively; represents all sets obtainable from the joint distribution; γ denotes any path from the joint distribution; c(m,k) represents the transition cost, here defined as ‖m–k‖. CycleGAN is an asymmetric image translation network that can be trained without relying on paired data26,28. It demonstrates strong generalization capabilities and enables diverse image transformations by capturing feature mapping relationships across different image domains. The loss function of CycleGAN primarily encompasses three components: adversarial loss, cycle consistency loss, and identity loss. The structure of the adversarial loss function is as follows: In Eqs. (6) and (7), denotes the adversarial loss function, and E represents the mathematical expectation. S denotes the source domain, s represents an image in the source domain, T denotes the target domain, t represents an image in the target domain, F is the generator from the target domain to the source domain, G is the generator from the source domain to the target domain, is the discriminator in the source domain, and is the discriminator in the target domain. The structure of CycleGAN is illustrated in Fig. 6. CycleGAN Network Architecture Diagram. Evaluation criteria for tobacco shred image segmentation networks To quantitatively evaluate the semantic segmentation performance of the UNet model on the tobacco leaf image dataset, this paper uses pixel accuracy (PA) and mean intersection over union (mIoU)—common metrics in image segmentation tasks—as criteria for assessing segmentation quality 29. The model’s segmentation results are compared with manually annotated images (ground truth labels) for analysis. PA and mIoU are defined in Eqs. (8) and (9). In the formula, N denotes the number of semantic categories, where N=4 in this paper. represents the number of actual pixels belonging to the semantic category i. denotes the number of pixels incorrectly classified as class j when they belong to class i. In the formula, denotes the number of pixels where semantic class j is identified as class i. Among the aforementioned metrics, MIOU correlates positively with segmentation performance. Due to its simplicity and strong representativeness, it is frequently used as the primary basis for evaluating semantic segmentation model performance. Test results and analysis Real-time Segmentation of Tobacco Shreds Images Based on Generative Adversarial Networks and UNet. The process consists of three main steps: 1) Acquire tobacco shred images, normalize them, and create the original image dataset. 2) Generate tobacco shred images using a generative adversarial network to obtain the optimal model. 3) Perform real-time segmentation of tobacco shred images using UNet and conduct a comparative analysis. Test platform To ensure the fairness of the comparative experiments and the comparability of the results, this study re-implemented and evaluated baseline models such as UNet, PSPNet, DeepLabV3 and Segformer under standardised experimental conditions. All models utilized the same tobacco leaf shredding image dataset, data splitting strategy, image pre-processing workflow, input dimensions, training strategy and key parameter settings. The experiments were implemented using the Python programming language and the TensorFlow deep learning framework, running on a Windows 11 Pro operating system with GPU acceleration via the CUDA 11.8 parallel computing framework. The hardware platform was a DELL G15 graphics workstation, configured with an 8-core processor, 64 GB of RAM and an NVIDIA RTX 3060 GPU. These consistent experimental settings helped to minimise the impact of variations in data processing, training configurations and computing environments on the results of the model performance comparison. Image generation of tobacco shreds based on generative adversarial networks Tobacco shreds dataset Due to the high resolution of the collected tobacco leaf dataset, it consumed substantial memory. Using it directly as input for subsequent model training would significantly increase the number of computational nodes required, potentially leading to computational overflow. Therefore, this study reduced the resolution of the raw tobacco leaf images from 3000px × 4000px to 2048px × 1536px. After manually screening and removing distorted images and enhancing the contrast between the target and background, 1000 images were selected as the final tobacco leaf image dataset. An example of the dataset is shown in Fig. 7. Example of Raw Image Dataset for Tobacco Shreds. Training generative adversarial network models This paper trained DCGAN, WGAN, and CycleGAN on the raw tobacco leaf image dataset. Figures 8, 9 and 10 shows the trends of G_Loss and D_Loss over iterations for the three generative adversarial networks (GANs) during training on the raw tobacco leaf image dataset. Figure 8 reveals that during the initial training phase, G_Loss in the DCGAN network decreased rapidly with each iteration, followed by a slow, oscillating increase. The trend of D_Loss over iterations mirrors that of G_Loss, with both metrics crossing paths as they rose or fell with increasing iterations. The G_Loss of the WGAN network rose slowly during the initial iterations, then increased significantly around iteration 10,000 before rapidly declining to its maximum. The D_Loss exhibited substantial fluctuations as iterations increase, yet its overall magnitude remained smaller than G_Loss, indicating the discriminator holds an advantage over the generator. Figure 10 shows that the CycleGAN generators (G_A, G_B) exhibited significant fluctuations in magnitude during the early stages of training. Subsequently, the overall oscillation amplitude converges and gradually stabilizes into a relatively steady fluctuation state. The discriminator loss (D_A, D_B) exhibited a trend of maintaining minor oscillations within a low-level range as training epochs progressed, without showing any significant upward or downward trend. Additionally, cycle consistency (Cycle_A, Cycle_B) and identity loss (idt_A, idt_B) simultaneously exhibited oscillatory convergence characteristics similar to those of the generator and discriminator losses. Overall, the losses fluctuate within relatively stable ranges during the later stages of training. DCGAN Network Evolution Chart. Trend Chart of WGAN Network Changes. CycleGAN Network Trend Chart. By comparing the changes in G_Loss and D_Loss across the three networks, it can be concluded that, for generating images from the tobacco leaf dataset, the discriminator outperforms the generator by a narrow margin. Tobacco shreds image segmentation based on UNet Image annotation and dataset preparation The training of the real-time semantic segmentation model was based on supervised learning; therefore, manual annotation was required to provide semantic labels for the collected tobacco shred images. A total of 3,000 original tobacco shred images were collected using the image acquisition platform. In addition, 1,000 GAN-generated images were used as augmented samples, resulting in a composite dataset with an original-to-augmented image ratio of 3:1. The composite dataset contained 4,000 images in total, including 2,000 images for training, 1,000 images for testing, and 1,000 images for validation. Four tobacco shred categories were defined for annotation: leaf tobacco shreds (TS), cut stem shreds (CS), expanded tobacco shreds (ETS), and reconstituted tobacco shreds (RTS). The images were annotated using LabelME, and the annotation results were converted into pixel-level semantic masks for subsequent semantic segmentation model training (Fig. 11). Representative tobacco shred images generated by different GAN models. UNet model recognition performance Figure 12 shows the training loss and average intersection-over-union (IoU) curves of the UNet model. In the early training stage, the loss decreased rapidly and the average IoU increased, indicating that the model gradually learned the main semantic features of different tobacco shred components. After approximately 50 iterations, slight fluctuations appeared in both curves, which may be related to the transition from transferred parameter initialization to task-specific training. After approximately 100 iterations, the curves became relatively stable, suggesting that the model had reached a steady training state under the current experimental settings. The final model parameters were selected according to the highest average IoU during training and were then used for validation. The dataset was divided into independent training, validation, and test sets before model training, and the validation and test samples were not used for parameter updating. Model convergence was assessed based on the overall trends of the loss and IoU curves rather than a fixed number of iterations alone. a UNet Network Loss Function Curve Variation Chart. b UNet Average Intersection-Over-Union Ratio Variation Curve. The UNet model was used to predict samples from the validation set. Three samples were randomly selected from the validation set, and their segmentation results are shown in Fig. 13. As illustrated, both prominent stem fibers and leaf fibers are accurately detected, with scattered stem fibers remaining distinguishable. Although expanded fibers and leaf fibers exhibit high morphological similarity, their close intermingling or mutual occlusion may cause boundary adhesion and local category confusion. Nevertheless, the model effectively identifies the distinctive textural regions characteristic of expanded fibers. In practical applications, uniformity assessment of tobacco leaf piles prioritizes the macroscopic distribution patterns and proportions of various leaf types over the pixel-level distribution of individual leaf categories. Therefore, under continuous material conditions, this segmentation performance sufficiently meets the requirements for detecting mixing uniformity. Segmentation results of the UNet network on validation set samples. Improvements to the UNet model The Unet network performs end-to-end semantic segmentation for tobacco leaf type recognition using an encoder-decoder architecture. The encoder primarily extracts features through convolutions, but this approach is limited by incomplete feature capture and susceptibility to overfitting, resulting in inaccurate segmentation. Directly combining shallow and deep features during the decoding process places both feature layers on equal footing, leading to low segmentation accuracy and blurred edges. To address the limitations of UNet, the SE (Squeeze and Excitation) channel attention mechanism is integrated into the encoder, resulting in the SE-UNet architecture30. SE-UNet is a U-shaped network consisting of three main stages: encoding, skip connections, and decoding. The encoding path contains five stages, each including a convolutional layer, a Batch Normalization layer, a ReLU activation function, an SE attention module, and a 2 × 2 max-pooling layer. The decoding path also contains five stages, in which upsampling and feature concatenation are followed by two 3 × 3 convolutional layers. In this study, the integration of the SE module into U-Net is regarded as a task-oriented improvement rather than a fundamentally new network architecture. The purpose of introducing the SE module is to alleviate the difficulty of distinguishing visually similar tobacco components, especially expanded tobacco shreds and leaf tobacco shreds, which often share similar elongated morphology, texture, and overlapping spatial distribution. By explicitly modeling channel dependencies and adaptively recalibrating channel-wise feature responses, the SE module enhances feature channels related to discriminative texture, edge, color, and morphology, while suppressing less relevant or noisy channels. Therefore, the SE module helps the model focus on subtle inter-category differences among tobacco shred components, rather than merely improving feature extraction in a general sense31. To further clarify the calculation process of channel-wise weights in the SE attention module, the feature recalibration procedure is described as follows. Let the input feature map be , where H, W, and C represent the height, width, and number of channels, respectively. First, a squeeze operation is performed by global average pooling to convert each two-dimensional feature channel into a scalar descriptor: where represents the global response of the c-th channel. Then, the channel weight vector is generated through two fully connected layers and a sigmoid activation function: where , and are the parameters of the fully connected layers, denotes the ReLU function, and denotes the sigmoid function. Finally, each channel is recalibrated by multiplying the original feature map with the corresponding learned weight: Thus, the SE module adaptively enhances informative feature channels and suppresses less relevant channels. These weights are learned automatically during network training through backpropagation rather than being manually assigned (Fig. 14). Schematic Diagram of the SE Attention Mechanism Module. Visual comparison analysis of partition results To further evaluate the performance of the improved UNet model, it was compared with mainstream semantic segmentation models, including Segformer, PSPNet, and DeepLabV3. The comparison involved different backbone networks: VGG16, ResNet50, MobileNetV2, MobileNetV3, and Xception. The comparative analysis results are presented in Table 2. Additionally, four samples were randomly selected from the validation set to assess the proportion of different tobacco leaf types on the surface using various models. The results are shown in Fig. 15. As shown in Table 3 of the ablation study, the SE-UNet model achieved an average intersection-over-union (IoU) of 82.82% and a pixel accuracy of 80.88%, representing improvements of 2.77 and 1.89 percentage points, respectively, over the original UNet model. However, the average detection time per image increased by 0.72 seconds32. Consequently, the model demonstrated significantly improved discrimination capability for tobacco target regions, resulting in higher mIoU and pixel accuracy compared to the original model. However, the introduction of this structure added new computational nodes, inevitably increasing the number of floating-point operations and computational costs, which led to a higher average detection time per image. The Segformer model performed the worst among all models due to its reliance on self-attention mechanisms to model global dependencies. While it excels in long-range modeling, it lacks the strong prior assumptions about local spatial information inherent in convolutions33. Consequently, its ability to capture fine details during the feature extraction stage was weaker than that of UNet and DeepLabV3, resulting in lower mIoU and pixel accuracy (PA) metrics. The loss function curves for all partitioned model training processes are shown in Fig. 16. As shown in Table 2, the proposed SE-UNet achieved higher segmentation accuracy whilst maintaining a testing time comparable to that of the baseline model. Nevertheless, this study primarily quantifies computational cost in terms of inference time, and more detailed complexity metrics—such as the number of parameters, floating-point operations (FLOPs) and GPU memory consumption—have not been systematically analysed. Future research will take these metrics into account to provide a more comprehensive assessment of the model’s efficiency. Table 2Comparison of prediction results between the proposed model and other semantic segmentation models.Semantic segmentation model (backbone network)Average intersection ratio (MIOU%)Pixel accuracy (PA%)Testing time (Time/s)UNet (VGG16)80.0578.991.61PSPNet (MobileNetV2)69.2168.382.42PSPNet (MobileNetV3)65.4967.232.66DeepLabV3(MobileNetV2)61.9967.302.10DeepLabV3 (Xception)74.1276.611.94PSPNet (ResNet50)69.4370.982.54UNet-SE (VGG16)82.8280.882.33Segformer65.3660.202.74 Comparison of prediction results between the proposed model and other semantic segmentation models. Schematic Diagram of Tested Tobacco Shreds in Constructed Model. Table 3Analysis comparison table for ablation experiments.ModelSEAverage intersection ratio (MIOU%)Pixel accuracy (PA%)Testing time (Time/s)UNet80.0578.991.61UNet-SE√82.8280.882.33 Analysis comparison table for ablation experiments. Loss Function Curve Diagram for Different Models and Backbone Networks. As illustrated in Fig. 15, local misclassification remains in some segmentation results. Specifically, several tobacco shred regions are incorrectly classified as background, while some background regions are assigned to tobacco categories. This phenomenon may be attributed to the irregular morphology, random stacking, and mutual occlusion of tobacco shreds. In overlapping or adhesive areas, object boundaries are often ambiguous, and the visual features of small or shadowed shreds may become weak. In addition, shadows, residues, or color interference in the background may resemble tobacco components, resulting in incorrect pixel-level classification. Therefore, although the proposed model improves the overall segmentation performance, its robustness remains limited in regions with occlusion, adhesion, weak boundaries, and background interference. Extraction of tobacco shred area based on pixel statistics Visual area proportion is a key metric for evaluating spatial distribution differences among components in mixed materials. In this study, visual area proportion refers to the ratio of pixels corresponding to a specific tobacco leaf region within a semantic segmentation mask to the total effective area of the image. Since digital images are essentially matrices composed of discrete pixels carrying color (or grayscale) information, quantifying the distribution of projected areas for each component on a two-dimensional plane can be achieved by statistically analyzing the number of pixels assigned to different semantic labels34. Based on the segmentation results from the deep learning model, each pixel point in the mask image is assigned a specific category label. Due to the discretization characteristics of the image, a statistical mapping relationship exists between the total number of pixels for a specific category of tobacco shreds and the area proportion of each component. First, the calculation formula for the pixel statistic of Category i tobacco shreds (regions of different colors) is defined as follows: In the Eq. (13), W and H represent the width and height of the image, respectively. C(x,y) is the pixel color value at the coordinate (x,y), is the specific color index corresponding to tobacco shreds of Class i, When C(x,y) and match, the value is 1; otherwise, it is 0. Based on this, the visual area proportion of a certain type of tobacco shreds in the image can be calculated using the following formula: In the Eq. (14), represents the area proportion of tobacco shreds of category i, expressed as a percentage, and k denotes the total number of detected valid tobacco shred categories (excluding background), is the total sum of valid pixels after background removal. To further validate the effectiveness of this method, several random mask images were selected from the tobacco shred dataset for experimentation35,36. The pixel area distribution across regions within the tobacco shred images, the total number of valid pixels in each image, and the actual measured weight distribution of each tobacco shred type are summarized in Table 4. Table 4Statistical table of area proportion and actual error for each color region in tobacco shreds images.NumberPercentage of blue area (%)Percentage of green area (%)Percentage of red area(%)Percentage of yellow area(%)Effective total pixelsActual error (%)14726.4419.0541.9812.5332090815.1749749.8017.9418.0614.2071524613.84820.0021.1641.1737.6723436711.76850.0022.5077.500.0043545810.84900.0025.9825.1048.9234346716.35 Statistical table of area proportion and actual error for each color region in tobacco shreds images. Since tobacco shreds were randomly stacked on the conveyor belt, the upper layers obscured the lower layers, preventing the image capture system from acquiring pixel data from the obscured areas. This results in a visual area smaller than the actual physical weight. Additionally, different tobacco shred types (e.g., stem shreds, leaf shreds, expanded shreds) have varying mass per unit area, meaning simple area proportion calculations cannot fully correspond to weight proportions. As shown in Table 3, samples with relatively homogeneous compositions, such as Sample No. 85, exhibit lower actual errors. Conversely, samples with complex compositions or high proportions of specific components, like Sample No. 90, show errors as high as 16.35%. This suggests that tobacco leaf categories corresponding to different colors may present varying degrees of difficulty in edge segmentation or exhibit greater density variations, further amplifying cumulative calculation errors. It should be emphasized that the pixel-area proportion obtained from semantic segmentation represents the visible surface distribution of different tobacco components, rather than their exact physical mass proportion. Due to differences in density, thickness, curl degree, and stacking state among tobacco shreds, the relationship between visible area and actual weight is not strictly linear. In addition, overlap and occlusion may cause some components to be partially invisible, further increasing the deviation between area proportion and mass proportion. Therefore, the current pixel-area-based method should be regarded as a visual estimation basis for mixing uniformity rather than a direct substitute for mass measurement. For practical application, a calibration model should be introduced to map segmented pixel-area proportions to physical mass proportions. where represents the segmented pixel-area proportion of the i-th tobacco component, is the component-specific calibration coefficient related to density, thickness, and surface exposure, and is the estimated mass proportion. The calibration coefficient can be determined using manually weighed samples. In future work, a regression model can also be established by using segmented area proportions as inputs and measured mass proportions as reference outputs, so as to learn the mapping relationship between two-dimensional visual area and three-dimensional physical mass. Conclusion To address the challenge of obtaining a large volume of tobacco leaf images for semantic segmentation, this study explores tobacco leaf image generation techniques using generative adversarial networks (GANs). Comparative experiments were conducted with three high-resolution image generation networks—CycleGAN, WGAN, and DCGAN—under identical learning rate and momentum parameters. The experimental results show that, for tobacco leaf images, the CycleGAN network generates images that better preserve semantic information, most closely resemble the original images, and exhibit superior quality. A composite dataset of tobacco leaf images was created using both generated and original images. Image semantic segmentation testing was performed on the composite dataset using the UNet model. The results show that the enhanced UNet achieves a pixel accuracy of 80.88% and a mean intersection-over-union (mIoU) of 82.82% for tobacco leaf images, with a single-image processing time of 2.33 s. These results indicate that the proposed model achieved improved segmentation accuracy under the current experimental conditions. However, the single-image processing time of 2.33 s also suggests that further optimization is still required before deployment in high-speed industrial conveyor-belt scenarios. Before actual deployment, further validation using field images or other independent datasets should be carried out, and the inference efficiency should be optimized according to conveyor speed, image acquisition frequency, and allowable detection delay. Three image semantic segmentation models—DeepLabV3, PSPNet, and Segformer—were used for comparative experiments on composite tobacco leaf image datasets. DeepLabV3, PSPNet, and Segformer exhibited comparable processing times per image to the improved UNet. However, under the same experimental setting, the improved UNet achieved higher mIoU and pixel accuracy (PA) than DeepLabV3, PSPNet, and Segformer, with increases of 8.7%, 13.39%, 17.46%, and 4.27%, 9.9%, and 20.68%, respectively.The comparison results demonstrate that the improved UNet achieves the best overall segmentation performance. Although the proposed SE-UNet achieves an overall improvement in segmentation performance in terms of pixel accuracy and mIoU, current evaluations primarily focus on global segmentation metrics. Whilst these metrics reflect the overall pixel-level consistency between the predicted mask and the manually annotated labels, they fail to fully reveal the model’s specific recognition behaviour across different categories. In particular, due to differences in colour, texture, shape, size and spatial distribution, different tobacco components may present varying degrees of segmentation difficulty. The ultimate goal of researching real-time semantic segmentation methods for tobacco flake images is to enable rapid detection and assessment of surface uniformity on tobacco flake production lines, thereby establishing digital standards for practical tobacco manufacturing. However, uneven surfaces and significant stacking thickness in actual production present substantial challenges to detecting surface uniformity. Moving forward, we will explore the development of correlation models between the physical properties of different tobacco shreds and their segmented areas, while maintaining segmentation accuracy and real-time performance. We will continue to refine these models and evaluate them on different datasets to further validate their robustness and transferability. It should also be noted that the current study was mainly conducted under laboratory conditions, and the sensitivity of the model to field disturbances such as dust, conveyor-belt illumination variation, and industrial camera vibration has not yet been systematically evaluated. Future work will further investigate these field-condition factors to narrow the gap between laboratory validation and practical industrial deployment. In subsequent research, a more comprehensive diagnostic evaluation will be conducted by incorporating per-class IoU, class accuracy, recall, F1 score and pixel-level confusion matrix analysis. At the same time, a visual analysis of representative failure cases will be carried out to identify the primary sources of segmentation errors. Author contributions Xianhui Peng: Writing-original draft, review & editing. Shucai Wang: Methodology. Song Chen: Data curation, Meizhou Ding: Writing—review and editing, Xiaonan Ji: Supervision Quansheng Li: Funding acquisition Helan Zou Zhiwei Fan: Software. Funding Authors wish to thank the funding from the Key Laboratory Project on Tobacco Processing Forms in the Tobacco Industry (AW2024048). Data availability The datasets used in this study include original tobacco shred images, GAN-generated images, and corresponding annotation files. Representative original images, generated samples, annotation examples, and the core experimental code can be provided as supplementary materials or made available from the corresponding author upon reasonable request. Due to data management restrictions associated with the cooperative enterprise, full public release of all raw tobacco shred images may require permission from the data provider. Should you require access to all the data used in this study, please feel free to contact the first author, Meizhou Ding, or the corresponding author, Xianhui Peng, directly at any time. Meizhou Ding: pxh18645198824@163.com Xianhui Peng: 1,435,037,737@qq.com. Code availability The core experimental code used in this study, including the SE-UNet implementation, SE attention module, model training and prediction scripts, semantic mask processing scripts, and GAN-based image generation scripts, has been deposited in Zenodo. The code is available at: https://doi.org/10.5281/zenodo.20604426. Declarations Competing interests The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement The datasets used in this study include original tobacco shred images, GAN-generated images, and corresponding annotation files. Representative original images, generated samples, annotation examples, and the core experimental code can be provided as supplementary materials or made available from the corresponding author upon reasonable request. Due to data management restrictions associated with the cooperative enterprise, full public release of all raw tobacco shred images may require permission from the data provider. Should you require access to all the data used in this study, please feel free to contact the first author, Meizhou Ding, or the corresponding author, Xianhui Peng, directly at any time. Meizhou Ding: pxh18645198824@163.com Xianhui Peng: 1,435,037,737@qq.com. The core experimental code used in this study, including the SE-UNet implementation, SE attention module, model training and prediction scripts, semantic mask processing scripts, and GAN-based image generation scripts, has been deposited in Zenodo. The code is available at: https://doi.org/10.5281/zenodo.20604426.