SAGE-GAN: Towards Realistic and Robust Segmentation of Spatially Ordered Nanoparticles via Attention-Guided GANs
Anindya Pal 1, Varun Ajith 1 ⋆, Saumik Bhattacharya 2, Sayantari Ghosh 1 · 2026
Read SAGE-GAN: Towards Realistic and Robust Segmentation of Spatially Ordered Nanoparticles via Attention-Guided GANs by Anindya Pal 1, Varun Ajith 1 ⋆, Saumik Bhattacharya 2, Sayantari Ghosh 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03637.txt.
Introduction
Pal and colleagues build SAGE-GAN, a two-step system for cutting out nanoparticles in EM images and making lifelike fake image-mask pairs. First, a self-focus U-Net learns to cut out nanoparticle features from real images. It focuses on physical and shape details while skipping background and noise. Then the trained U-Net sits inside a cycle-true GAN framework, making fake pairs that mirror learned shape patterns and augment training data without human input.
Catalog record
| CREATOR | Anindya Pal 1, Varun Ajith 1 ⋆, Saumik Bhattacharya 2, Sayantari Ghosh 1 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40022 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.03637
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SAGE-GAN: Towards Realistic and Robust Segmentation of Spatially Ordered Nanoparticles via Attention-Guided GANs Anindya Pal 1, Varun Ajith 1 ⋆, Saumik Bhattacharya 2, Sayantari Ghosh 1 Abstract Precise analysis of nanoparticles for characterization in electron microscopy images is essential for advancing nanomaterial development. Yet it remains challenging due to the time-consuming nature of manual methods and the shortcomings of traditional automated segmentation techniques, especially when dealing with complex shapes and imaging artifacts. While conventional methods yield promising results, they depend on a large volume of labeled training data, which is both difficult to acquire and highly time-consuming to generate. In order to overcome these challenges, we have developed a two-step solution: Firstly, our system learns to segment the key features of nanoparticles from a dataset of real images using a self-attention driven U-Net architecture that focuses on important physical and morphological details while ignoring background features and noise. Secondly, this trained Attention U-Net is embedded in a cycle-consistent generative adversarial network (CycleGAN) framework, inspired by the cGAN-Seg model introduced by Abzargar et al. This integration allows for the creation of highly realistic synthetic electron microscopy image-mask pairs that naturally reflect the structural patterns learned by the Attention U-Net. Consequently, the model can accurately detect features in a diverse array of real-world nanoparticle images and autonomously augment the training dataset without requiring human input. Cycle consistency enforces a direct correspondence between synthetic images and ground-truth masks, ensuring realistic features, which is crucial for accurate segmentation training. Visualizations of attention overlays on the validation set, comparison of real images with their corresponding attention maps, show that the model achieves precise segmentation by concentrating on morphologically relevant regions. Unlike simpler simulation methods or standard image generators, our approach ensures that the synthetic images maintain accurate feature and shapes. Experimental validation on real electron microscopy images demonstrated excellent segmentation accuracy and robustness, particularly under data-limited conditions and complicated morphologies. † † institute: National Institute of Technology Durgapur, Department of Physics , and Indian Institute of Technology Kharagpur 1 Introduction Nanoparticles, typically defined as particles with at least one dimension between 1 and 100 nanometers exhibit distinctive physical, chemical, and biological properties that set them apart from their bulk counterparts. These unique characteristics stem from their high surface-to-volume ratio, quantum effects, and size-dependent optical behaviors. Such nanoscale properties, including enhanced reactivity and tunable optical responses, form the basis for their diverse applications in fields like medicine, energy storage, sensing, drug delivery, and materials science 4 ; 12 ; 25 ; 22 ; 28 . As research continues, nanoparticles are expected to lead major advances in fields like clean energy and personalized healthcare. Their growing use in different industries shows how important they have become for technological progress and daily life. Accurate characterization is essential for understanding and leveraging the unique properties of nanoparticles, such as their size, shape, surface chemistry, crystallinity, and degree of aggregation. However, realizing their full potential requires precise structural characterizations. These characteristics directly influence the behavior and performance of nanomaterials in various applications, ranging from cosmetics, textiles, food and beverages to medicine and healthcare. Without precise characterization, it becomes challenging to control or predict these behaviors, ultimately limiting the development of high-performance nanomaterials tailored for specific industrial, biomedical, or energy applications. Among various techniques, electron microscopy (EM) methods such as Scanning Electron Microscopy (SEM) 21 and Transmission Electron Microscopy (TEM) 2 remain standard due to their ability to capture fine structural details, including particle size, shape, and spatial distribution. However, manual analysis of these images is labor intensive, subjective, and error-prone, particularly for complex nanostructures with overlapping particles or heterogeneous morphologies. The growing complexity of nanomaterials and the increasing resolution of microscopy techniques further amplify the need for robust computational tools that can reliably extract quantitative data from images while minimizing human intervention. These challenges create a need for automated image analysis techniques that can deliver fast, objective, and reproducible results. With the advancement of machine learning-driven image analyses through the last decade, automated image segmentation has emerged as a critical tool for quantitative nanoparticle analysis. Conventional approaches, including thresholding 7 , edge detection 13 and watershed algorithms 10 , struggle with challenges such as low contrast, imaging artifacts, and agglomerated particles. Recent advances in deep learning, particularly U-Net-based architectures 18 , have improved segmentation accuracy by learning hierarchical features directly from the data. For example, Mask R-CNN 15 and lightweight networks such as NSNet 23 demonstrate promising results in separating nanoparticles. However, these models require large, diverse, and meticulously annotated datasets to generalize effectively. This remians as a major bottleneck in EM imaging, where acquiring and labeling high-resolution images is time-consuming, costly, and domain specific. To address these challenges, the use of synthetic data generation has recently gained significant attention. Liang et al. 11 augmented real TEM datasets with synthetic image-mask pairs, achieving optimal segmentation with only 15% authentic data. Their core idea was to augment the real dataset with the incorporation of software-generated image mask pairs. However, their model required critical human supervision and guidance throughout the creation of synthetic images. The model also failed to incorporate crucial physical details present in the actual dataset. Shah et al. 19 applied DeepLIFT analysis 20 to study 2D materials like graphene, highlighting several distinct challenges. These included inconsistent sizes and shapes, varying contrast ratios, complex background features, and a shallow depth of field. The authors noted that the model struggled to replicate the natural variations found in real microscopy images. Mill et al. 14 proposed a segmentation workflow for complex nanoparticle structures in high-resolution microscopy images using a semi-automated synthetic data pipeline based on realistic rendering using software such as Blender 5 . Their approach was able to generate an unlimited number of synthetic data with the corresponding ground truth mask, allowing training of convolutional neural networks(CNNs) 17 . However, these methods were based on manual template design or rigid physical models, which lacked the diversity and realism needed to capture the full complexity of experimental electron microscopy images. This trade-off between scalability and realism restricted their utility for analyzing heterogeneous or irregular nanostructures. Furthermore, vanilla generative adversarial networks (GANs) 6 , while capable of producing realistic synthetic images, often failed to preserve structural fidelity to ground-truth masks, limiting their value for training segmentation models. In this paper, we have introduced a two-phase architecture (Fig- 1 ) that integrates an attention-guided 26 U-Net with a cycle-consistent generative adversarial network (CycleGAN) 29 inspired by the cGAN-Seg model 27 which was introduced to address data scarcity without sacrificing morphological accuracy. In Phase 1 , the Attention U-Net 16 is first trained on the available real SEM images and their corresponding manually annotated masks. This stage focuses on learning segmentation by leveraging spatial attention gates to prioritize regions of interest (e.g., nanoparticle boundaries and spatial features). The trained model captures domain-specific structural features, such as size, shape, and spatial distribution, while mitigating noise and artifacts through its attention mechanism. Next, in Phase 2 , the trained Attention U-Net 16 is integrated into a CycleGAN 29 framework to enforce cycle consistency. By integrating the trained U-Net with the CycleGAN 29 framework, the generator learns to produce synthetic images that align with the structural priors encoded