Beyond a Single Mode: GAN Ensembles for Diverse Medical Data Generation

Lorenzo Tronchin, Tommy Löfstedt, Paolo Soda, Valerio Guarrasi · 2025

Read Beyond a Single Mode: GAN Ensembles for Diverse Medical Data Generation by Lorenzo Tronchin, Tommy Löfstedt, Paolo Soda, Valerio Guarrasi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-24258.txt.

Introduction

Lorenzo Tronchin, Tommy Löfstedt, Paolo Soda, and Valerio Guarrasi team up rival image generators for medical data in a 2025 paper. Synthetic medical images must be faithful, diverse, and cheap to make. Single GANs often collapse to a few modes. That collapse leaves real data patterns uncovered.

The team pick an ensemble of GANs by multi-objective tuning. The tuning balances fidelity against diversity across the set. Each chosen model adds a distinct contribution with little overlap. They tested 22 GAN designs across three medical datasets. Sampling training epochs gave 110 model variants to choose from. The ensembles made richer, more useful synthetic images.

Catalog record

CREATORLorenzo Tronchin, Tommy Löfstedt, Paolo Soda, Valerio Guarrasi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-24258

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Beyond a Single Mode: GAN Ensembles for Diverse Medical Data Generation Lorenzo Tronchin, Tommy Löfstedt, Paolo Soda, Valerio Guarrasi Abstract The advancement of generative AI, particularly in medical imaging, confronts the trilemma of ensuring high fidelity, diversity, and efficiency in synthetic data generation. While Generative Adversarial Networks (GANs) have shown promise across various applications, they still face challenges like mode collapse and insufficient coverage of real data distributions. This work explores the use of GAN ensembles to overcome these limitations, specifically in the context of medical imaging. By solving a multi-objective optimisation problem that balances fidelity and diversity, we propose a method for selecting an optimal ensemble of GANs tailored for medical data. The selected ensemble is capable of generating diverse synthetic medical images that are representative of true data distributions and computationally efficient. Each model in the ensemble brings a unique contribution, ensuring minimal redundancy. We conducted a comprehensive evaluation using three distinct medical datasets, testing 22 different GAN architectures with various loss functions and regularisation techniques. By sampling models at different training epochs, we crafted 110 unique configurations. The results highlight the capability of GAN ensembles to enhance the quality and utility of synthetic medical images, thereby improving the efficacy of downstream tasks such as diagnostic modelling. 1 Introduction Generative Adversarial Networks (GANs) [ 1 ] have proven effective in various medical imaging tasks [ 2 , 3 ] . However, synthetic data from GANs may fail due to mode collapse, memorisation of training data, and poor coverage of the real manifold [ 4 , 5 ] . The generative learning trilemma [ 5 ] includes three crucial requirements for practical applications: high-quality sampling (fidelity), mode coverage (diversity), and fast, computationally inexpensive sampling. Fidelity measures how well-generated samples resemble real ones [ 6 ] , while diversity assesses the coverage of the variability in real samples [ 6 ] . GANs generate high-quality samples rapidly but have poor mode coverage [ 7 ] , often missing rare diseases or anomalies in medical imaging. Precision and variability in synthetic data are critical in medical imaging, where diverse pathologies and patient-specific variations must be represented. This is essential for basic research, clinical training, and algorithmic validation, where realistic, diverse datasets develop robust diagnostic tools. Therefore, an ensemble of GANs can generate high-quality, diverse medical images, enhancing machine learning models’ ability to generalise across unseen conditions [ 8 , 9 ] . A large body of literature [ 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ] has investigated GAN ensembles to solve diversity issues. Approaches include boosting strategies [ 10 , 11 ] , multiple generators [ 12 , 13 ] , discriminators [ 14 ] , or dropout [ 15 ] . Eilertsen et al. [ 16 ] and Van Breugel et al. [ 17 ] evaluated ensemble performance on various datasets, showing improved results. However, they focused on two architectures, DCGAN [ 20 ] and PG-GAN [ 21 ] , without combining different architectures or sampling at various training steps. Huang et al. [ 18 ] and Wang et al. [ 19 ] saved model parameters at different local minima but did not explore different architectures. These approaches typically focus solely on generative model performance, neglecting the impact of synthetic data on downstream tasks. Our approach leverages pre-existing GANs, integrating multiple architectures and sampling at different training iterations to create a robust ensemble capturing real data complexity. This work explores building a deep ensemble of GANs to recover as many training data modes as possible. We investigated GANs focusing on: (i) training different GANs (e.g., architecture, adversarial loss, regularisation) and (ii) sampling GANs at different epochs within a single model’s training. We hypothesised that the min-max game dynamics [ 1 ] continuously change the generator network, allowing an ensemble of GANs based on the same initialisation but sampled at different epochs. Our method is agnostic to the GANs included in the search space, helping answer the questions: “Which GAN should we use?” and “When should we stop adversarial training?” . We considered 22 GANs, covering a wide range of architectures [ 22 , 23 , 24 , 25 ] , conditioning methods [ 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ] , adversarial losses and regularisation [ 23 , 35 , 36 ] , and differentiable augmentation modules [ 37 , 38 ] . We aimed to select a combination of GANs that maximised quality and minimised overlap, preventing mode collapse and redundancy. This is the first extensive study on constructing GAN ensembles in the medical domain. Our ensemble selection method balances high-fidelity sample generation and spans the variability of real data with the smallest number of GANs necessary to ensure unique contributions. Our main contributions are: 1. Introducing a new method to build GAN ensembles agnostic to the GAN model and training iterations, balancing data representation and diversity while minimizing overlap and redundancy. 2. Systematically analysing the impact of different backbone architectures for embedding extraction in GAN evaluation, including variations of ImageNet-pre-trained models, fine-tuned to the medical domain, and testing unsupervised backbones. 3. Reporting and analyzing extensive tests across three distinct medical datasets, considering the effect of the proposed ensemble method on downstream tasks and from fidelity and diversity perspectives. The code for our experiments is available at: https://github.com/ltronchin/GAN-Ensembles . 2 Methods Figure 1 : An illustration of the manifold occupied by real data, R , alongside the manifold covered by various GAN configurations, S_{i} . Single : Showcases three scenarios: S_{i} entirely within the R , indicating high fidelity; S_{i} partially overlapping with the R representing medium fidelity; S_{i} entirely outside the R suggesting low fidelity. Single GAN selection inherently has a low diversity due to limited real data coverage. Naive (A) : Depicts the collective space covered by all GANs, encompassing areas both inside and outside the R . This approach results in medium fidelity and high diversity. G^{*} : Demonstrates the space covered by an ensemble of GANs selected through the proposed optimisation method, which aims to maximise coverage of the R to ensure diversity discharging GAN outside the R , to ensure fidelity, using as few GANs as possible. To select an appropriate GAN for data generation is a central decision that substantially impacts the quality and utility of the generated data, especially in the field of medical imaging where accuracy and variety in the data are paramount [ 39 ] . A straightforward approach would be to select a single GAN from a pool of available models. However, this approach, while simple, often leads to sub-optimal results due to inherent trade-offs in fidelity and diversity inherent in any single GAN model. A single GAN, depending on its architecture and training, may exhibit varying degrees of alignment with the real data space. Some may generate data with high fidelity, others might partially overlap with the real data space offering a balance, while some might completely miss the mark, generating data with low fidelity. Moreover, relying on a single GAN inherently limits the diversity of the generated data, which is particularly detrimental in

Read the complete text