FederatedFactory: Generative One-Shot Learning for Extremely Non-IID Distributed Scenarios

Andrea Moleri *, Christian Internò *, Ali Raza, Markus Olhofer, David Klindt, Fabio Stella †, Barbara Hammer † · 2026

Read FederatedFactory: Generative One-Shot Learning for Extremely Non-IID Distributed Scenarios by Andrea Moleri *, Christian Internò *, Ali Raza, Markus Olhofer, David Klindt, Fabio Stella †, Barbara Hammer † (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16370.txt.

Introduction

Moleri and co-authors rethink federated learning for sites with disjoint label sets. Standard weight averaging fails when local training paths conflict. Their FederatedFactory swaps generative modules instead of model weights. One communication round shares those modules. Each site can then synthesize a balanced dataset from scratch. Tests on medical image benchmarks matched centralized training. On CIFAR-10, accuracy jumped from 11.36% to 90.57% under extreme splits.

Catalog record

CREATORAndrea Moleri *, Christian Internò *, Ali Raza, Markus Olhofer, David Klindt, Fabio Stella †, Barbara Hammer †
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60417
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.16370

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FederatedFactory: Generative One-Shot Learning for Extremely Non-IID Distributed Scenarios Andrea Moleri *, Christian Internò *, Ali Raza, Markus Olhofer, David Klindt, Fabio Stella †, Barbara Hammer † Abstract Federated Learning ( FL ) enables distributed optimization without compromising data sovereignty. Yet, where local label distributions are mutually exclusive, standard weight aggregation fails due to conflicting optimization trajectories. Often, FL methods rely on pretrained foundation models, introducing unrealistic assumptions. We introduce FederatedFactory , a zero-dependency framework that inverts the unit of federation from discriminative parameters to generative priors. By exchanging generative modules in a single communication round, our architecture supports ex nihilo synthesis of universally class-balanced datasets, eliminating gradient conflict and external prior bias entirely. Evaluations across diverse medical imagery benchmarks, including MedMNIST and ISIC2019 , demonstrate that our approach recovers centralized upper-bound performance. Under pathological heterogeneity, it lifts baseline accuracy from a collapsed 11.36 % 11.36\% to 90.57 % 90.57\% on CIFAR-10 and restores ISIC2019 AUROC to 90.57 % 90.57\% . Additionally, this framework facilitates exact modular unlearning through the deterministic deletion of specific generative modules. † † footnotetext: Correspondence: † † footnotetext: Code : https://github.com/andreamoleri/FederatedFactory * Equal contribution. † Co-advised. 1 Introduction FL provides a decentralized framework to optimize statistical models across K distinct clients while strictly preserving local data sovereignty [ 36 ] . However, the theoretical convergence guarantees of traditional FL rely on independently and identically distributed (IID) data, an assumption violated by extreme statistical heterogeneity in real-world applications like multi-institutional medical imaging [ 41 , 40 ] . The structural fragility of standard parameter aggregation is exposed most aggressively under pathological label skew [ 21 , 25 ] , where each client holds samples from only a few, or even just one, class of data. Specifically, we define and analyze the extreme single-class silo regime, where each client k\in\{1,\dots,K\} possesses a local dataset \mathcal{D}_{k} that contains exclusively one class, yielding mutually disjoint label supports \mathcal{Y}_{k} . Optimizing shared discriminative parameters \mathbf{w} fails unconditionally in this setting. Lacking the counterfactual data necessary to form an inter-class decision boundary, the local empirical risks \mathcal{L}_{k}(\mathbf{w}) produces optimization trajectories that interfere with one another. To avoid communication bottlenecks and conflicts in gradient trajectories of iterative FL , One-Shot Federated Learning ( OSFL ) attempts to aggregate knowledge in a single round of communication [ 14 , 32 ] . Recent generative OSFL methods synthesize datasets using pretrained Foundation Models ( FM s) as universal priors [ 2 , 49 ] . While effective for general domains, this dependency is unreliable for specialized applications such as medical diagnosis [ 50 , 35 , 23 ] . By projecting a rare target distribution onto an external representation space, these methods effectively discard the off-manifold feature components \mathbf{x}_{\perp} that constitute the true diagnostic signal [ 16 ] . We introduce FederatedFactory , a framework that aims to recover the centralized upper-bound performance under the extreme single-class silo assumption via a novel, strictly zero-dependency Generative OSFL architecture. We invert the unit of federation from discriminative parameter matrices \mathbf{W} to localized generative prior parameters \boldsymbol{\theta}_{k} . Each client independently trains and transmits a generative Factory G_{\boldsymbol{\theta}_{k}} exactly once, supporting both a centralized architecture (in consortiums where a central aggregator can be trusted) and a fully decentralized Peer-to-Peer (P2P) network mesh (in consortiums in which a central aggregator cannot be trusted). Specifically to our work, we use the computationally efficient EDM2 diffusion model [ 28 , 29 ] . The server concatenates these decoupled generative structures into a universal prior, synthesizing class-balanced datasets ex nihilo from a standard latent space \mathcal{Z} . By relying exclusively on models trained directly on the true localized data distributions, FederatedFactory explicitly avoids projection errors and eliminates the external prior bias inherent to FM -reliant methods. Hypothesis : We hypothesize that shifting the unit of federation from discriminative parameters to generative priors enables OSFL to achieve performance comparable to centralized baselines in pathologically non-IID scenarios, strictly without raw data exchange or reliance on external FM s. We summarize our contributions as follows: • Robustness to Extreme Heterogeneity: FederatedFactory recovers centralized performance under pathological single-class silos where standard methods collapse (e.g, CIFAR10 [ 31 ] Accuracy 11.36\%\rightarrow 90.57\% , and ISIC2019 [ 43 ] AUROC 47.31\%\rightarrow 90.57\% ). • Zero-Dependency Federation: By decoupling data synthesis from external pre-trained FM s, our protocol relies exclusively on localized priors. As formalized in Theorem 3.1 , this bounds the global risk strictly by the local generative error ( \bar{\epsilon} ). • One-Shot Communication Efficiency: FederatedFactory relies on a single communication round ( \mathcal{C}_{\mathrm{rounds}}=1 ), avoiding expensive iterations. • Modular Machine Unlearning: The framework guarantees exact modular unlearning [ 10 ] . Removing a client requires only the structural deletion of their corresponding parameter coordinates ( \boldsymbol{\Gamma}_{:,k}\leftarrow\emptyset ). 2 Related Work Optimization Under Distribution Shift. Federated methods (e.g., FedDyn [ 1 ] , FedProx [ 33 ] , SCAFFOLD [ 27 ] ) handle statistical heterogeneity by bounding local updates. However, they fail when local label sets are entirely disjoint ( \mathcal{Y}_{i}\cap\mathcal{Y}_{j}=\emptyset ). Despite strong collaboration incentives in this extreme regime (e.g., isolated hospitals needing generalized models), disjoint labels cause actively divergent gradients [ 51 ] . Without overlapping classes to anchor a shared feature space, proximal constraints cannot align these conflicting trajectories to form coherent inter-class decision boundaries [ 33 ] . One-Shot FL and External Priors. OSFL bypasses iterative gradient conflicts via a single upstream transmission. Recent diffusion frameworks (e.g., FedLMG [ 46 ] , FedSDE [ 38 ] , FedDEO [ 47 ] ) synthesize datasets centrally but rely heavily on pretrained FM s (e.g., CLIP [ 39 ] , Stable Diffusion [ 4 ] ). Relying on an external manifold \mathcal{M}_{\mathrm{FM}} inherently projects away rare features \mathbf{x}_{\perp} ; in medical imaging, this risks texture bias and semantic hallucination [ 11 ] . We eliminate this FM dependency by transmitting locally trained generative priors \boldsymbol{\theta}_{k} , achieving ex nihilo one-shot synthesis exclusively from true local distributions. Other approaches collaboratively train generative models to construct synthetic datasets. Notably, the Diffusion Federated Dataset [ 15 ] models generation as cooperative sampling from diffusion energy-based models. While analytically robust, its reliance on iterative communication rounds increases communication overhead. 3 Background and Mathematical Preliminaries Federated Learning under Single-Class Silo Regime. In standard FL model aggregation, a decentralized network of K clients aims to aggregate local empirical risks over shared discriminative parameters \mathbf{w}\in\mathbb{R}^{m} in order to effectively approximate the global optimum. The de facto standard FL approach [ 36 ] formally attempts to minimize the global objective function \min_{\mathbf{w}}\mathcal{L}(\mathbf{w}):=\frac{1}{K}\sum_{k=1}^{K}\mathcal{L}_{k}(\mathbf{w}) , where \mathcal{L}_{k}(\mathbf{w})=\mathbb{E}_{(x,y)\sim p_{k}}[\ell(\mathbf{w};x,y)] is the local objective representing the expected loss over the true data distribution p_{k} at client k , and \ell denotes the per-sample loss function. While aggregators such as FedAvg [ 36 ] succeed when the local datasets \mathcal{D}_{k} are IID, this paradigm collapses under severe non-IID settings. We parameterize label skewness via a Dirichlet distribution \mathrm{Dir}_{C}(\alpha) . As \alpha\to 0 in a cross-silo setting with K clients and C global classes, we reach the pathological Single-Class Silo regime (Figure 1 c). Here, each client dataset \mathcal{D}_{k} contains exactly one unique class, yielding strictly disjoint label sets ( \mathcal{Y}_{i}\cap\mathcal{Y}_{j}=\emptyset for i\neq j ). Consequently, local

Read the complete text