FedFusion: Federated Learning with Diversity- and Cluster-Aware Encoders for Robust Adaptation under Label Scarcity

Ferdinand Kahenga † , Antoine Bagula † , Patrick Sello † , and Sajal K. Das ‡ {\ddagger} · 2025

Read FedFusion: Federated Learning with Diversity- and Cluster-Aware Encoders for Robust Adaptation under Label Scarcity by Ferdinand Kahenga † , Antoine Bagula † , Patrick Sello † , and Sajal K. Das ‡ {\ddagger} (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-19220.txt.

Introduction

Ferdinand Kahenga and three co-authors return to federated learning, where sites train a shared model without sharing raw data, this time under label scarcity. Real clients differ in their feature spaces, hold skewed data, and can label only a few examples. Their framework, FedFusion, gives each client a personalized encoder shaped to its own data. Encoders come in diversity- and cluster-aware variants that keep client models from collapsing into sameness.

Well-labeled teacher clients guide the others through confidence-filtered pseudo-labels, labels the model assigns to its own confident guesses. Classifiers are coupled by similarity, with optional cluster averaging, so data-rich sites cannot dominate and minority clients fare better. A frugal labeling pipeline cuts annotation needs through self-supervised pretraining and selective fine-tuning. Across tabular and imaging tests, FedFusion beat leading baselines in accuracy, robustness, and fairness at similar cost.

Catalog record

CREATORFerdinand Kahenga † , Antoine Bagula † , Patrick Sello † , and Sajal K. Das ‡ {\ddagger}
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-19220

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FedFusion: Federated Learning with Diversity- and Cluster-Aware Encoders for Robust Adaptation under Label Scarcity Ferdinand Kahenga † , Antoine Bagula † , Patrick Sello † , and Sajal K. Das ‡ {\ddagger} Abstract Federated learning in practice must contend with heterogeneous feature spaces, severe non-IID data, and scarce labels across clients. We present FedFusion ( FedFusion ), a federated transfer-learning framework that unifies domain adaptation and frugal labelling with diversity-/cluster-aware encoders (DivEn, DivEn-mix, DivEn-c). Labelled teacher clients guide learner clients via confidence-filtered pseudo-labels and domain-adaptive transfer, while clients maintain personalised encoders tailored to local data. To preserve global coherence under heterogeneity, FedFusion employs similarity-weighted classifier coupling (with optional cluster-wise averaging), mitigating dominance by data-rich sites and improving minority-client performance. The frugal-labelling pipeline combines self-/semi-supervised pretext training with selective fine-tuning, reducing annotation demands without sharing raw data. Across tabular and imaging benchmarks under IID, non-IID, and label-scarce regimes, FedFusion consistently outperforms state-of-the-art baselines in accuracy, robustness, and fairness while maintaining comparable communication and computation budgets. These results show that harmonising personalisation, domain adaptation, and label efficiency is an effective recipe for robust federated learning under real-world constraints. {\dagger} Department of Computer Science, University of the Western Cape, Cape Town, South Africa {\ddagger} Department of Computer Science, Missouri University of Science and Technology, Rolla, Missouri, USA E-mail: [email protected],[email protected],[email protected], [email protected] I Introduction Federated learning (FL) enables decentralised model training while keeping raw data local, aligning with data minimisation and privacy requirements [ 1 ] . This paradigm has spurred a broad class of optimisation strategies across academia and industry [ 2 , 3 , 4 ] , with particular relevance to privacy-sensitive sectors such as healthcare [ 5 , 6 , 7 ] . In parallel, transfer learning (TL) can be combined with FL—often termed federated transfer learning—to let models trained in one domain generalise to another, thereby reducing reliance on large labelled datasets [ 8 ] . Despite this progress, FL degrades in practice when client data are non-IID [ 9 ] , and global models can underperform under severe heterogeneity [ 10 ] . Two patterns are especially common: (i) feature-space heterogeneity , where clients expose non-aligned and variably sized feature sets; and (ii) label scarcity under domain shift , where clients have fully/partially/unlabelled data from different domains. Figure 1 illustrates two typical scenarios: (a) clients with non-aligned, variably sized feature sets (Fig. 1(a) ); and (b) mixed label availability across domains (Fig. 1(b) ). (a) Uneven feature distribution: different features across clients. (b) Data distribution: nodes’ data are fully, partially, or not labelled. Fig. 1 : Federated diversity learning: Colour differences indicate variation in feature sets and data distributions. In the configuration of Fig. 1(b) , clients may possess fully labelled, partially labelled, or entirely unlabelled data originating from diverse domains, yet all contribute to the same classification task. As an overview, these examples highlight the dual sources of heterogeneity—features and labels—encountered in practice. These settings call for diversity-aware personalisation alongside mechanisms that exploit unlabelled data without compromising privacy. We propose FedFusion , a federated transfer-learning framework that integrates domain adaptation with frugal labelling in a multi-teacher/multi-learner setting. FedFusion addresses: (i) heterogeneous feature partitions in tabular data (common in healthcare, where sources differ despite shared tasks); and (ii) cross-domain variability in images that impairs generalisation. Concretely, FedFusion combines personalised encoders with similarity-weighted classifier sharing (DivEn/DivEn-mix) and a cluster-aware variant (DivEn-c) to share structure among similar clients while preserving local specialisation. It further couples a two-step pipeline—self-supervised pretext training followed by confidence-filtered pseudo-label fine-tuning—to leverage unlabelled data under domain shift. Building on the PRIS line of work, FedFusion unifies federated learning, transfer learning, and domain adaptation to support both tabular and imaging tasks while reducing dependence on extensive labels [ 11 ] . The framework explicitly targets realistic federated deployments (e.g., Gauteng Department of Health) where data are fragmented, sparsely labelled, and domain-diverse. This article substantially extends our preliminary FedDAFL [ 12 ] work by introducing: • Personalised diversity/cluster-aware encoders (DivEn/DivEn-mix/DivEn-c) with similarity-weighted classifier aggregation to handle heterogeneous feature spaces. • Integrated two-step domain-adaptation with frugal labelling (pretext \rightarrow pseudo-labels), including confidence filtering and encoder-drift control for unlabelled clients. • Broader evaluation across non-IID, label-scarce tabular (heterogeneous features) and imaging (domain shift) benchmarks, with expanded ablations, fairness metrics, and efficiency analyses. The remainder of the paper is organised as follows: Section II reviews related work; Section III presents the FedFusion methodology (DivEn/DivEn-mix/DivEn-c) and the two-step frugal-label domain-adaptation pipeline); Section IV details the algorithms and training protocols; Section V reports performance evaluation across tabular feature-distribution settings and imaging domain-adaptation benchmarks; Section VI provides performance justification; and Section VII concludes and outlines directions for future work. II Related work Federated Learning [ 13 ] trains global models without sharing local data, thereby preserving user privacy. Despite progress on data heterogeneity and communication constraints, FL still struggles with uneven feature distributions and unlabeled data, limiting applicability in domains such as healthcare. We summarise closely related work in personalised federated learning and domain adaptation within FL . II-A Personalised Federated Learning Personalised FL develops methods that allow clients to learn individualised models while leveraging global knowledge. Zhou et al. [ 14 ] propose M-Fed , a multi-task encoder–decoder framework: clients on distinct tasks maintain local encoders/classifiers; encoders for the same task are aggregated, while a global encoder provides shared abstract knowledge with a regulariser controlling its influence during local updates. Gain et al. [ 15 ] introduce FedFeat+ , which decomposes models into a feature extractor and a classifier. Clients transmit both components along with differentially private, noise-added features; the server aggregates extractors/classifiers and retrains only the global classifier on noisy labelled features, improving robustness without exposing raw data. FedPAC [ 16 ] employs a personalised strategy using a regulariser of the form |f(x)-c| , where f(x) is the local embedding and c is the global class centroid; clients collaborate to compute centroids and weight classifiers, yielding class-aware updates while retaining personalisation. Complementary to these, correlation-alignment methods (e.g., [ 17 ] ) harmonise intermediate representations prior to aggregation by penalising divergence of correlation matrices from a global target, thereby encouraging feature-space consistency across clients. These directions illustrate an evolution from multi-task collaboration to privacy-preserving model construction and personalised optimisation, via encoder sharing, feature-alignment regularisation, and classifier aggregation. However, most approaches either assume uniform model architectures across clients or primarily address task divergence . They do not explicitly tackle unequal and stochastic feature-dimension distributions across clients in tabular settings, nor do they natively account for mixed labelled/partially labelled/unlabelled regimes. Our approach leverages latent client commonalities even when feature spaces differ, promoting effective collaboration while preserving local structural diversity. II-B Domain Adaptation Transfer learning seeks to improve learning in a target task by transferring knowledge from a related source task [ 18 ] . Within deep transfer learning, two primary cases are common: fine-tuning and domain adaptation. Traditionally, domain adaptation has been studied in centralised settings with joint access to source and target data. Within FL, unsupervised federated domain adaptation methods address decentralised data and label scarcity. FADA [ 19 ] and FedKA [ 20 ] represent distinct strategies. FADA pursues adversarial alignment in settings where both labelled source and unlabelled target data remain inaccessible to each

Read the complete text