Client-Conditional Federated Learning via Local Training Data Statistics

Rickard Brännvall · 2026

Read Client-Conditional Federated Learning via Local Training Data Statistics by Rickard Brännvall (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11307.txt.

Introduction

Rickard Brännvall conditions a single global federated model on each client's own training data statistics, sidestepping the usual trade between ignoring client differences and paying for clusters or per-client models. Each client computes principal component statistics of its local data locally, so the conditioning costs zero extra communication. Across ninety-seven configurations spanning four heterogeneity types, four datasets, and seven baselines, the method matches an oracle knowing the true clusters. It beats the oracle under combined heterogeneity and stays uniquely robust when data are sparse.

Catalog record

CREATORRickard Brännvall
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44752
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.11307

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Client-Conditional Federated Learning via Local Training Data Statistics Rickard Brännvall Abstract Federated learning (FL) under data heterogeneity remains challenging: existing methods either ignore client differences (FedAvg), require costly cluster discovery (IFCA), or maintain per-client models (Ditto). All degrade when data is sparse or heterogeneity is multi-dimensional. We propose conditioning a single global model on locally-computed PCA statistics of each client’s training data, requiring zero additional communication. Evaluating across 97 configurations spanning four heterogeneity types (label shift, covariate shift, concept shift, and combined heterogeneity), four datasets (MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100), and seven FL baseline methods, we find that our method matches the Oracle baseline—which knows true cluster assignments—across all settings, surpasses it by 1–6% on combined heterogeneity where continuous statistics are richer than discrete cluster identifiers, and is uniquely sparsity-robust among all tested methods. I Introduction Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data [ 16 , 7 ] . A core promise of FL is that collaboration improves upon what each client could achieve alone: by pooling gradient information across many devices, the shared model can generalize better than any single client’s local model. However, this promise breaks down under data heterogeneity —the realistic situation where different clients have different data distributions [ 23 , 6 ] . Heterogeneity manifests in multiple forms. Under label shift (E1), different clients observe different subsets of classes, so averaging their gradients toward conflicting class boundaries destroys information. Under covariate shift (E2), clients observe different feature distributions even for the same task, causing the global model to average over incompatible input statistics. Under concept shift (E3), clients may apply different classification rules to the same inputs, making their objectives fundamentally incompatible. When these forms combine (E4), the challenge is compounded. The standard federated averaging algorithm (FedAvg) [ 16 ] produces a single global model that compromises across all local objectives. Under severe heterogeneity, this compromise can be catastrophic: our experiments show FedAvg accuracy dropping from 73.5% to 17.2% on CIFAR-10 as the number of distinct client clusters increases from 2 to 10. Existing approaches to heterogeneous FL fall into three categories, each with significant limitations: • Clustered FL methods (IFCA [ 5 ] , CFL [ 19 ] ) attempt to discover groups of similar clients and train separate models per group. They require iterative cluster estimation, which is unreliable under data sparsity, disclose additional information, and assume that a small number of discrete clusters fully capture the heterogeneity structure. • Personalized FL methods (Ditto [ 10 ] , Per-FedAvg [ 3 ] ) maintain per-client model components, increasing communication and storage with the number of clients. • Decentralized FL methods (DAC [ 13 ] ) use peer-to-peer communication weighted by model similarity, but still require iterative discovery of the collaboration structure. All three categories share a common weakness: they discover client similarity from model behavior—gradient directions, loss values, or weight distances—which becomes unreliable when individual clients have little data (sparsity) or when heterogeneity has multiple simultaneous dimensions. Furthermore, the discovery process itself—exchanging cluster assignments, model similarity scores, or pairwise loss evaluations—can disclose sensitive information about clients’ data distributions beyond what standard federated averaging reveals. TABLE I : Summary of representative results across all experiments E1–E4. Bold indicates best non-Oracle method. Conditional consistently matches or exceeds Oracle. Exp. Config FedAvg Oracle Cond Best Other E1 CIFAR-10 K{=}5 .325 .927 .929 .906 (IFCA) E2a CIFAR-10 K{=}3 .943 .979 .981 .972 (DAC) E2b FMNIST rot. K{=}4 .789 .900 .895 .868 (DAC) E3a FMNIST sem. Rich .671 .950 .950 .943 (IFCA) E3b CIFAR-10 perm. K{=}2 .408 .795 .808 .773 (DAC) E4a MNIST+FM Rich .939 .957 .955 .953 (IFCA) E4b CIFAR-10 C{=}3 .743 .914 .935 .880 (Ditto) In this paper, we take a fundamentally different approach. Instead of discovering client relationships, we characterize each client’s data distribution directly. Each client computes a compact vector of PCA eigenvalues on its local data—a low-dimensional fingerprint capturing the principal modes of variation in its joint feature-label distribution. A single global model is then conditioned on this statistics vector via concatenation before the fully-connected layers, enabling it to adapt its predictions to each client’s distribution. No separate models are maintained, no cluster structure is discovered, and no client statistics are communicated. Our contributions are: 1. A method that conditions a single federated model on locally-computed training data PCA statistics, requiring zero additional communication beyond standard FL (Sec. III ). 2. A comprehensive evaluation across 97 configurations : 4 heterogeneity types (label shift, covariate shift, concept shift, combined), 4 datasets, 7 baselines, and sparsity ranging from 6,000 to 200 samples per client (Sec. IV ). 3. Evidence that continuous distribution statistics outperform discrete cluster knowledge : our method surpasses the Oracle baseline (which knows true cluster assignments) by 1–6% on combined heterogeneity, because continuous statistics capture variation that a discrete cluster ID cannot (Sec. V-B ). 4. Demonstration of unique sparsity robustness : our method maintains near-constant accuracy as client data decreases 20-fold from 6,000 to 200 samples, while all other tested methods degrade by 6–85% (Sec. V-C ). II Related Work (\mathbf{x},y)\in\mathcal{D}_{i}^{\text{train}} Prepare: \bigl[\phi(\mathbf{x})\,\|\,\text{onehot}(y)\bigr] PCA \mathbf{s}_{i} Train: \nabla_{\boldsymbol{\theta}}\,\mathcal{L}\bigl(y,\,f(\mathbf{x};\boldsymbol{\theta},\mathbf{s}_{i})\bigr) FL \boldsymbol{\theta} Infer: \mathbf{x}\in\mathcal{D}_{i}^{\text{test}} f(\mathbf{x};\boldsymbol{\theta},\mathbf{s}_{i}) \hat{y} Fig. 1 : Client-conditional pipeline for client i . Green boxes are client-local; orange boxes involve the federation. Prepare : PCA eigenvalues \mathbf{s}_{i} are computed once from the training data. Train : model updates are computed locally and aggregated via federated learning to produce the shared model \boldsymbol{\theta} . Infer : the client uses the shared model \boldsymbol{\theta} and its own \mathbf{s}_{i} for predictions. Clustered federated learning. IFCA [ 5 ] maintains K cluster models and assigns clients to the model with lowest loss, requiring knowledge of K and \mathcal{O}(K) communication per round. CFL [ 19 ] recursively bipartitions clients based on cosine similarity of gradient updates. FedEM [ 15 ] fits a mixture of distributions but requires maintaining multiple model components per client. A common assumption is that heterogeneity has a discrete cluster structure—an assumption that fails when heterogeneity varies along continuous axes or has multiple simultaneous dimensions. Personalized federated learning. Ditto [ 10 ] learns a global model and then fine-tunes per-client models with a proximal regularizer, alternating between global and local updates. Per-FedAvg [ 3 ] uses MAML-style [ 4 ] meta-learning to train a global model that adapts quickly to each client. FedProx [ 11 ] adds a proximal term to the local objective to prevent client drift from the global model. Federated multi-task learning [ 20 ] explicitly models task relationships via a shared structure. These approaches require per-client model components or adaptation steps at inference time, scaling with the number of clients. See Tan et al. [ 21 ] for a comprehensive survey. Decentralized federated learning. Decentralized SGD [ 12 ] replaces the central server with peer-to-peer communication over a graph topology, with convergence guarantees under various mixing conditions [ 17 ] . DAC [ 13 ] extends decentralized learning with adaptive aggregation weights: each client adjusts how much it averages with each neighbor based on model similarity, enabling implicit clustering without central coordination. Our gossip baseline uses random pairwise model averaging without such adaptation. Our approach differs from all the above in that it does

Read the complete text