Decentralized Domain Generalization with Style Sharing: Formal Model and Convergence Analysis
Shahryar Zehtabi a , Dong-Jun Han b , Seyyedali Hosseinalipour c , Christopher G. Brinton a · 2025
Read Decentralized Domain Generalization with Style Sharing: Formal Model and Convergence Analysis by Shahryar Zehtabi a , Dong-Jun Han b , Seyyedali Hosseinalipour c , Christopher G. Brinton a (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06235.txt.
Introduction
Shahryar Zehtabi and three co-authors build a shared learning way that lets a web of peer devices train a model that works in new places, by sharing style facts. Their paper is from two thousand twenty-five.
Shared learning most often takes local data to stay the same from train time to test time. In real life, the data shift. Past work on this task lacked formal math, and stuck to a star shape with one hub. The new way, StyleDDG, works peer to peer, with no hub. Devices share style facts drawn from their own data. The team gives the first step-by-step frame for the aim and proves the method settles as it runs.
Catalog record
| CREATOR | Shahryar Zehtabi a , Dong-Jun Han b , Seyyedali Hosseinalipour c , Christopher G. Brinton a |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-06235
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Decentralized Domain Generalization with Style Sharing: Formal Model and Convergence Analysis Shahryar Zehtabi a , Dong-Jun Han b , Seyyedali Hosseinalipour c , Christopher G. Brinton a Abstract Much of federated learning (FL) focuses on settings where the local dataset statistics remain the same between training and testing. However, this assumption often does not hold in practice due to distribution shifts, motivating the development of domain generalization (DG) approaches that leverage source domain data to train models capable of generalizing to unseen target domains. In this paper, we are motivated by two major gaps in existing work on FL and DG: (1) the lack of formal mathematical analysis of DG objectives; and (2) DG research in FL being limited to the star-topology architecture. We develop Decentralized Federated Domain Generalization with Style Sharing ( StyleDDG ), a decentralized DG algorithm that allows devices in a peer-to-peer network to achieve DG based on sharing style information inferred from their datasets. Additionally, we provide the first systematic approach to analyzing style-based DG training in decentralized networks. We cast existing centralized DG algorithms within our framework and employ their formalisms to model StyleDDG . We then obtain analytical conditions under which the convergence of StyleDDG can be guaranteed. Through experiments on popular DG datasets, we demonstrate that StyleDDG can obtain significant improvements in accuracy across target domains with minimal communication overhead compared to baselines. I Introduction Fig. 1 : Proposed DDG methodology. (a) A decentralized network comprised of m=9 devices, with each device having data from one of three domains (painting, cartoon, photo). The performance of DDG is evaluated on the fourth domain (sketch). (b) In StyleDDG , each device first uses the style statistics of a neighbor for style shifting (getting to red dots from the black ones) for a portion its batch size. Then it concatenates the original plus the shifted points and chooses a new portion of them to do style extrapolation with and generate new styles (grey dots). The circles shown illustrate the style space at the output of a neural network layer. (c) After randomly shifting a portion of the points in the original mini-batch to a new style, and then randomly extrapolating a new portion of the outputs, we are left with a new set of styles with size equal to the original mini-batch. In StyleDDG , we use these new points (red triangles) and apply \MixStyle within them. Federated learning (FL) enables multiple edge devices to collaboratively train a machine learning (ML) model using data stored locally on each device [ 1 ] . Existing conventional “star-topology” FL methods rely on a central server to aggregate the models sent from the clients [ 2 ] . However, in decentralized networks, a central server may not always be available to perform model aggregations [ 3 ] . Motivated by this, recent research has investigated decentralized federated learning (DFL), in which edge devices iteratively communicate their model parameters with their one-hop neighbors after completing their local model training [ 4 ] . Although synchronization among edge devices that participate in conventional FL is readily guaranteed as they overwrite their local models with the one receiver from the server at the end of each training round, DFL algorithms must incorporate a cooperative consensus strategy into their learning process so that devices gradually learn the same model [ 5 ] . When training ML models using FL/DFL, it is typically assumed that test data are drawn from the same probability distribution as training data [ 6 ] . However, this assumption does not always hold; in many real-world applications, a distribution shift can occur between the training set (source) and the test set (target) [ 7 ] . For example, consider a network of autonomous vehicles [ 8 ] with the aim of improving their image classification models using DFL. As these devices are driven mainly during the morning and afternoon, they mostly contain images in daylight (source domains), while their goal is to train a generalized model capable of classifying images taken at night (target domain) as well. ML algorithms have been shown to drastically fail in achieving generalization to such unseen target domains without making special adjustments to their training processes [ 9 ] . Domain generalization (DG) aims to mitigate this issue, with the goal of preventing performance degradation under such distribution shifts [ 6 ] . Due to the practical importance of enabling DG in FL settings, recent research has focused on federated domain generalization (FDG) [ 10 ] , where DG algorithms are designed to operate in a distributed manner across edge devices. FDG introduces additional challenges compared to centralized DG, mainly due to the limited number of source domains accessible to each device [ 11 ] . In FDG, in addition to label heterogeneity between devices, source heterogeneity must also be considered, further complicating the goal of achieving DG [ 12 ] . For instance, in the previously mentioned example, one device may contain images collected in the morning, while another holds images taken in the afternoon, yet the constructed global model is expected to make reliable predictions on night-time images. Research objectives. A large class of DG algorithms are style-based , with style statistics estimated from the source dataset and explored for generalization to the target. These works, whether for centralized training [ 6 ] or for star-topology FL [ 10 ] , lack a concrete mathematical formulation of the objective function to be optimized in training domain-invariant models. Furthermore, the studied DG algorithms in FL are constrained by their star-topology networks, and cannot be readily extended to arbitrary decentralized networks. In this paper, we present one of the first style-based decentralized FDG algorithms, called Decentralized Federated Domain Generalization with Style Sharing ( StyleDDG ), in which edge devices communicate style information of their local datasets with their one-hop neighbors so that collectively they can train ML models that generalize well to unseen domains. Our StyleDDG algorithm, illustrated in Fig. 1 , is inspired by the work in [ 13 ] , where the authors present an FDG algorithm called StableFDG. Unlike [ 13 ] , StyleDDG is a fully decentralized DG algorithm and does not add any additional oversampling or attention layers as in StableFDG. The computational overhead of StableFDG is at least doubled compared to its analogous models, due to its oversampling step and the added attention layer, and this in turn obstructs our understanding of its style exploration strategy in achieving DG performance. Lastly, our other main contribution is to provide the first formal modeling of style-based DG training, both for well-known DG methods and for StyleDDG . Contributions. Our key contributions are as follows: • We present StyleDDG , one of the first fully decentralized domain generalization (DDG) algorithms. In our setup, devices in a decentralized system aim to achieve DG without having any information about the whole network graph, and can only communicate with their one-hop neighbors, unlike in star-topology FL. Specifically, each device in DDG trains an ML model locally using its local dataset, and then it exchanges (i) its ML model parameters and (ii) style statistics of its current batch with its neighbors (see Fig. 1 -(a)).