FedSEA: Achieving Benefit of Parallelization in Federated Online Learning
Harekrushna Sahu, Pratik Jawanpuria, Pranay Sharma · 2026
Read FedSEA: Achieving Benefit of Parallelization in Federated Online Learning by Harekrushna Sahu, Pratik Jawanpuria, Pranay Sharma (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19336.txt.
Introduction
Sahu, Jawanpuria, and Sharma extend online federated learning, where clients learn from live data streams without sharing private data, by adding a stochastically extended adversary. In the standard setup, the adversary model rules out any benefit from parallel work and misses sources of spread in these problems. Under their framework, the loss stays fixed across clients, while the adversary picks each client's data mix on its own at each step. Their method pairs random gradient descent at clients with regular server pooling, and they prove bounds on network results.
Catalog record
| CREATOR | Harekrushna Sahu, Pratik Jawanpuria, Pranay Sharma |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42553 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.19336
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FedSEA: Achieving Benefit of Parallelization in Federated Online Learning Harekrushna Sahu, Pratik Jawanpuria, Pranay Sharma Abstract Online federated learning (OFL) has emerged as a popular framework for decentralized decision-making over continuous data streams without compromising client privacy. However, the adversary model assumed in standard OFL typically precludes any potential benefits of parallelization. Further, it fails to adequately capture the different sources of statistical variation in OFL problems. In this paper, we extend the OFL paradigm by integrating a stochastically extended adversary (SEA). Under this framework, the loss function remains fixed across clients over time. However, the adversary dynamically and independently selects the data distribution for each client at each time. We propose the FedSEA algorithm to solve this problem, which utilizes online stochastic gradient descent at the clients, along with periodic global aggregation via the server. We establish bounds on the global network regret over a time horizon T T for two classes of functions: (1) for smooth and convex losses, we prove an 𝒪 ( T ) \mathcal{O}(\sqrt{T}) bound, and (2) for smooth and strongly convex losses, we prove an 𝒪 ( log T ) \mathcal{O}(\log T) bound. Through careful analysis, we quantify the individual impact of both spatial (across clients) and temporal (over time) data heterogeneity on the regret bounds. Consequently, we identify a regime of mild temporal variation (relative to stochastic gradient variance), where the network regret improves with parallelization. Hence, in the SEA setting, our results improve the existing pessimistic worst-case results in online federated learning. 1 Introduction Online Learning. With the rapid growth of big data, traditional batch learning paradigms face significant limitations when data arrives and evolves continuously. In batch learning, the primary aim is to learn a model using a static dataset that generalizes well to unseen data. However, real-world applications such as financial market forecasting, autonomous vehicles, and recommendation systems ( 3 ) often get continuous streams of data, and the underlying distribution shifts over time ( 13 ) . The online learning (OL) paradigm addresses this problem of learning in dynamic environments. Instead of retraining the model from scratch, an online learner updates its model incrementally as new data arrives ( 3 ; 15 ) . Unlike batch learning, the goal in OL is to produce a sequence of models that minimizes the cumulative regret of the learner, defined as the difference between the cumulative loss of the learner and that of an idealized comparator with prior knowledge of the system dynamics ( 4 ) . The Shift to Federated Learning. The standard OL frameworks assume that the incoming data streams can be collected at a centralized server. As data generation rapidly shifts to decentralized edge devices ( 8 ) , centralization introduces significant challenges such as increased communication overhead and severe privacy concerns. Federated learning (FL) mitigates these issues by ensuring that the raw data never leaves the client devices ( 12 ) . The clients use their local data to train models and communicate with the server intermittently, which in turn periodically aggregates these local updates into a global model ( 12 ) . By distributing the training process, FL ensures data privacy and reduces communication overhead by leveraging local computational resources ( 5 ) . FL has been successfully used in various sectors like healthcare, Internet of Things (IoT), and decentralized wireless communication networks ( 20 ) . However, the existing work in FL primarily focuses on batch learning problems ( 12 ; 6 ; 11 ; 7 ; 18 ) . Online Federated Learning (OFL). Some recent works explore FL for continuous data streams ( 14 ; 16 ) . In standard OFL, clients do not hold a static dataset. Instead, an adversary sequentially dictates the loss functions for each client in each round ( 14 ; 2 ) . Although there are two sources of variation in the data distribution: spatial (across clients) and temporal (over time), existing regret bounds often fail to explicitly capture their individual impact. This is due to the obscuring effect of the restrictive bounded gradient (Lipschitz) assumption ( 14 ; 16 ) . While recent work by Kwon et al. ( 10 ) captures the spatial heterogeneity by relaxing the bounded gradient assumption, capturing both spatial and temporal heterogeneity simultaneously remains an open challenge. Furthermore, a significant limitation in much of the existing OFL literature is the reliance on true (sub)gradient computation ( 14 ; 2 ) . This is often impractical for applications that consistently generate massive volumes of data, such as real-time recommendation systems. On the other hand, using stochastic gradients is computationally cheap. In addition, averaging these stochastic gradient-based updates across clients results in reduced error due to variance reduction. Consequently, the speed of learning improves with an increasing number of participating clients ( 16 ) . To overcome the limitations of full gradient feedback and capture the benefits of linear speedup resulting from parallelization, we need more realistic assumptions. The Stochastically Extended Adversary (SEA) framework, initially introduced by Sachs et al. ( 17 ) provides a robust alternative. In the SEA setting, the learner and the adversary make simultaneous moves at each time step: the learner plays a decision, and the adversary selects a data distribution. Both parties then observe a random sample drawn from the chosen distribution, and the learner incurs a loss based on this sample ( 17 ; 1 ) . The SEA framework gracefully bridges the gap between stochastic optimization (adversary is restricted to a single fixed distribution) and adversarial online optimization (adversary selects Dirac delta distributions). Figure 1: Illustration of online federated learning in load forecasting in smart grids. The substations can be modeled as different clients, each with its unique load distribution. The curves above show the average hourly electricity load (in million kWh) in 4 regions of the US during a typical day in a few selected months (Source: U.S. Energy Information Administration ). The spatial variations in the load are evident from the y-axis scaling in the four regions. The temporal variations depend on the time of day and month, with the highest load during summer afternoons. Proposed framework - OFL with SEA. We propose integrating the OFL framework with SEA. Under the proposed setting, the clients receive continuously evolving data distributions, chosen by the SEA. By modeling clients as individually operating within SEA environments, we can accurately capture both the stochastic variance inherent in physical data observations and temporal distribution shifts. The proposed setting is useful for modeling complex real-world decentralized systems, such as recommendation engines or load forecasting in a smart grid, where a global model must aggregate knowledge from clients, facing both spatial heterogeneity and temporal evolution simultaneously. See Figure 1 for an illustration of the spatial and temporal heterogeneity inherent in a smart grid application. Our main contributions are summarized as follows: • Integration of SEA with OFL: We formalize a novel distributed online learning framework incorporating a Stochastically Extended Adversary (SEA) and propose the FedSEA algorithm (Algorithm 1 ). Our framework allows us to capture the individual effects of spatial (across clients) and temporal (across time) variations in data distributions. • Regret Bounds: For smooth and convex loss functions, our algorithm