Federated Structured Sparse PCA for Anomaly Detection in IoT Networks

Chenyi Huang, Xianchao Xiu · 2025

Read Federated Structured Sparse PCA for Anomaly Detection in IoT Networks by Chenyi Huang, Xianchao Xiu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23981.txt.

Introduction

Chenyi Huang and Xianchao Xiu propose, in 2025, a way to spot odd traffic in networks of linked devices while keeping data on each device. Their method joins two known tools: a data squeeze that finds main patterns, and learning split across many sites. Past split versions of the squeeze lacked sparsity, which means most weights stay at zero. Sparsity helps staff read the model and pick out the odd signs with more care.

The new method adds two kinds of sparsity at once, one that drops whole rows of traits and one that trims single weights prone to noise. An update scheme solves the hard split task step by step. Tests show the sparse form is both easier to read and more exact at flagging odd traffic.

Catalog record

CREATORChenyi Huang, Xianchao Xiu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-23981

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Federated Structured Sparse PCA for Anomaly Detection in IoT Networks Thanks: This work was supported by the National Natural Science Foundation of China under Grant 12371306. ( Corresponding author: Xianchao Xiu. ) Chenyi Huang, Xianchao Xiu Abstract Although federated learning has gained prominence as a privacy-preserving framework tailored for distributed Internet of Things (IoT) environments, current federated principal component analysis (PCA) methods lack integration of sparsity, a critical feature for robust anomaly detection. To address this limitation, we propose a novel federated structured sparse PCA (FedSSP) approach for anomaly detection in IoT networks. The proposed model uniquely integrates double sparsity regularization: (1) row-wise sparsity governed by ℓ 2 , p \ell_{2,p} -norm with p ∈ [ 0 , 1 ) p\in[0,1) to eliminate redundant feature dimensions, and (2) element-wise sparsity via ℓ q \ell_{q} -norm with q ∈ [ 0 , 1 ) q\in[0,1) to suppress noise-sensitive components. To solve this nonconvex problem in a distributed setting, we devise an efficient optimization algorithm based on the proximal alternating minimization (PAM). Numerical experiments validate that incorporating structured sparsity enhances both model interpretability and detection accuracy. Our code is available at https://github.com/xianchaoxiu/FedSSP . I Introduction Internet of Things (IoT) provides us with great intelligence and has become an indispensable part of our lives [ 1 ] . However, the rapid development of IoT networks has also brought new challenges, such as security and privacy [ 2 ] . To ensure the security of IoT networks, anomaly detection has been widely studied with the aim of identifying potential security threats, see [ 3 , 4 , 5 ] for recent surveys. Principal component analysis (PCA), as an outstanding dimensionality reduction technique, has been broadly considered in anomaly detection due to its unsupervised nature and effectiveness [ 6 , 7 ] . The key idea behind is that anomalies are different from normal behaviors, i.e., they are likely to have larger reconstruction errors compared to normal observations [ 8 , 9 ] . In many modern applications, especially in distributed systems like IoT networks, data is spread across multiple local gateways [ 10 ] . In such settings, traditional PCA is often impractical. To address these challenges, a distributed PCA model is employed, which can be written as \displaystyle\min_{W} \displaystyle\sum_{t=1}^{N}\|(I-WW^{\top})X_{t}\|_{\textrm{F}}^{2} (1) \displaystyle\rm{s.t.} \displaystyle W^{\top}W=I, where X_{t}\in\mathbb{R}^{d\times n} is the data at the t -th local gateway, and W\in\mathbb{R}^{d\times m} is the global variable that needs to be learned, I\in\mathbb{R}^{d\times d} denotes the identity matrix. The objective function measures the reconstruction error of the data across all local gateways. However, this model suffers from data redundancy and requires all data to be aggregated to a central server, which is not feasible in many practical applications [ 11 ] . To this end, Nguyen et al. [ 12 ] proposed a federated PCA model, called FedPG, which is formulated as \displaystyle\min_{W_{t},Z} \displaystyle\sum_{t=1}^{N}\|(I-W_{t}W_{t}^{\top})X_{t}\|_{\textrm{F}}^{2} (2) \displaystyle\rm{s.t.} \displaystyle W_{t}=Z,\quad\forall t\in[N] \displaystyle W_{t}^{\top}W_{t}=I,\quad\forall t\in[N], where [N]=\{1,2,\ldots,N\} , W_{t} is the local variable learned at the t -th local gateway, and Z is the global variable that needs to be learned. By introducing the global variable Z , FedPG allows local gateways to learn the global variable without sharing their data. However, it does not consider the sparsity of data, which is crucial for anomaly detection as verified in [ 13 , 14 , 15 ] . A natural question is whether we can integrate sparsity into the federated PCA framework. In this paper, we propose the following federated structured sparse PCA (FedSSP) model \displaystyle\min_{W} \displaystyle\sum_{t=1}^{N}\|(I-WW^{\top})X_{t}\|_{\textrm{F}}^{2}+\lambda_{1}\|W\|_{2,p}^{p}+\lambda_{2}\|W\|_{q}^{q} (3) \displaystyle\rm{s.t.} \displaystyle W^{\top}W=I, of which \lambda_{1} and \lambda_{2} are regularization parameters. \|W\|_{2,p}^{p} is the \ell_{2,p} -norm, defined as \sum_{i=1}^{d}\|\mathbf{w}^{i}\|_{2}^{p} , indicating row-wise sparsity, where \mathbf{w}^{i} represents the i -th row [ 16 , 17 ] . \|W\|_{q}^{q} is the \ell_{q} -norm, defined as \sum_{i=1}^{d}\sum_{j=1}^{m}|w_{ij}|^{q} , characterizing element-wise sparsity [ 18 ] . More specifically, p and q are within the range of [0,1) , which is a unified formulation that seamlessly integrates \ell_{0} -norm and \ell_{p} -norm with 0<p<1 [ 19 ] . The objective function measures the data reconstruction error across all local gateways, while also considering the sparsity of data. Overall, the contributions of this paper can be summarized as follows. • We propose a novel federated learning framework, which introduces structured sparsity to consider local gateways. • We develop an efficient optimization algorithm using the conjugate gradient method on the Grassmann manifold. • We conduct extensive experiments on real-world datasets to evaluate the performance of the proposed framework. II Optimization Algorithm In order to solve this problem in a consensus framework, it is necessary to introduce a global variable Z that is shared among all local gateways. In this regard, our proposed FedSSP can be reformulated as \displaystyle\min_{W_{t},Z} \displaystyle\sum_{t=1}^{N}\{\|(I-W_{t}W_{t}^{\top})X_{t}\|_{\textrm{F}}^{2} (4) \displaystyle+\lambda_{1}\|W_{t}\|_{2,p}^{p}+\lambda_{2}\|W_{t}\|_{q}^{q}\} \displaystyle\rm{s.t.} \displaystyle W_{t}^{\top}W_{t}=I,\quad\forall t\in[N] \displaystyle W_{t}=Z,\quad\forall t\in[N]. Furthermore, we introduce two auxiliary variables U_{t} and V_{t} , and rewrite the above problem as \displaystyle\min_{W_{t},U_{t},V_{t},Z} \displaystyle\sum_{t=1}^{N}\{\|(I-W_{t}W_{t}^{\top})X_{t}\|_{\textrm{F}}^{2} (5) \displaystyle+\lambda_{1}\|V_{t}\|_{2,p}^{p}+\lambda_{2}\|U_{t}\|_{q}^{q}\} \displaystyle\rm{s.t.} \displaystyle W_{t}^{\top}W_{t}=I,\quad\forall t\in[N] \displaystyle V_{t}=W_{t},\quad\forall t\in[N] \displaystyle U_{t}=W_{t},\quad\forall t\in[N] \displaystyle W_{t}=Z,\quad\forall t\in[N], which can be transformed into an unconstrained optimization problem as follows \displaystyle\min_{W_{t},U_{t},V_{t},Z} \displaystyle\sum_{t=1}^{N}\{\|(I-W_{t}W_{t}^{\top})X_{t}\|_{\textrm{F}}^{2}+\lambda_{1}\|V_{t}\|_{2,p}^{p} (6) \displaystyle+\lambda_{2}\|U_{t}\|_{q}^{q}+\Phi(W_{t})+\frac{\beta_{1}}{2}\|W_{t}-U_{t}\|_{\textrm{F}}^{2} \displaystyle+\frac{\beta_{2}}{2}\|W_{t}-V_{t}\|_{\textrm{F}}^{2}+\frac{\beta_{3}}{2}\|W_{t}-Z\|_{\textrm{F}}^{2}\}, where \Phi(W_{t}) is the indicator function defined as \Phi(W_{t})=\left\{\begin{aligned} &0,~~~~~~W_{t}^{\top}W_{t}=I,\\ &+\infty,~\text{otherwise},\end{aligned}\right. (7) and \beta_{1},\beta_{2},\beta_{3} are the penalty parameters. For ease of description, denote the objective function of problem ( 6 ) as \sum_{t=1}^{N}f(W_{t},U_{t},V_{t},Z) . Then according to the PAM algorithm [ 20 , 21 ] , it can be solved by iteratively updating (W_{t},U_{t},V_{t},Z) , as shown in Algorithm 1 . Algorithm 1 Optimization algorithm for FedSSP Input: X_{t} Output: Z For k\leq maxit do 1: Update W_{t}^{k+1} by \underset{W_{t}}{\min}~f(W_{t},U_{t}^{k},V_{t}^{k},Z^{k})+\frac{\tau_{1}}{2}\|W_{t}-W_{t}^{k}\|^{2}_{\textrm{F}} 2: Update U_{t}^{k+1} by \underset{U_{t}}{\min}~f(W_{t}^{k+1},U_{t},V_{t}^{k},Z^{k})+\frac{\tau_{2}}{2}\|U_{t}-U_{t}^{k}\|^{2}_{\textrm{F}} 3: Update V_{t}^{k+1} by \underset{V_{t}}{\min}~f(W_{t}^{k+1},U_{t}^{k+1},V_{t},Z^{k})+\frac{\tau_{3}}{2}\|V_{t}-V_{t}^{k}\|^{2}_{\textrm{F}} 4: Update Z^{k+1} by \underset{Z}{\min}~\sum_{t=1}^{N}f(W_{t}^{k+1},U_{t}^{k+1},V_{t}^{k+1},Z)+\frac{\tau_{4}}{2}\|Z-Z^{k}\|^{2}_{\textrm{F}} End for II-A Update W_{t} The W_{t} -subproblem can be reduced to \displaystyle\underset{W_{t}\in\textrm{Gr}(d,m)}{\min} \displaystyle\|(I-W_{t}W_{t}^{\top})X_{t}\|_{\textrm{F}}^{2}+\frac{\beta_{1}}{2}\|W_{t}-U_{t}^{k}\|_{\textrm{F}}^{2} (8) \displaystyle+\frac{\beta_{2}}{2}\|W_{t}-V_{t}^{k}\|_{\textrm{F}}^{2}+\frac{\beta_{3}}{2}\|W_{t}-Z^{k}\|_{\textrm{F}}^{2} \displaystyle+\frac{\tau_{1}}{2}\|W_{t}-W_{t}^{k}\|^{2}_{\textrm{F}}, where \textrm{Gr}(d,m) denotes the Grassmann manifold, defined as \textrm{Gr}(d,m)=\{W_{t}\in\mathbb{R}^{d\times m}\mid W_{t}^{\top}W_{t}=I\}. (9) The objective function can be reformulated by transitioning it from Euclidean space to the Grassmann manifold. Define the objective function in ( 8 ) as g(W_{t}) . Thus, the optimization on \textrm{Gr}(d,m) can be equivalent to \displaystyle\underset{W_{t}\in\textrm{Gr}(d,m)}{\min} \displaystyle g(W_{t}). (10) The Euclidean gradient of g(W_{t}) is \displaystyle\nabla_{W_{t}}g(W_{t})= \displaystyle-2S_{t}W_{t}+\beta_{1}(W_{t}-U_{t}) (11) \displaystyle+\beta_{2}(W_{t}-V_{t})+\beta_{3}(W_{t}-Z), where S_{t}=X_{t}X_{t}^{\top}. Then, the gradient of the objective function, i.e., \text{grad}g(W_{t}) , can be obtained by projecting the Euclidean gradient onto the tangent space of Grassmann manifold at W_{t} , denoted as \mathcal{P}_{W_{t}} . Following [ 22 ] , this projection can be expressed as \displaystyle\operatorname{grad}g(W_{t}) \displaystyle=\mathcal{P}_{W_{t}}(\nabla g(W_{t})) (12) \displaystyle=\nabla g(W_{t})-W_{t}\operatorname{sym}(W_{t}^{\top}\nabla g(W_{t})), where \operatorname{sym}(A)=(A+A^{\top})/2 means extracting the symmetric part of A . As a result, we do not need to compute the gradient \operatorname{grad}g(W_{t}) . Instead, only the Euclidean gradient \nabla g(W_{t}) is needed. Hence, to solve the W_{t} -subproblem, we employ the conjugate gradient method on the Grassmann manifold, as given in Algorithm 2 . Algorithm 2 Solution for W_{t} Input: X_{t},U_{t}^{k},V_{t}^{k},Z^{k} Output: W_{t} For k\leq maxit do 1: Compute the Riemannian gradient \eta^{k} by ( 12 ) 2: Compute the weighted value \beta^{k}=\textrm{Tr}(\eta^{k\top}\eta^{k})/\textrm{Tr}(\eta^{(k-1)\top}\eta^{k-1}) 3: Compute the transport direction \mathcal{T}_{W^{k-1}\rightarrow W^{k}}(\xi^{k-1})=\mathcal{P}_{W^{k}}(\xi^{k-1}) 4: Compute the search direction \xi^{k}=-\eta^{k}+\beta^{k}\mathcal{T}_{W^{k-1}\rightarrow W^{k}}(\xi^{k-1}) 5: Compute the step size t^{k} using backtracking line search f(\mathcal{R}_{W^{k}}(t^{k}\xi^{k}))\geq f(W^{k})+ct^{k}\operatorname{Tr}(\eta^{k\top}\xi^{k}) 6: Update W using retraction W^{k+1}=\mathcal{R}_{W^{k}}(t^{k}\xi^{k}) End for II-B Update U_{t} The U_{t} -subproblem can be simplified to \displaystyle\underset{U_{t}}{\min}~\lambda_{2}\|U_{t}\|_{q}^{q}+\frac{\beta_{1}}{2}\|U_{t}-W_{t}^{k+1}\|_{\textrm{F}}^{2}+\frac{\tau_{2}}{2}\|U_{t}-U_{t}^{k}\|_{\textrm{F}}^{2}.

Read the complete text