Mitigating Privacy–Utility Trade-off in Decentralized Federated Learning via f f -Differential Privacy

Xiang Li, Buxin Su, Chendi Wang, Qi Long, Weijie Su · 2025

Read Mitigating Privacy–Utility Trade-off in Decentralized Federated Learning via f f -Differential Privacy by Xiang Li, Buxin Su, Chendi Wang, Qi Long, Weijie Su (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19934.txt.

Introduction

Xiang Li and four co-authors tighten the privacy accounting for decentralized federated learning, where users train a shared model without a central server. Adding differential privacy to such systems is complicated by peer-to-peer communication, local updates, and shared randomness. Working within the f-differential privacy framework, the authors develop two accounting methods tailored to this setting.

One method tracks privacy leakage between pairs of users under random-walk communication; the other exploits shared secrets to structure the injected noise. Both capture the privacy amplification that sparse communication, local iterations, and correlated noise provide, using Markov chain concentration tools. Experiments on synthetic and real data gave consistently tighter privacy bounds and better utility than the standard Rényi-based accounting.

Catalog record

CREATORXiang Li, Buxin Su, Chendi Wang, Qi Long, Weijie Su
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE111 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-19934

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Mitigating Privacy–Utility Trade-off in Decentralized Federated Learning via f f -Differential Privacy Xiang Li, Buxin Su, Chendi Wang, Qi Long, Weijie Su Abstract Differentially private (DP) decentralized Federated Learning (FL) allows local users to collaborate without sharing their data with a central server. However, accurately quantifying the privacy budget of private FL algorithms is challenging due to the co-existence of complex algorithmic components such as decentralized communication and local updates. This paper addresses privacy accounting for two decentralized FL algorithms within the f f -differential privacy ( f f -DP) framework. We develop two new f f -DP–based accounting methods tailored to decentralized settings: Pairwise Network f f -DP (PN- f f -DP), which quantifies privacy leakage between user pairs under random-walk communication, and Secret-based f f -Local DP (Sec- f f -LDP), which supports structured noise injection via shared secrets. By combining tools from f f -DP theory and Markov chain concentration, our accounting framework captures privacy amplification arising from sparse communication, local iterations, and correlated noise. Experiments on synthetic and real datasets demonstrate that our methods yield consistently tighter ( ϵ , δ ) (\epsilon,\delta) bounds and improved utility compared to Rényi DP–based approaches, illustrating the benefits of f f -DP in decentralized privacy accounting. 1 Introduction Federated Learning (FL) has emerged as a powerful paradigm for privacy-preserving machine learning, enabling user devices to collaboratively train models without sharing raw data ( DBLP:conf/aistats/McMahanMRHA17 ; DBLP:journals/ftml/KairouzMABBBBCC21 ) . In scenarios where a central server is unavailable, untrusted, or communication is expensive, decentralized FL becomes a compelling alternative. Algorithms such as decentralized SGD ( lian2017can ; DBLP:conf/iclr/LiHYWZ20 ) offer scalability and low communication overhead, making them well-suited for edge computing and peer-to-peer applications ( ram2010asynchronous ; yuan2016convergence ; sirb2016consensus ; lan2017communication ; tang2018d ; koloskova2019decentralized ; wang2019matcha ; DBLP:conf/icml/Wang0S024 ) . Despite its decentralized nature, FL remains susceptible to privacy leakage. Even without access to raw data, model updates can reveal sensitive information ( DBLP:conf/sp/ShokriSSS17 ; DBLP:conf/ccs/0001MMBS22 ) , especially under inference or reconstruction attacks ( DBLP:conf/sp/NasrSH19 ; DBLP:conf/nips/GeipingBD020 ) . Differential Privacy (DP) ( dwork2006differential ) offers a formal and principled solution by bounding the effect of any individual data on model outputs. A widely used approach is differentially private stochastic gradient descent (DP-SGD), which adds noise to stochastic gradients to ensure privacy ( DBLP:conf/ccs/AbadiCGMMT016 ; DBLP:conf/focs/BassilyST14 ) . In general, more accurate privacy accounting, i.e., quantifying cumulative privacy loss over multiple training rounds, allows for adding less noise to achieve the same privacy guarantee, thereby improving utility. However, in decentralized settings—where users communicate over graphs, activate randomly, and share information sparsely—privacy accounting remains a significant challenge ( DBLP:journals/tsp/LopesS07 ; DBLP:journals/siamjo/JohanssonRJ09 ; DBLP:journals/tsp/MaoYHGSY20 ; DBLP:conf/icml/Even23 ) . A key factor influencing the difficulty of privacy accounting is the choice of privacy notion, which governs how adversarial capabilities are modeled and how much noise must be added to ensure privacy. Several existing decentralized DP-SGD protocols adopt strong privacy notions such as local differential privacy (Local DP) ( DBLP:conf/pods/EvfimievskiGS03 ; DBLP:journals/siamcomp/KasiviswanathanLNRS11 ; DBLP:conf/nips/WangG018 ; liew2022network ; DBLP:conf/nips/MaZCY23 ) , which assume adversaries can observe all local updates. While offering strong protection, Local DP requires injecting substantial noise, significantly degrading model performance. To address this limitation, recent work has proposed relaxed notions like Pairwise Network DP (PN-DP) ( DBLP:conf/nips/CyffersEBM22 ; Cyffers2024differentially ) , which better capture the partial observability inherent in decentralized systems. In parallel, the Secret-based Local DP (SecLDP) framework ( DBLP:conf/icml/AllouahKFJG24 ; vithana2024correlated ) considers scenarios where user pairs conspire and share secrets with their neighbors and coordinate to add correlated noise to their updates, enabling improved utility-privacy trade-offs under limited collusion. However, existing analyses of both PN-DP and SecLDP rely on Rényi differential privacy (RDP) ( DBLP:conf/csfw/Mironov17 ) or (\epsilon,\delta) -DP, whose privacy bounds are often loose—especially for iterative algorithms. Given the prevalence of iterative algorithms such as DP-SGD, we ask: Is it possible to develop a fine-grained privacy accounting framework for decentralized FL that consistently improves the privacy–utility trade-off across different algorithmic designs? Contribution. Our answer is affirmative: we show that f -differential privacy ( f -DP) ( dong2019gaussian ) provides a unified and effective analytical framework for privacy accounting in decentralized FL. We summarize our contribution below. 1. New f -DP notions for decentralized FL. To enable tight privacy accounting, we introduce two f -DP–based notions tailored to decentralized settings: (1) Pairwise Network f -DP (PN- f -DP) extends PN-DP to quantify fine-grained privacy loss between user pairs in random walk communication, supporting both user-level (Theorem 4.1 ) and record-level (Theorem E.1 ) guarantees; (2) Secret-based f -Local DP (Sec- f -LDP) generalizes f -DP to regimes with correlated noise and shared secrets, as in DecoR-style algorithms (Theorem 4.2 ). 2. Refined privacy guarantees via f -DP. To illustrate, we analyze two representative and practical variants of DP-SGD: (1) Decentralized DP-SGD with random-walk communication (Algorithm 1 ), where updates propagate via a random walk in a communication graph ( Cyffers2024differentially ) ; and (2) DP-SGD with correlated noise (Algorithm 2 ), where user pairs inject structured noise via shared secrets ( DBLP:conf/icml/AllouahKFJG24 ) . For random-walk communication , we develop a PN- f -DP analysis that captures privacy amplification from: (i) communication sparsity , (ii) local iteration updates , and (iii) Markov hitting times . Our approach leverages the joint concavity of trade-off functions and Markov concentration to yield tighter (\epsilon,\delta) bounds than RDP. For correlated-noise methods , we extend f -DP via Sec- f -LDP to account for secret sharing and partial trust, producing sharper bounds under limited collusion. 3. Improved privacy–utility trade-offs. Empirical results on both real and synthetic datasets show that f -DP–based accounting consistently yields tighter (\epsilon,\delta) bounds than existing methods, requiring less noise for a given privacy level and leading to significantly improved model utility. 1.1 Related works DP notions in FL. Local DP assumes all local models are observable by potential attackers ( DBLP:conf/pods/EvfimievskiGS03 ; DBLP:journals/siamcomp/KasiviswanathanLNRS11 ; DBLP:conf/nips/WangG018 ; liew2022network ; DBLP:journals/jmlr/MaY24 ) . However, this strong observability assumption often requires injecting substantial noise, which severely degrades utility in practice. To mitigate this issue, Pairwise Network DP (PN-DP) was recently proposed ( DBLP:conf/nips/CyffersEBM22 ; Cyffers2024differentially ) . PN-DP relaxes the Local DP assumption by modeling only pairwise observability—attackers can access the local model involved in each decentralized communication round, rather than all local models. The prefix “PN” reflects that data exchange occurs pairwise between connected nodes in a network. This relaxation enables the privacy amplification by decentralization ( DBLP:conf/aistats/CyffersB22 ) , providing more fine-grained DP guarantees and improving utility in decentralized learning. Building upon this line of work, our PN- f -DP framework extends PN-DP by adopting the f -DP formalism ( dong2019gaussian ) , which provides lossless privacy accounting through a hypothesis testing interpretation. Existing PN-DP analyses rely mainly on Rényi DP (PN-RDP) ( DBLP:conf/csfw/Mironov17 ) , which often yields loose bounds, while f -DP achieves tighter characterization. Furthermore, unlike previous federated f -DP formulations ( DBLP:conf/aistats/ZhengCLS21 ) , our PN- f -DP captures the additional privacy amplification arising from decentralization, random walks, and iterative communication—crucial algorimithic components in realistic networked settings. In some decentralized scenarios, users may further collude and share secrets ( MR4514170 ; DBLP:conf/icml/AllouahKFJG24 ; vithana2024correlated )

Read the complete text