Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation Conference: ; ;
César Sabater, Sonia Ben Mokhtar, Jan Ramon · 2025
Read Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation Conference: ; ; by César Sabater, Sonia Ben Mokhtar, Jan Ramon (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-03746.txt.
Introduction
César Sabater, Sonia Ben Mokhtar, and Jan Ramon design a protocol for computing an average across a network where no central server exists and privacy must hold. Each party's data needs differential privacy protection. Cryptographic fixes tend to cost heavy communication or fall back on a center when connections fail. Older decentralized methods lose much of their accuracy when parties drop out mid-run.
Their protocol, IncA, weaves low-variance shared noise into the computation and feeds sensitive values in step by step. When nobody disconnects for good, the team proves its accuracy matches the centralized setting, which already beats most decentralized rivals. Tests then show the shared-noise design holds its accuracy far better when dropouts strike. The authors aim the protocol at data-heavy processing tasks spread across many machines, where both privacy and resilience are required at once.
Catalog record
| CREATOR | César Sabater, Sonia Ben Mokhtar, Jan Ramon |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 131 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-03746
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation Conference: ; ; César Sabater, Sonia Ben Mokhtar, Jan Ramon Abstract . Achieving differentially private computations in decentralized settings poses significant challenges, particularly regarding accuracy, communication cost, and robustness against information leakage. While cryptographic solutions offer promise, they often suffer from high communication overhead or require centralization in the presence of network failures. Conversely, existing fully decentralized approaches typically rely on relaxed adversarial models or pairwise noise cancellation, the latter suffering from substantial accuracy degradation if parties unexpectedly disconnect. In this work, we propose IncA , a new protocol for fully decentralized mean estimation, a widely used primitive in data-intensive processing. Our protocol, which enforces differential privacy, requires no central orchestration and employs low-variance correlated noise, achieved by incrementally injecting sensitive information into the computation. First, we theoretically demonstrate that, when no parties permanently disconnect, our protocol achieves accuracy comparable to that of a centralized setting—already an improvement over most existing decentralized differentially private techniques. Second, we empirically show that our use of low-variance correlated noise significantly mitigates the accuracy loss experienced by existing techniques in the presence of dropouts. 1. Introduction The training of machine learning models commonly relies on the centralized processing of large datasets. While this approach offers simplicity, it presents substantial privacy implications when sensitive data is involved, or when data aggregation is limited by sharing constraints. For instance, private companies or hospitals might be reluctent to share their data for jointly training prediction models due to competitive secrecy or juridic regulations. For these reasons, algorithms where the data remains local to their owners have gained significant popularity in the last decade. One prominent example is Federated Learning ( McMahan et al. 2017 ; Kairouz et al. 2021b ) , a framework in which participating entities iteratively perform local model training and subsequently transmit intermediate model updates to a central server for aggregation. While this architecture enhances data governance, it nonetheless presents residual challenges in terms of privacy and robustness. Indeed, computational processes relying on a large participant base introduce vulnerabilities to participant behavior, including failures or corruption by adversaries. This may compromise both the integrity of the outcome and participant privacy through inference attacks ( Melis et al. 2019 ; Geiping et al. 2020 ; Mrini et al. 2024 ; Touat et al. 2024 ; Kariyappa et al. 2023 ; Pasquini et al. 2023 ) . Furthermore, the centralized visibility of all model updates to a single party in federated learning establishes a high-risk single point of failure. To address the aforementioned limitations, entirely decentralized machine learning algorithms have been developed ( Hegedűs et al. 2021 ; Beltrán et al. 2023 ) . These solutions present a promising alternative to their federated counterparts, as they remove the need for any centralized entity. In this context, decentralized averaging algorithms, which are the primary focus of this paper, constitute a fundamental component enabling parties to average their model parameters or gradients towards model convergence. Specifically, we concentrate on the canonical task of averaging a set of private values held by participants. Despite its apparent simplicity, averaging serves as a critical primitive in decentralized learning and various other machine learning and data mining tasks, including recommendation systems ( Nicolas et al. 2025 ) , clustering, matrix factorization ( Nicolas et al. 2024 ) , decision trees, empirical cumulative distribution functions ( Barczewski et al. 2025 ) and linear regression. Fundamentally, its applicability extends to any task amenable to decomposition into local computations followed by private averaging. Our objective is then to analyze the privacy of decentralized averaging algorithms through the lens of differential privacy (DP) ( Dwork et al. 2006b ) . This framework has emerged as a gold standard in privacy research due to its solid theoretical guarantees. Nevertheless, demonstrating that an algorithm satisfies DP is often challenging, typically necessitating the injection of noise into computations, which degrades accuracy. This challenge is particularly acute within the local differential privacy (LDP) framework ( Duchi et al. 2013b ; Kasiviswanathan et al. 2011 ; Kairouz et al. 2014 ; Kairouz et al. 2016 ; Chen et al. 2020 ) . In LDP, where data is fully privatized before it is used in collaborative computations, the required noise levels are frequently prohibitively high, rendering computations practically unusable in many real-world scenarios. The integration of cryptographic primitives, such as secure multiparty computation (specifically, Secure Aggregation ( Bonawitz et al. 2017 ; Bell et al. 2020 ) ) or fully homomorphic encryption, into decentralized computations offers a promising solution to substantially reduce the noise levels required for achieving differential privacy (DP) guarantees. This approach can yield privacy-accuracy trade-offs comparable to those observed in central DP settings ( Agarwal et al. 2021 ; Kairouz et al. 2021a ; Heikkilä 2025 ; Chen et al. 2022 ; Jayaraman et al. 2018 ; Dwork et al. 2006a ) , where a single trusted party or curator is responsible for noise injection. However, a significant drawback of these cryptographic methods is their high computational cost, particularly as the number of involved parties increases, or their reliance on a trusted central entity for orchestration and fault tolerance (e.g., handling participant drop-outs). In the decentralized setting, several techniques have been proposed to reduce the noise of local DP. These solutions rely on gossip algorithms in which exchanges are similar to the protocols proposed in ( Boyd et al. 2006 ) . However, they often rely on DP relaxations ( Cyffers and Bellet 2022 ; Cyffers et al. 2022 ) , which can limit practical applicability, or necessitate correlated bounded noise (e.g., pairwise canceling Gaussian noise ( Sabater et al. 2022 ; Allouah et al. 2024 ; Vithana et al. 2025 ) ). The latter approaches either incur high communication costs or demand the cancellation of high-variance noise, which substantially impacts computational efficiency, particularly in the presence of dropouts. In our work, we propose protocols that (i) have the same privacy accuracy trade-offs of Central DP when no drop-outs are present, which is similar to the trusted curator model of DP, (ii) satisfy the classical notion of DP, (iii) are robust to a percentage of parties colluding to infer information of the honest participants and (iv) have a bounded impact in drop-outs thanks to the use of low variance correlated noise. 1.1. Contributions Our contributions are the following: (1) We propose novel protocol for private averaging. It involves iterative exchanges of messages in a way similar to the gossip protocols in ( Boyd et al. 2006 ) with the addition of correlated noise. The latter ensures privacy with a minimal harm in accuracy. We provide a generic construction in which the private values as well as the noise injected in the computation are customizable, allowing to choose the most adequate strategy for different scenarios. Our approach differs from the common technique of previous correlated noise approaches ( Sabater et al. 2022 ; Allouah et al. 2024 ; Vithana et al. 2025 ) and Secure Aggregation ( Dwork et al. 2006a ; Bonawitz et al. 2017 ; Bell et al. 2020 ; Taiello et al. 2024 ; Bell et al. 2023 ) , that are based in pairwise noise masks