Tackling Federated Unlearning as a Parameter Estimation Problem

Antonio Balordi, Lorenzo Manini, Fabio Stella, Alessio Merlo · 2025

Read Tackling Federated Unlearning as a Parameter Estimation Problem by Antonio Balordi, Lorenzo Manini, Fabio Stella, Alessio Merlo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-19065.txt.

Introduction

Antonio Balordi, Lorenzo Manini, Fabio Stella, and Alessio Merlo recast federated unlearning as parameter estimation in a 2025 paper. Privacy law requires erasing data from trained models, and federated setups make that harder because data stay on client devices. Full retraining is often out of reach.

Their method uses second order curvature data to find the parameters most sensitive to the data being forgotten. Only those are reset, followed by a short federated retraining. The server never needs raw client data after an initial summary step. Benchmarks showed privacy attacks succeeding at near chance rates and accuracy near retrained models. A planted backdoor trigger was also neutralized, restoring model integrity.

Catalog record

CREATORAntonio Balordi, Lorenzo Manini, Fabio Stella, Alessio Merlo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-19065

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tackling Federated Unlearning as a Parameter Estimation Problem Antonio Balordi, Lorenzo Manini, Fabio Stella, Alessio Merlo Abstract Privacy regulations require the erasure of data from deep learning models. This is a significant challenge that is amplified in Federated Learning, where data remains on clients, making full retraining or coordinated updates often infeasible. This work introduces an efficient Federated Unlearning framework based on information theory, modeling leakage as a parameter estimation problem. Our method uses second-order Hessian information to identify and selectively reset only the parameters most sensitive to the data being forgotten, followed by minimal federated retraining. This model-agnostic approach supports categorical and client unlearning without requiring server access to raw client data after initial information aggregation. Evaluations on benchmark datasets demonstrate strong privacy (MIA success near random, categorical knowledge erased) and high performance (Normalized Accuracy against re-trained benchmarks of ≈ \approx 0.9), while aiming for increased efficiency over complete retraining. Furthermore, in a targeted backdoor attack scenario, our framework effectively neutralizes the malicious trigger, restoring model integrity. This offers a practical solution for data forgetting in FL. K eywords Federated Unlearning \cdot Parameter estimation \cdot Right To Be Forgotten \cdot Data Privacy \cdot Hessian computation. 1 Introduction The rapid adoption of deep learning has led to models trained on massive datasets that achieve outstanding results across domains. However, such outcomes come at the cost of privacy: Neural networks (NN) are known to memorize training data, potentially leaking sensitive information ( [ 1 ] , [ 2 ] ). This privacy concern has led to machine unlearning gaining momentum, which generally refers to a set of techniques aimed at allowing organizations to meet the proper privacy constraints by selectively removing the influence of specific data from trained models. This capability has recently become crucial as privacy regulations, such as GDPR ( [ 3 ] , [ 4 ] ) or CCPA ( [ 5 ] , [ 6 ] ), provide the user with the “ right to be forgotten ”. As a consequence, clients can request the complete removal of their data in any training scenario. This work will focus on Federated Learning (FL), a learning technique that allows users to collectively obtain the benefits of shared models without the need to share the data [ 7 ] . Despite this privacy-preserving design, the need to retroactively remove a user’s data contribution presents a significant challenge, leading to the development of the field of Federated Unlearning (FU) . Here, the gold standard refers to the practice of full retraining , that is, the model is discarded and a new one is trained from scratch without the removed client; this approach grants an exact unlearning ( [ 8 ] , [ 9 ] ). However, it often leads to significant communication overhead, inefficient resource utilization, and high computational costs, rendering it impractical for many real-world federated learning (FL) deployments ( [ 10 ] , [ 11 ] ). Consequently, applying exact unlearning through full retraining reveals that it is often unfeasible in FL scenarios with high client turnover or when dealing with large models and datasets [ 12 ] . Therefore, there is a strong need for FU methods that can approximate the guarantees provided by the gold standard while operating within feasible resource constraints and avoiding the need to retrain from scratch. Contributions. This paper introduces a new federated unlearning framework that conceptualizes information leakage as a parameter estimation problem . This information-theoretic perspective enables the development of a high-performing unlearning method that selectively resets and retrains (minimally, for one or a few epochs) only those parameters identified as most informative about the data to be unlearned, utilizing second-order information (Hessian diagonals) computed post-training. Our approach is broadly applicable across diverse network architectures, supports arbitrary dataset unlearning, and integrates seamlessly into the federated setting, enabling server-side unlearning with no client data access and retraining overhead. Paper Structure . The remainder of the paper is organized as follows. Section 2 reviews relevant previous work on federated unlearning. Section 3 introduces the federated unlearning problem. Our proposed framework is detailed in Section 4 , followed by a description of the experimental setup and datasets in Section 6 . We present some numerical results in Section 7 and discuss them in Section 8 . Section 9 concludes the paper, summarizing key findings and describing future directions. 2 Related Work Machine Unlearning (MU) addresses the need to remove the influence of specific data from a trained model, a capability required to ensure privacy and comply with regulatory requirements [ 13 ] . In centralized settings, extensive research has explored various techniques. Among these, methods such as SISA (sharded retraining) [ 11 ] , which focuses on efficient exact removal via sharding, and influence functions [ 14 ] , which offer a theoretical lens on the impact of data, have been extensively explored. However, translating these MU principles to Federated Learning (FL) contexts, a domain we refer to as Federated Unlearning (FU), introduces new challenges. The direct application of centralized MU methods to FU scenarios is often problematic. For example, techniques such as SISA or influence functions can be complex to implement effectively with deep learning models typically employed in FL, and their unlearning guarantees may not extend to these distributed environments [ 15 ] . For SISA, adapting its isolated shard retraining paradigm to the naturally distributed individuals’ data in FL without substantial re-coordination or compromising global model integrity is complex. Influence functions, on the other hand, typically require computationally intensive calculations that are challenging for deep FL models and often need direct data access, which violates FL’s core privacy tenets. Moreover, FU must inherently work with the complexities of FL itself, including the distributed nature of client data, communication bottlenecks, and increased privacy considerations [ 16 , 17 ] . Current FU approaches can be broadly categorized as follows, each with significant limitations. • Retraining-Based/Approximation : These methods approximate complete retraining, for example, through knowledge distillation [ 18 ] , [ 19 ] . Although faster, they still require multiple communication rounds between clients participating in the training and significant computation. More critically, in knowledge-distillation setups the student often inherits the teacher’s membership signals, so a model can ‘pass’ one audit while still retaining information—making the quality of forgetting hard to measure and increasingly fragile as the forget set or model complexity grows [ 15 , 20 ] . • Gradient/Update Manipulation : Several approaches attempt to reverse the impact of unlearned data by manipulating gradients or model updates. For example, [ 21 ] proposed methods including gradient ascent on the forgotten client’s data. However, these are often heuristic; forcing misprediction does not guarantee removal of memorized information and can leave the model vulnerable to reconstruction or membership inference attacks since it could still retain encoded information, as shown in [ 2 ] and in [ 22 ] . • Parameter Masking/Perturbation : These techniques alter parameters related to forgotten data, for example, pruning or adding noise [ 23 ] , [ 24 ] . Identifying relevant parameters is non-trivial; currently, most methods typically fail to capture complex dependencies [ 25 ] . Determining the extent of the modification requires tuning and

Read the complete text