LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty
Christoforos N. Spartalis, Theodoros Semertzidis, Petros Daras, Efstratios Gavves · 2025
Read LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty by Christoforos N. Spartalis, Theodoros Semertzidis, Petros Daras, Efstratios Gavves (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18314.txt.
Introduction
Christoforos Spartalis and co-authors wrote a 2025 paper on making trained models forget select training data. Rules and ethics can demand that a model's memory of some samples be erased. Retraining from scratch is often too costly, above all on huge data sets.
Their method smooths the model's output odds up to a bound drawn from information theory. That caps the overconfidence that comes from rote memory of samples. They tested it on two model types against eight rivals across five public sets. A new score lets them judge forgetting where no retrain check exists. On a million image set, where retraining is out of reach, the method still won. It beat rivals on both speed and effect.
Catalog record
| CREATOR | Christoforos N. Spartalis, Theodoros Semertzidis, Petros Daras, Efstratios Gavves |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18314
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty Christoforos N. Spartalis, Theodoros Semertzidis, Petros Daras, Efstratios Gavves Abstract We present LoTUS, a novel Machine Unlearning (MU) method that eliminates the influence of training samples from pre-trained models, avoiding retraining from scratch. LoTUS smooths the prediction probabilities of the model up to an information-theoretic bound, mitigating its over-confidence stemming from data memorization. We evaluate LoTUS on Transformer and ResNet18 models against eight baselines across five public datasets. Beyond established MU benchmarks, we evaluate unlearning on ImageNet1k, a large-scale dataset, where retraining is impractical, simulating real-world conditions. Moreover, we introduce the novel Retrain-Free Jensen-Shannon Divergence (RF-JSD) metric to enable evaluation under real-world conditions. The experimental results show that LoTUS outperforms state-of-the-art methods in terms of both efficiency and effectiveness. Code: https://github.com/cspartalis/LoTUS . Conference on Computer Vision and Pattern Recognition, 2025. 1 Introduction Machine Unlearning focuses on removing the influence of training samples from pre-trained models without retraining the model entirely [ 26 ] . Its applications include privacy protection in Machine Learning [ 14 , 15 , 3 ] . As an alternative to retraining a model from scratch, Machine Unlearning addresses three principal challenges: 1 minimizing the time window during which the model is vulnerable, 2 minimizing the cost in terms of time and computational resources, and 3 minimizing the dependency on access to all training data to retain the utility of the pre-trained model, as full data access is often limited due to privacy policies and storage limitations. Therefore, an effective and efficient unlearning algorithm should meet the following requirements [ 13 ] : 1 Effectively eliminate the impact of specific training samples from the model. 2 Retain the model’s performance on the remaining training samples, even if access to the training set is limited. 3 Be efficient in terms of both time and computational resources. Considering only the effectiveness of unlearning, the gold standard is to retrain the model from scratch without the samples designated for unlearning (also known as forget samples). To this end, two main taxonomy classes have been developed: exact unlearning , which aims to produce a model that is statistically indistinguishable from the gold standard, which is often infeasible for complex algorithms [ 4 ] or inefficient [ 3 ] , and approximate unlearning , which relaxes the constraints of exact unlearning and adopts a suite of evaluation metrics that typically measure how well the unlearned model approximates the gold standard in terms of accuracy and resilience to privacy attacks [ 12 ] . The scope of this study concerns the following questions: Q1: Can an unlearning method efficiently eliminate the influence of training samples from a pre-trained model while approximating the effectiveness of the gold standard? Q2: Can this unlearning method effectively handle large-scale datasets and models under real-world constraints, including limited data access? To answer these questions we propose the Lo gits T empering U nlearning S trategy ( LoTUS for short, such as the fruit that made Ulysses’ comrades forget). LoTUS leverages the known tendency of Deep Neural Networks (DNNs) to memorize sample-specific features from training data and output over-confident predictions [ 38 ] , a vulnerability exploited by Membership Inference Attacks (MIAs) to assess whether a sample is a member of the training set [ 32 ] . To this end, LoTUS smooths the model’s output probabilities, as shown in Fig. 1 , increasing the entropy to resemble that of unseen (during training) samples. This unseen set, which may include synthetic data, enables LoTUS to calibrate the retained information for forget samples post-unlearning and replicate the decision-making process of the gold standard model. Since the gold standard model was not trained on the forget samples, it naturally avoids overconfident predictions and typically exhibits lower accuracy on them. To better approximate the gold standard’s performance, LoTUS also introduces Gumbel noise into the pre-trained model’s output distribution. This encourages diverse predictions and helps reduce the pre-trained model’s accuracy on forget samples to resemble that of the gold standard. Figure 1 : Machine Unlearning via smoothing prediction probabilities: LoTUS eliminates sample-specific information (e.g., unique fur patterns in cat images) that the DNN memorized and exposed through overconfident predictions. Then, the DNN responds to unlearned samples as if they were never part of the training set. In contrast to previous studies that have focused mainly on the input or model space, LoTUS follows an entropy-based approach that directly modifies the model’s output probabilities, emphasizing an underexplored unlearning approach. The difference from the existing method which indiscriminately maximizes entropy using random labeling [ 17 ] is that LoTUS uses an information-theoretical bound to control the uncertainty introduced to the model. Our main contributions are as follows: 1. We introduce LoTUS, the first unlearning method that operates directly in the model’s output space, while following an information-theoretic approach to determine the amount of entropy increase. This bound enables cautious unlearning that approximates the gold standard. 2. We introduce the Retrain-Free Jensen-Shannon Divergence (RF-JSD) unlearning metric, which enables evaluation in real-world scenarios. RF-JSD exhibits a strong Pearson correlation ( \text{PCC}\!=\!0.92\textsubscript{\scriptsize{$\pm$0.04}} ) with the established JSD score while eliminating the need to retrain the model. Compared to the existing retrain-free ZRF score, RF-JSD offers enhanced interpretability and efficiency. 3. We introduce a novel large-scale experimental setup that incorporates a large-scale dataset (ImageNet1k), and limited access to the training set, with the aim of simulating real-world conditions, where model retraining is infeasible. Overall, we evaluated LoTUS on the Vision Tranformer and ResNet18 modes, against eight baseline methods on five public datasets. Extensive experiments demonstrate that LoTUS outperforms state-of-the-art approaches in terms of both unlearning effectiveness and efficiency, in all benchmarks (novel and established). 2 Related Work Machine Unlearning was first introduced in [ 4 ] with an approach that decomposes traditional Machine Learning algorithms into summations, enabling the reduction of the influence of specific data points for exact unlearning. Subsequently, a theoretical framework for approximate unlearning was proposed in [ 18 ] , suggesting a Hessian-based regularization technique limited to models with convex loss functions to mitigate membership inference risks. Unlearning was subsequently extended to deep neural networks in [ 15 ] by introducing a Lagrangian regularization approach that utilizes the Fisher Information Matrix as a Hessian approximation. More recent works have improved unlearning effectiveness and efficiency [ 23 , 14 ] and expanded machine unlearning applications in diverse areas, including user privacy [ 30 ] , security defense [ 29 ] , toxic content removal [ 12 , 20 ] , copyright protection [ 16 ] , and bias mitigation [ 7 ] . Emphasizing on privacy applications, Machine Unlearning has been defined as a privacy game aiming to reduce the accuracy of MIAs [ 3 ] . Algorithms are categorized into two classes depending on where the manipulations are applied: model space and data space [ 37 ] . In model-space approaches, manipulations include regularizing the loss function to shift model weights far from the pre-trained model and close to the gold standard [ 23 , 9 , 15 ] . Another approach is pruning, which involves identifying and reducing the influence of weights that are most affected by the data to be unlearned [ 14 ,