Machine Unlearning for Robust DNNs: Attribution-Guided Partitioning and Neuron Pruning in Noisy Environments

Deliang Jin, Gang Chen, Shuo Feng, Yufeng Ling, Haoran Zhu · 2025

Read Machine Unlearning for Robust DNNs: Attribution-Guided Partitioning and Neuron Pruning in Noisy Environments by Deliang Jin, Gang Chen, Shuo Feng, Yufeng Ling, Haoran Zhu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11615.txt.

Introduction

Deliang Jin and co-authors borrow the logic of machine unlearning to protect neural networks from noisy training data. Corrupted labels can drag a model's accuracy down, and standard fixes either assume a known noise pattern or demand full retraining. The team's framework sorts training examples by gradient-based attribution scores, treating the high scorers as clean data. It then finds and prunes the neurons most sensitive to noise, using regression-based checks.

After pruning, the network is fine-tuned on the clean subset alone. On CIFAR-10 with injected label noise, the method beat plain retraining by about 10 points of accuracy. It also cut retraining time by up to 47 percent in some settings, and it held up on a speech task as well. The authors frame the mix of data-level removal and model-level repair as a scalable route to robustness.

Catalog record

CREATORDeliang Jin, Gang Chen, Shuo Feng, Yufeng Ling, Haoran Zhu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-11615

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Machine Unlearning for Robust DNNs: Attribution-Guided Partitioning and Neuron Pruning in Noisy Environments Deliang Jin, Gang Chen, Shuo Feng, Yufeng Ling, Haoran Zhu Abstract Deep neural networks (DNNs) have achieved remarkable success across diverse domains, but their performance can be severely degraded by noisy or corrupted training data. Conventional noise mitigation methods often rely on explicit assumptions about noise distributions or require extensive retraining, which can be impractical for large-scale models. Inspired by the principles of machine unlearning, we propose a novel framework that integrates attribution-guided data partitioning, discriminative neuron pruning, and targeted fine-tuning to mitigate the impact of noisy samples. Our approach employs gradient-based attribution to probabilistically distinguish high-quality examples from potentially corrupted ones without imposing restrictive assumptions on the noise. It then applies regression-based sensitivity analysis to identify and prune neurons that are most vulnerable to noise. Finally, the resulting network is fine-tuned on the high-quality data subset to efficiently recover and enhance its generalization performance. This integrated unlearning-inspired framework provides several advantages over conventional noise-robust learning approaches. Notably, it combines data-level unlearning with model-level adaptation, thereby avoiding the need for full model retraining or explicit noise modeling. We evaluate our method on representative tasks (e.g., CIFAR-10 image classification and speech recognition) under various noise levels and observe substantial gains in both accuracy and efficiency. For example, our framework achieves approximately a 10% absolute accuracy improvement over standard retraining on CIFAR-10 with injected label noise, while reducing retraining time by up to 47% in some settings. These results demonstrate the effectiveness and scalability of the proposed approach for achieving robust generalization in noisy environments. Keywords: machine unlearning; neuron pruning; fine-tuning; attribution methods. 1 Introduction Deep neural networks (DNNs) have achieved unprecedented success across diverse domains, fundamentally transforming fields such as computer vision krizhevsky2012imagenet ; he2016deep ; dosovitskiy2020image , natural language processing devlin2019bert ; vaswani2017attention ; brown2020language , speech recognition graves2013speech ; hinton2012deep ; hannun2014deep , medical diagnosis esteva2017dermatologist ; litjens2017survey , and reinforcement learning mnih2015human ; silver2016mastering ; schulman2017proximal . These remarkable achievements stem largely from groundbreaking architectural innovations including convolutional neural networks (CNNs) krizhevsky2012imagenet , residual networks he2016deep , transformer architectures vaswani2017attention , and generative adversarial networks (GANs) goodfellow2014generative . Each of these developments has contributed substantially to enhanced model performance, training stability, and the ability to capture complex data representations. However, despite these architectural advances, the effectiveness and reliability of DNNs remain fundamentally constrained by the quality and integrity of their training data. Real-world datasets are inherently susceptible to various forms of noise that compromise both input features and target labels, presenting significant challenges for robust model development. Feature noise manifests through multiple sources including sensor measurement inaccuracies, data acquisition errors, missing or corrupted values, and deliberate adversarial perturbations biggio2013evasion designed to mislead model predictions frenay2013classification . Concurrently, label noise emerges from human annotation inconsistencies, subjective interpretation differences among annotators northcutt2021pervasive , ambiguous category boundaries, automated labeling system errors, and temporal changes in labeling criteria song2019selfie . The pervasiveness of these noise sources is exemplified in widely-adopted benchmark datasets: comprehensive analysis reveals that approximately 4% of ImageNet labels contain errors or ambiguities rolnick2017deep , while clinical datasets such as MIMIC-III johnson2016mimic exhibit substantial missing physiological measurements and inconsistent diagnostic annotations that further complicate reliable model training. The fundamental challenge lies in the demonstrated capacity of DNNs to memorize arbitrary noise patterns, a phenomenon that severely undermines their generalization capabilities arpit2017closer . High-capacity networks can achieve near-zero training error even when trained on datasets with completely randomized labels, indicating their propensity to overfit to spurious correlations rather than learning meaningful underlying patterns zhang2016understanding ; belkin2019reconciling . This memorization behavior becomes particularly problematic in practical applications where model reliability is critical, as networks may confidently make incorrect predictions based on learned noise patterns rather than genuine signal amodei2016concrete . Existing approaches to address noise-related challenges in deep learning can be broadly categorized into several paradigms, each with distinct advantages and limitations. Data preprocessing and cleaning methods lee2018cleannet ; lai2021tods ; brodley1999identifying attempt to identify and remove noisy samples before training, but often require substantial domain expertise and may inadvertently eliminate valuable edge cases or minority class examples. Noise-robust loss functions wang2019symmetric ; zhang2018generalized modify the training objective to reduce sensitivity to label errors ma2018dimensionality ; han2018co , yet these approaches typically rely on strong assumptions about noise distribution characteristics that may not hold in practice. Sample re-weighting jiang2018mentornet ; patrini2017making and curriculum learning techniques ren2018learning ; liu2015classification dynamically adjust the influence of training examples during optimization, but struggle with complex, asymmetric, and instance-dependent noise patterns commonly encountered in real-world scenarios xia2019anchor . Meta-learning li2019learning ; shu2019meta approaches attempt to learn noise-robust representations, but require additional computational overhead and may not scale effectively to large-scale problems. Machine unlearning emerges as a particularly promising paradigm for addressing data quality issues, originally developed to meet privacy compliance requirements by efficiently removing the influence of specific training samples without complete model retraining cao2015towards ; bourtoule2021machine ; graves2021amnesiac . Unlike traditional data cleaning approaches that operate at the dataset level, unlearning techniques provide fine-grained control over individual sample influences within trained models. Recent theoretical and empirical investigations thudi2022necessity have demonstrated that strategic application of unlearning methods can significantly enhance model robustness against noisy labels by selectively eliminating the impact of detrimental examples while preserving valuable generalizable knowledge golatkar2020eternal . This selective forgetting capability offers computational efficiency advantages over complete retraining while maintaining model performance on high-quality data. Despite these advances, existing noise mitigation strategies face several critical limitations that constrain their practical applicability. Many approaches require explicit assumptions about noise characteristics, distributions, or generation processes that are difficult to verify in real-world settings. Others demand extensive manual intervention, domain-specific expertise, or substantial computational resources that may not be available in resource-constrained environments. To address these limitations, we propose a novel robust learning framework that synergistically combines attribution-based data sundararajan2017axiomatic ; simonyan2013deep quality assessment, discriminative neuron pruning han2015learning ; louizos2017learning , and targeted fine-tuning howard2018universal ; ruder2019neural to comprehensively address the challenges posed by both noisy input features and corrupted labels. Our approach leverages gradient-based attribution methods to quantify sample quality without requiring explicit noise distribution assumptions, employs neuron-level analysis to identify and remove noise-sensitive model components, and applies selective fine-tuning to restore and enhance model performance on high-quality data. The key contributions of this work include: 1. Attribution-guided Data Partitioning : We utilize gradient-based attribution scores to reliably distinguish high-quality from noisy samples. Leveraging Gaussian mixture models allows for probabilistic clustering without imposing restrictive assumptions about the noise distributions. 2. Discriminative Neuron Pruning : We introduce a novel methodology for quantifying neuron sensitivity to noise as a linear regression problem based on neuron activations. This strategy enables precise identification and removal of neurons primarily influenced by noisy samples. 3. Targeted Fine-Tuning : After pruning, the network is fine-tuned exclusively on high-quality data subsets, effectively recovering and enhancing its generalization capability without incurring substantial computational costs. Our proposed framework addresses critical gaps in existing noise mitigation strategies by providing a unified approach that operates without explicit noise modeling requirements, scales effectively to large-scale models and datasets, and integrates seamlessly into standard deep learning workflows. The empirical results demonstrate significant improvements in model robustness and

Read the complete text