How Sparsity Allocation Shapes Label-Free Post-Pruning Recoverability

Qishi Zhan, Minxuan Hu, Liang He, [4pt] Marquette University · 2026

Read How Sparsity Allocation Shapes Label-Free Post-Pruning Recoverability by Qishi Zhan, Minxuan Hu, Liang He, [4pt] Marquette University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-21972.txt.

Introduction

Qishi Zhan, Minxuan Hu, and Liang He study how sparsity allocation shapes whether label-free repair can rescue a pruned neural network. Heavy unstructured magnitude pruning can drop accuracy near random, and labeled retraining may be unavailable in deployment. Label-free post-pruning repair can partly recover collapsed sparse models, but its success depends on the sparse model left upstream. They map a recoverable regime that ends when activation signal runs too low, with activation-statistic repair still recovering nontrivial accuracy past the point where other methods fail.

Catalog record

CREATORQishi Zhan, Minxuan Hu, Liang He, [4pt] Marquette University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34538
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.21972

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

How Sparsity Allocation Shapes Label-Free Post-Pruning Recoverability Qishi Zhan, Minxuan Hu, Liang He, [4pt] Marquette University Abstract Unstructured magnitude pruning at high sparsity ratios can reduce a neural network’s accuracy to near-random performance, yet retraining with labeled data is often infeasible in practical deployment settings. Recent work on Adaptive Signal Resuscitation (ASR) shows that label-free, gradient-free channelwise repair can recover substantial accuracy from collapsed sparse models [ 10 ] . Building on this repair operator as a fixed backend, we ask a different question: how does the choice of upstream sparsity allocation affect the amount of activation signal that remains recoverable after pruning? We evaluate ERK and LAMP allocations under the same ASR repair protocol across CIFAR-10, CIFAR-100, and Imagenette with ResNet-18, ResNet-34, and ResNet-50, at sparsities from 90% to 95.5%. Our results show that allocation choice systematically changes post-repair accuracy at the same global sparsity, that a repair-sensitive transition regime exists in which BN recalibration fails but ASR still recovers nontrivial accuracy, and that the width and location of this regime vary with architecture and dataset difficulty. Additional validations on ImageNet-100 and DenseNet-121 further characterize how recoverability varies with data scale and connectivity pattern. 1 Introduction Neural network pruning reduces inference cost by removing weights, but aggressive pruning at high sparsity can cause severe accuracy collapse, a regime in which the pruned model performs at or near random-chance levels, well beyond ordinary accuracy degradation. Recovering from this collapse typically requires fine-tuning with labeled data, which may be unavailable when models are deployed in data-restricted environments or when the original training pipeline is inaccessible. Prior work on pruning has concentrated primarily on how to prune . Sparsity allocation strategies such as ERK [ 1 ] and LAMP [ 6 ] determine how density budgets are distributed across layers, and magnitude-based criteria remove weights of smallest absolute value. These methods address the allocation problem but do not specify what should be done after pruning, particularly in the label-free setting. Recent work on Adaptive Signal Resuscitation (ASR) fills this gap by formulating label-free post-pruning recovery as a channelwise activation-statistic repair problem [ 10 ] . ASR estimates a variance-matching correction for each output channel and stabilizes it with a data-driven shrinkage rule, requiring only forward passes on a small calibration set. This repair operator has been shown to substantially outperform BatchNorm (BN) recalibration at high sparsity across several architectures and datasets. This paper asks a complementary question. Rather than asking how to repair a pruned model, we ask when a pruned model remains repairable: given a fixed label-free repair backend, how does the upstream sparsity allocation affect the amount of activation signal that survives pruning and can be recovered? We treat ASR as a fixed repair protocol and vary only the allocation rule, examining how ERK and LAMP preserve or destroy the channelwise activation signal that ASR can exploit. This perspective shifts the focus from repair operator design to the recoverability consequences of allocation choice. Figure 1 illustrates the overall pipeline. Figure 1 : The post-pruning repair pipeline studied in this work. A dense pretrained model is pruned via a sparsity allocation rule (ERK or LAMP) followed by magnitude pruning. The ASR repair operator [ 10 ] is then applied as a fixed post-processing stage using only unlabeled calibration images \mathcal{C} . We study how the allocation choice affects post-repair accuracy under this fixed backend. Contributions. 1. We study the interaction between layerwise sparsity allocation and label-free post-pruning repairability, using ASR [ 10 ] as a fixed repair backend rather than proposing a new repair operator. 2. We show that allocation choice changes the amount of recoverable activation signal left after pruning: ERK and LAMP yield substantially different post-repair accuracies at the same global sparsity, with the direction of the advantage changing across architectures and datasets. 3. We identify repair-sensitive transition regimes in which BN recalibration begins to fail but ASR still recovers nontrivial accuracy. Outside this band, either BN already performs well or the sparse model is close to irrecoverable collapse; the width and location of the transition vary with architecture, allocation, and dataset difficulty. 4. Across ResNet-18, ResNet-34, and ResNet-50 on CIFAR-10, CIFAR-100, and Imagenette, with additional ImageNet-100 and DenseNet-121 validations, we show that post-repair recoverability varies systematically with allocation, sparsity, dataset difficulty, and architecture, and that the appropriate ASR variant (aggressive versus conservative) depends on the activation landscape left by the allocation. 2 Related Work Sparsity allocation. ERK [ 1 ] distributes the density budget across layers using a structural rule based on layer shape. LAMP [ 6 ] derives a layer-adaptive magnitude pruning score by accounting for layerwise distortion. Both address the allocation problem but do not specify a post-pruning repair operator. This work treats ERK and LAMP as upstream allocation rules and examines how their structural differences affect the activation signal available to a fixed downstream repair stage. Post-pruning fine-tuning. The standard practice after pruning is to fine-tune the sparse model with labeled data and gradient-based optimization [ 3 , 2 ] . This requires labeled training data and a full backward pass. Our setting prohibits both, and we treat training-based recovery as a distinct operational regime rather than a direct baseline. BatchNorm recalibration. Recomputing BN statistics from a calibration set without labels or gradients is a common lightweight adjustment after pruning or quantization [ 7 ] . We treat BN recalibration as the primary label-free baseline and characterize the sparsity regime in which it begins to fail, which defines the lower boundary of the repair-sensitive transition region. Post-training quantization and calibration repair. 8 propose data-free quantization via weight equalization and bias correction. 5 study post-training pruning through layerwise calibration, including bias correction followed by BN recalibration. These approaches target mean-shift distortion; ASR differs by addressing variance-level degradation, which we find necessary for reliable recovery in high-sparsity regimes. Label-free post-pruning repair. Adaptive Signal Resuscitation (ASR) proposes a channelwise repair operator for sparse vision networks, showing that matching the granularity of repair to the granularity of damage substantially improves recovery over layer-wise methods [ 10 ] . The present work uses ASR as a fixed backend and studies a different question: how does upstream allocation determine the amount of signal available for ASR to repair? Activation-statistic correction. Activation statistics have been used in related contexts to reduce distributional mismatch after model transformation. REPAIR renormalizes preactivations against reference statistics to mitigate variance distortion in model merging [ 4 ] . ASR shares the calibration-only philosophy but applies channelwise correction to pruning-induced activation damage. The present work does not introduce a new correction operator; instead, it uses ASR to study how allocation choices determine the activation landscape available for repair. Concurrent weight-rescaling approaches. Recent concurrent work proposes energy compensation for post-training LLM pruning using column-wise and row-wise weight energy ratios to rescale pruned weights [ 9 ] . That correction is weight-based and data-free, applied at moderate sparsity on transformer architectures without BatchNorm. The setting differs from ours along three axes: we use activation statistics from calibration images, target extreme sparsity regimes where activation collapse is severe, and operate on convolutional networks with BatchNorm layers. 3 Problem Setting Let f_{\theta} denote a dense pretrained network with parameters \theta , and

Read the complete text