Spectral Complex Autoencoder Pruning: A Fidelity-Guided Criterion for Extreme Structured Channel Compression

Wei Liu, Xing Deng, Haijian Shao, Yingtao Jiang · 2026

Read Spectral Complex Autoencoder Pruning: A Fidelity-Guided Criterion for Extreme Structured Channel Compression by Wei Liu, Xing Deng, Haijian Shao, Yingtao Jiang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-09352.txt.

Introduction

Wei Liu and three co-authors cut channels from a vision net by a new grade of waste. Cut work trims a net to run fast. Most grades ask if a channel's own marks vary much. This team asks a deep ask: if we drop a channel, how much of its work can the rest still do. A channel whose work is held by peers is waste, in truth.

Their grade is called SCAP. It builds a joint field from the full input and one channel's out marks. That field shows how the channel talks with the whole. A reader who trims nets gets a grade that may cut far more, with less loss in score.

Catalog record

CREATORWei Liu, Xing Deng, Haijian Shao, Yingtao Jiang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-09352

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Spectral Complex Autoencoder Pruning: A Fidelity-Guided Criterion for Extreme Structured Channel Compression Wei Liu, Xing Deng, Haijian Shao, Yingtao Jiang Abstract We propose Spectral Complex Autoencoder Pruning (SCAP), a reconstruction-based criterion that measures functional redundancy at the level of individual output channels. For each convolutional layer, we construct a complex interaction field by pairing the full multi-channel input activation as the real part with a single output-channel activation (spatially aligned and broadcast across input channels) as the imaginary part. We transform this complex field to the frequency domain and train a low-capacity autoencoder to reconstruct normalized spectra. Channels whose spectra are reconstructed with high fidelity are interpreted as lying close to a low-dimensional manifold captured by the autoencoder and are therefore more compressible; conversely, channels with low fidelity are retained as they encode information that cannot be compactly represented by the learned manifold. This yields an importance score (optionally fused with the filter ℓ 1 \ell_{1} norm) that supports simple threshold-based pruning and produces a structurally consistent pruned network. On VGG16 trained on CIFAR-10, at a fixed threshold of 0.6 0.6 , we obtain 90.11 % 90.11\% FLOP reduction and 96.30 % 96.30\% parameter reduction with an absolute Top-1 accuracy drop of 1.67 % 1.67\% from a 93.44 % 93.44\% baseline after fine-tuning, demonstrating that spectral reconstruction fidelity of complex interaction fields is an effective proxy for channel-level redundancy under aggressive compression. I Introduction Modern convolutional neural networks (CNNs) remain a strong backbone for visual recognition on resource-constrained devices, yet their deployment is often bottlenecked by compute (FLOPs) and memory footprint (parameters). Structured channel pruning —removing entire convolutional output channels and rewiring subsequent layers accordingly—is particularly attractive because it yields practical speedups on standard hardware. However, achieving extreme compression (e.g., simultaneously reducing FLOPs and parameters by more than 90\% ) while keeping the post–fine-tuning accuracy drop within a small margin remains challenging. The core difficulty is not the mechanics of removing channels, but the design of a criterion that reliably identifies functionally redundant channels under distribution shift induced by pruning. A large body of pruning work estimates channel importance from easily computed proxies such as filter magnitudes, activation statistics, gradient/Taylor saliency, or combinations thereof. These proxies are computationally efficient and often work well at moderate pruning ratios, but they can become brittle at high compression: (i) magnitude-based pruning may discard low-norm channels that nevertheless capture rare but critical patterns; (ii) gradient-based criteria can be noisy and sensitive to the current mini-batch and optimization state; and (iii) layer-wise normalization and global thresholds can cause inconsistent pruning strength across layers, occasionally collapsing a crucial layer to too few channels and causing irreversible accuracy loss. More fundamentally, many proxies assess a channel in isolation, while redundancy is inherently relational : a channel is redundant only relative to what other channels already explain about the input-to-output transformation. This paper proposes Spectral Complex Autoencoder Pruning (SCAP), a reconstruction-based criterion designed to measure redundancy at the level of a channel’s input–output interaction . Consider a convolutional layer with input activation X\in\mathbb{R}^{B\times C_{\mathrm{in}}\times H\times W} and output activation Y\in\mathbb{R}^{B\times C_{\mathrm{out}}\times H_{1}\times W_{1}} . For each output channel k , SCAP constructs a complex interaction field Z_{k}\;=\;X\;+\;i\,\mathrm{Broadcast}(\mathrm{Resize}(Y_{k}))\;\in\;\mathbb{C}^{B\times C_{\mathrm{in}}\times H\times W}, where Y_{k} denotes the k -th output feature map, spatially aligned to (H,W) and broadcast across input channels. We then transform Z_{k} to the frequency domain and train a small-capacity autoencoder (one per layer) to reconstruct the normalized spectra. At scoring time, the autoencoder produces a reconstruction \widehat{Z}_{k} and we compute a fidelity score as the cosine similarity between the vectorized original and reconstruction (real and imaginary parts concatenated), averaged over the batch. Intuitively, if a channel’s interaction field lies close to a low-dimensional manifold captured by the layer-wise autoencoder, then the channel expresses patterns already well explained by that manifold and is therefore more compressible; conversely, low fidelity indicates patterns that the low-capacity model cannot reproduce, suggesting higher novelty and importance. This intuition can be stated more formally under a standard manifold-learning viewpoint. Let \mathcal{M}_{\ell} denote the set of typical spectral interaction fields for layer \ell induced by data and the current network. A sufficiently constrained autoencoder trained to minimize reconstruction error over samples from \mathcal{M}_{\ell} learns a mapping g_{\ell} that approximates the identity on directions that are common and compressible under its bottleneck, while necessarily incurring larger error on directions that are rare or require higher intrinsic dimensionality than the bottleneck permits. Therefore, the reconstruction fidelity provides a principled model-based measure of compressibility: high fidelity implies that the interaction field is well represented by the learned low-dimensional structure, and pruning such channels is less likely to remove unique information relative to the manifold captured by the autoencoder . Importantly, SCAP does not claim that fidelity alone guarantees zero accuracy loss—pruning changes the network and the data-induced distribution of activations—but it provides a defensible, layer-local criterion grounded in compressibility rather than ad hoc statistics. In practice, we optionally fuse fidelity-derived importance with the filter \ell_{1} norm to guard against corner cases where a channel has high reconstructability yet remains salient due to amplitude. Beyond the criterion itself, a practical obstacle is memory. Naïvely scoring all channels by expanding a batch to size B\cdot C_{\mathrm{out}} is prohibitive under limited GPU memory. SCAP is implemented with strict memory discipline: we process output channels sequentially (or in small groups), use gradient accumulation for autoencoder training, and explicitly release intermediate tensors (FFT outputs, reconstructions) to keep peak memory bounded. This allows training and scoring on commodity GPUs (e.g., 11 GB) without sacrificing the conceptual model. Our principal contributions are as follows: • We introduce the complex interaction field representation that couples multi-channel inputs with per-channel outputs and enables relational redundancy analysis; • we propose a frequency-domain, low-capacity autoencoder criterion and a fidelity-based importance score (with optional magnitude fusion) for threshold-based structured pruning. • We provide a memory-efficient implementation via per-channel processing, gradient accumulation, and explicit tensor release. II Related Work II-A Structured and channel pruning for CNN acceleration Early pruning studies focused on removing individual weights using saliency measures derived from second-order approximations [ 1 , 2 ] . Modern deep model compression revisited sparsification at scale and combined pruning with quantization and entropy coding [ 3 , 4 ] . More recently, the community investigated the trainability and practicality of sparse subnetworks, including the lottery-ticket perspective and dynamic sparse training schemes [ 5 , 6 , 7 , 8 , 9 , 10 , 11 ] , together with large-scale empirical assessments and surveys [ 12 , 13 , 14 ] . For deployment-oriented speedups, structured pruning removes entire channels/filters and preserves dense tensor shapes. Representative criteria include magnitude-based heuristics [ 15 ] , sparsity regularization and re-scaling factors [ 16 , 17 , 18 ] , gradient- or sensitivity-based importance [ 19 , 20 ] , and discriminative pruning objectives [ 21 ] . A complementary thread formulates pruning as an optimization/automation problem, including reinforcement-learning or search based selection [ 22 , 23 , 24 ] . Many recent methods further improve stability by adopting “soft” or progressive deletion strategies [ 25 , 26 , 27 ] . These approaches are highly effective, but their scoring signals are often

Read the complete text