Compressing CNN models for resource-constrained systems by channel and layer pruning

Ahmed Sadaqa, Di Liu · 2025

Read Compressing CNN models for resource-constrained systems by channel and layer pruning by Ahmed Sadaqa, Di Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-08714.txt.

Introduction

Ahmed Sadaqa and 1 co-author propose hybrid pruning to compress CNN models for small devices in a 2025 paper. Convolutional networks are accurate but often too large for edge hardware. Pruning removes parts of a network to cut size and delay.

Their framework prunes both channels and layers together. It is inspired by EfficientNet scaling, applied in reverse to shrink a network. Experiments show a notable drop in complexity with only a small loss in accuracy. The pruned models also run with lower latency on an NVIDIA Jetson TX2 embedded device. The approach helps deploy vision models where power and memory are tight.

Catalog record

CREATORAhmed Sadaqa, Di Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-08714

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Compressing CNN models for resource-constrained systems by channel and layer pruning Ahmed Sadaqa, Di Liu Abstract Convolutional Neural Networks (CNNs) have achieved significant breakthroughs in various fields. However, these advancements have led to a substantial increase in the complexity and size of these networks. This poses a challenge when deploying large and complex networks on edge devices. Consequently, model compression has emerged as a research field aimed at reducing the size and complexity of CNNs. One prominent technique in model compression is model pruning. This paper will present a new technique of pruning that combines both channel and layer pruning in what is called a "hybrid pruning framework". Inspired by EfficientNet, a renowned CNN architecture known for scaling up networks from both channel and layer perspectives, this hybrid approach applies the same principles but in reverse, where it scales down the network through pruning. Experiments on the hybrid approach demonstrated a notable decrease in the overall complexity of the model, with only a minimal reduction in accuracy compared to the baseline model. This complexity reduction translates into reduced latency when deploying the pruned models on an NVIDIA JETSON TX2 embedded AI device. 1 Introduction 1.1 Overview In the last years, Convolutional Neural Networks (CNNs) have achieved significant performance in various tasks, including Object Detection [ 1 ] , Image Classification [ 6 , 17 , 4 ] , Semantic Segmentation [ 11 ] , and Image Captioning [ 20 ] . These advancements require models to become more complex to capture intricate patterns in the data. However, large and complex models consume substantial memory and computation time. For example, VGG16 [ 17 ] , with its 16 layers and over 130 million parameters, requires 96MB of memory storage per image for a single forward pass, and this requirement doubles when considering the backward pass. Another example is ResNet50 [ 4 ] , which requires 29 hours of training on ImageNet [ 16 ] using an Nvidia Tesla P100 [ 3 ] . Large and complex models are typically used to predict an output using new data, a computational process known as DNN inference . Due to their considerable size and complexity, inference is usually performed in the cloud. In this scenario, data generated by the edge device is sent to the cloud for inference, and the result is sent back to the edge device. This process of transferring data back and forth increases latency, which can be problematic for real-time applications. To address this issue, research indicates that deploying models on edge devices for inference can reduce latency and result in faster inference times [ 9 ] . Deploying large models on edge devices is not feasible due to their limited memory and computational capacity. Model Compression emerges as a field aimed at transforming large models into smaller ones that can achieve performance similar to the larger models. A common technique for model compression, and the one discussed in this paper, is model pruning . Model pruning aims to reduce the size and complexity of the model by removing redundant structures such as neurons, channels, or layers. By removing these structures, the number of parameters is reduced, resulting in a decrease in model complexity and size. In practice, channel and layer pruning stands out as the most prominent pruning techniques, and they will be the focus of this paper. 1.2 Main Contributions This paper proposes a novel pruning framework that combines channel and layer pruning to achieve maximal compression while maintaining the model’s predictive capacity. This will be achieved by using existing channel pruning algorithms in addition to an effective layer pruning algorithm that will be proposed in the following sections. The novel approach is not the layer or channel pruning algorithm per se but rather the concept of applying them sequentially to reduce the network’s width and depth. Furthermore, the paper tries to argue that utilizing existing criteria in research and applying them in a certain way can yield better results especially in terms on reducing complexity. The framework operates in two phases: it begins by executing the channel pruning algorithm, which iteratively removes redundant channels. Next, the pruned model is passed through the layer pruning algorithm, which prunes entire blocks of layers in a one-shot manner. This pruning framework reduces both the width (channels) and depth (layers) of the network. This framework is inspired by EfficientNet [ 19 ] , a CNN architecture developed by Google for image classification. The key innovation of EfficientNet is its method of scaling up the network’s depth and width. The depth of the network refers to the number of layers, while the width refers to the number of channels or filters in each convolutional layer. While EfficientNet focuses on scaling up the network, the concept of this proposed pruning framework revolves around scaling down the network both depthwise and widthwise through channel and layer pruning. The rationale behind this approach is that DNNs, and specifically CNNs, contain a lot of redundancies across channels and layers. For instance, ResNet-1000 achieves similar accuracy to ResNet-101, despite having significantly more layers [ 19 ] . This suggests that some layers (depthwise) and channels (widthwise) may not significantly contribute to the network’s output, and removing them can result in a smaller network with comparable performance. In summary, this paper offers the following contributions: • Proposing an effective layer pruning algorithm that prunes the network in a one-shot manner based on a generated importance score for each block of layers. • Combining existing channel pruning algorithm with the proposed layer pruning algorithm to create an aggressive hybrid pruning framework. • Empirically examining the reduction in complexity of the pruned model and latency reduction on edge devices using the proposed pruning framework. 2 Related Work In this section, we will discuss two pruning techniques in research which are channel and layer pruning. 2.1 Channel Pruning Channel pruning is considered one technique of structured pruning that focuses on removing structures from the network, such as channels in this case, thereby maintaining a more regular network architecture. In contrast, unstructured pruning (weight pruning) is a fine-grained technique used to remove irrelevant connections or weights from a model. Weight pruning is not very common because, even though it can significantly reduce the number of parameters, this reduction does not necessarily translate into a reduction in latency due to the irregular structure resulting from this pruning. There is an extensive amount of research done on channel pruning ranging from simple to sophisticated algorithms. In Li et al.’s approach [ 7 ] , a global pruning method is employed where all filters are ranked based on their absolute values. Filters with lower magnitudes are then removed. ThiNet, as described by Luo et al. [ 12 ] , adopts a data-driven strategy to select filters for pruning. It assesses the importance of each filter and considers its impact on the subsequent layer’s output, ensuring effective pruning decisions. Molchanov et al. [ 13 ] propose a method utilizing Taylor expansion of the first and second orders to capture the sensitivity of the loss function to any changes in the parameters. HRank, introduced by Lin et al. [ 8 ] , calculates the rank of filters

Read the complete text