ResNet: Enabling Deep Convolutional Neural Networks through Residual Learning
Xingyu Liu, Kun Ming Goh · 2025
Read ResNet: Enabling Deep Convolutional Neural Networks through Residual Learning by Xingyu Liu, Kun Ming Goh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-24036.txt.
Introduction
Xingyu Liu and a co-author revisit ResNet, the residual network design that let convolutional networks grow to hundreds of layers. Deep plain networks stall because gradients vanish as they pass through layer after layer. The skip connections introduced by He and colleagues in 2015 give gradients a direct highway, bypassing intermediate layers entirely.
The study re-implements the idea and tests it on CIFAR-10, a standard image classification benchmark. Their ResNet-18 reaches 89.9 percent accuracy, against 84.1 percent for a traditional deep network of similar depth. The residual model also converges faster and trains more stably. As a compact demonstration, the paper is less a new result than a clear confirmation of why residual learning became the default scaffolding of modern computer vision. For readers meeting deep networks for the first time, it offers a small, reproducible case where the famous fix visibly works.
Catalog record
| CREATOR | Xingyu Liu, Kun Ming Goh |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 10 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-24036
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ResNet: Enabling Deep Convolutional Neural Networks through Residual Learning Xingyu Liu, Kun Ming Goh Abstract Convolutional Neural Networks (CNNs) have revolutionised computer vision, but training very deep networks has been challenging due to the vanishing gradient problem. This paper explores Residual Networks (ResNet), introduced by He et al. (2015), which overcome this limitation by using skip connections. ResNet enables the training of networks with hundreds of layers by allowing gradients to flow directly through shortcut connections that bypass intermediate layers. In our implementation on the CIFAR-10 dataset, ResNet-18 achieves 89.9% accuracy compared to 84.1% for a traditional deep CNN of similar depth, while also converging faster and training more stably. I Introduction Deep Convolutional Neural Networks (CNNs) have become the foundation of modern computer vision, powering applications from image classification to object detection. Traditional CNN architectures consist of sequential layers, including convolutions, pooling, and fully connected layers. However, when training very deep networks, the problem of vanishing gradients becomes a major obstacle. This issue arises when gradients shrink exponentially as they are backpropagated through many layers, preventing earlier layers from learning effectively. Surprisingly, the addition of more layers can decrease the performance rather than improve it, a phenomenon known as the degradation problem . Before ResNet, most CNNs could not effectively exceed 20–30 layers; architectures such as VGG-16 and VGG-19 approached this limit using conventional training methods. The introduction of Residual Networks (ResNet) in 2015 marked a turning point, making it possible to train networks with 50, 101, or even 152 layers. The key innovation in ResNet is the use of residual (or skip) connections. Rather than learning a direct mapping from input to output, each block learns a residual mapping, which is the difference between the desired output and the input. This design facilitates gradient flow during training and allows for the construction of deeper networks without degradation in performance. In this paper, we examine ResNet’s architecture, implementation, and performance benefits. We show that residual connections not only enable deeper networks, but also result in more stable training and better accuracy on image classification tasks like CIFAR-10. II Related Work II-A Evolution of Deep CNN Architectures Several key developments have shaped the evolution of deep CNNs. LeNet-5 [ 1 ] introduced the basic CNN structure with convolutional, pooling, and fully connected layers. AlexNet [ 2 ] demonstrated the power of deeper networks combined with GPU acceleration, achieving breakthrough results on ImageNet with an eight-layer model. VGG networks [ 3 ] further pushed depth with 16–19 layers, confirming that deeper architectures tend to perform better while highlighting challenges such as vanishing gradients and high computational cost. GoogLeNet/Inception [ 4 ] introduced inception modules with multiple parallel convolution paths, allowing for a deep yet efficient architecture. II-B Addressing the Vanishing Gradient Problem Batch normalisation [ 5 ] stabilises training by normalising the input of the layers, enabling faster convergence and higher learning rates. Highway Networks [ 6 ] introduced gating mechanisms that allow information to pass through layers unchanged; this work directly inspired ResNet, but incurred additional architectural overhead due to learnt gates. II-C Residual Learning Framework ResNet [ 7 ] introduced residual learning: instead of learning a function H(x) , each layer learns a residual function F(x)=H(x)-x , resulting in the output F(x)+x . This makes it easier for layers to approximate identity mappings when necessary, allowing gradients to flow directly through skip connections. The original paper demonstrated networks up to 152 layers with state-of-the-art results on ImageNet and CIFAR-10. II-D Extensions and Improvements ResNet inspired several extensions. ResNeXt [ 8 ] combines residual learning with grouped convolutions to achieve accuracy gains at a similar cost. DenseNet [ 9 ] connects each layer to every other layer in a feed-forward manner. Wide ResNet [ 10 ] increases the width of the network rather than the depth to achieve comparable or better performance with fewer layers. III Dataset and Methodology III-A Dataset Description We use CIFAR-10, a widely used benchmark with 60,000 32{\times}32 colour images across 10 classes (50,000 train / 10,000 test). CIFAR-10 provides sufficient complexity to demonstrate the benefits of deeper networks while remaining computationally tractable for repeated experiments. III-B Data Preprocessing We apply standard preprocessing and augmentation: (i) pixel values scaled to [-1,1] using (x-0.5)/0.5 ; (ii) labels one-hot encoded; (iii) random horizontal flips and brightness jitter (max_delta =0.1 ); and (iv) an efficient tf.data input pipeline with shuffling, parallel mapping, batching, and prefetching. III-C Network Architectures III-C 1 Baseline CNN Four convolutional blocks with filters 32{\rightarrow}64{\rightarrow}128{\rightarrow}256 ; each block uses Conv2D \rightarrow BatchNorm \rightarrow ReLU \rightarrow MaxPooling. A global average pooling layer feeds a dense classifier; dropout ( p{=}0.5 ) regularises the head. Total parameters: \sim 392k. III-C 2 Mini-ResNet An efficient ResNet-style model: initial 32-filter convolution; four stages with filters [32,64,128,256] ; each stage has two residual blocks, the first using stride 2 and a projection shortcut. Global average pooling, dropout, and a softmax classifier. Total parameters: \sim 2.8M. III-C 3 ResNet-18 (Custom) Adapted to CIFAR-10: initial 64-filter convolution; four stages with filters [64,128,256,512] , two residual blocks per stage; the first block in each stage uses projection shortcuts for downsampling. Batch Normalisation and ReLU throughout; global average pooling, dropout, and a softmax classifier. Total parameters: \sim 11M. III-D Training Configuration All models are trained with Adam (initial learning rate 10^{-3} ); ReduceLROnPlateau (factor =0.2 , patience =3 ); batch size 64; up to 30 epochs with early stopping (patience =7 ) on validation loss; categorical cross-entropy; dropout ( p{=}0.5 ) before the final dense layer; and ModelCheckpoint for best weights. III-E Evaluation Metrics We report Top-1 accuracy, training stability (accuracy/loss curves), computational efficiency (time per epoch and memory), and confusion matrix / classification report for class-wise analysis. IV Results and Discussion IV-A Classification Performance Table I summarises the results. The ResNet-18 model achieves the highest accuracy at 89.9%, improving by 5.8 percentage points over the baseline CNN. Despite having more parameters, ResNet-based models converge more quickly and stably, confirming the efficacy of skip connections for training deeper networks. TABLE I: Model comparison on CIFAR-10. Model Test Acc. Train Time (min) Params Baseline CNN 84.1% 2 392k Mini-ResNet 87.0% 7 2.8M ResNet-18 (With Skip) 89.9% 24 11.2M ResNet-18 (No Skip) 86.0% 210 11.0M Fig. 1: Training accuracy and loss trends for all models. The ResNet-18 model (green) achieves faster convergence and lower training loss, indicating improved gradient flow and stable optimisation compared to the baseline CNN. Fig. 2: Validation accuracy and loss trends for all models. The ResNet-18 (green) converges faster and achieves the highest validation accuracy with the lowest validation loss, demonstrating the benefit of residual connections. IV-B Training Dynamics The baseline CNN exhibits slower convergence and minor overfitting; improvements plateau early, and the training is sensitive to learning rate and initialisation. In contrast, Mini-ResNet and ResNet-18 display smooth, consistent loss reduction and steadily improving validation accuracy, with reduced sensitivity to hyperparameters. IV-C Gradient Flow Analysis Fig. 3: Gradient magnitude distribution across layers for different models. The ResNet-18 with skip connections maintains stable gradient flow throughout the network, while the baseline CNN and non-residual variant experience severe vanishing gradients in earlier layers. We analyse gradient magnitudes (L2 norm) across layers after training. The CNN of