Empirical Ablation and Ensemble Optimization of a Convolutional Neural Network for CIFAR-10 Classification Journal: arXiv

Naser Khatti Dizabadi · 2026

Read Empirical Ablation and Ensemble Optimization of a Convolutional Neural Network for CIFAR-10 Classification Journal: arXiv by Naser Khatti Dizabadi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23861.txt.

Introduction

Dizabadi studies convolutional network optimisation for CIFAR-10 image classification through systematic ablation. He tests 17 progressive changes spanning training length, learning-rate schedules, dropout, pooling, depth, filters, and dense layers. Longer training helps steadily, while several structural redesigns hurt despite added variety. A weighted ensemble of the best configs reaches about 86 percent on reduced data and about 89 percent on the full set, up from under 80 percent at baseline.

Catalog record

CREATORNaser Khatti Dizabadi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19976
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.23861

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Empirical Ablation and Ensemble Optimization of a Convolutional Neural Network for CIFAR-10 Classification Journal: arXiv Naser Khatti Dizabadi Abstract Convolutional neural networks (CNNs) remain a central approach in image classification, but their performance depends strongly on architectural and training choices. This paper presents an empirical ablation-based study of CNN optimization for the CIFAR-10 benchmark. The study evaluates 17 progressive modifications involving training duration, learning-rate scheduling, dropout configuration, pooling strategy, network depth, filter arrangement, and dense-layer design. The goal is to identify which changes improve generalization and which increase complexity without improving performance. The baseline model achieved 79.5% test accuracy. Extending training duration improved performance steadily, whereas several structural redesigns reduced accuracy despite greater architectural variation. Based on the strongest individual configurations, a weighted ensemble was constructed, achieving 86.38% accuracy in the reduced-data setting and 89.23% when trained using the full CIFAR-10 dataset. These results suggest that performance gains in CNN-based classification depend less on indiscriminate increases in depth or parameter count than on careful empirical selection of training and architectural modifications. The study therefore highlights the practical value of ablation-oriented optimization and ensemble learning for small-image classification. 1 Introduction Image classification is one of the foundational tasks in computer vision, and convolutional neural networks (CNNs) have become a standard family of models for this task because they learn hierarchical visual features directly from raw image inputs. Landmark work such as AlexNet established the effectiveness of deep convolutional models for large-scale image recognition, while later architectures such as VGG and ResNet showed that depth can improve visual representation learning when supported by suitable design and optimization strategies [ 4 , 6 , 2 ] . At the same time, prior work on dropout and batch normalization demonstrated that training stability and regularization are often as important as raw architectural size [ 7 , 3 ] . These broader developments motivate an important practical question: when improving a CNN, which specific changes actually help, and which ones merely add complexity? This question is especially relevant on compact benchmarks such as CIFAR-10, which contains 60,000 color images of size 32\times 32 across 10 classes and has long served as a standard dataset for comparing image-classification methods [ 5 ] . This paper addresses that question through an empirical ablation-oriented analysis of CNN optimization for CIFAR-10 classification. Rather than proposing a fundamentally new architecture, the study starts from a baseline CNN and applies a sequence of controlled modifications involving training duration, learning-rate scheduling, dropout and pooling design, depth reduction and expansion, filter reconfiguration, and ensemble learning. The baseline model achieved 79.5% accuracy, and the best final configuration reached 89.23% accuracy after the strongest models were combined and evaluated with the full dataset. The contribution of this paper is therefore empirical rather than architectural. First, it provides a structured comparison of 17 CNN modifications derived from a common baseline. Second, it shows that some seemingly stronger changes, including certain deeper or more heavily modified variants, do not reliably improve classification accuracy. Third, it demonstrates that weighted ensembling of selected high-performing models yields the strongest overall result in this study. 2 Related Work CNNs became dominant in image classification largely because they can learn spatially organized feature hierarchies through repeated convolution, nonlinearity, and pooling. AlexNet showed that deep convolutional models trained with modern optimization methods could substantially improve large-scale visual recognition, and VGG later demonstrated that deeper networks built from small 3\times 3 filters could further improve accuracy [ 4 , 6 ] . ResNet then addressed the difficulty of training very deep models by introducing residual learning, showing that increased depth can be beneficial when optimization barriers are reduced [ 2 ] . Regularization and normalization also play a major role in CNN performance. Dropout was introduced to reduce overfitting by randomly omitting units during training, and batch normalization was proposed to stabilize layer input distributions and accelerate deep-network training [ 7 , 3 ] . These ideas are directly relevant to the present work because several experiments in this study test the effect of removing or modifying dropout, normalization, and related components in a baseline CNN. CIFAR-10 has remained an important benchmark for such experiments because it is computationally manageable while still challenging enough to reveal meaningful differences among architectures and training procedures [ 5 ] . Another relevant area is ensemble learning. The general principle of ensembling is that combining multiple predictors can reduce error when the component models capture partly different patterns or make different mistakes [ 1 ] . That logic remains especially useful in neural-network classification when no single model is clearly dominant but several strong variants exist. Unlike studies whose main contribution is a new CNN architecture, this paper offers a controlled empirical study of optimization choices within a CNN framework already built around established components. Its value lies in showing which modifications helped this baseline on CIFAR-10, which did not, and how selected variants could be combined to improve final performance. 3 Dataset and Baseline Model 3.1 Dataset The benchmark used in this study is CIFAR-10. Officially, CIFAR-10 contains 60,000 color images of size 32\times 32 , divided into 10 mutually exclusive classes [ 5 ] . In this paper, the first phase used only part of the dataset, specifically 25,000 images for training and 5,000 for testing. In the final phase, the full CIFAR-10 dataset was used to evaluate the strongest overall approach. 3.2 Baseline CNN The baseline model was a modified CNN that was trained for 25 epochs and achieved 79.5% test accuracy. The architecture contained six convolutional layers and 552,874 parameters. Its filter configuration was 32, 32, 64, 64, 128, and 128. The model also included seven batch-normalization layers, max-pooling after every two convolutional layers, dropout layers with rates of 0.3, 0.5, 0.5, and 0.5, and a dense layer with 128 neurons before the final classification layer. This model served as the experimental anchor for all later comparisons. 4 Methodology The methodology is based on progressive empirical comparison. Instead of replacing the baseline with a single alternative network, the project introduced a series of modifications and tracked their effect on test accuracy. In total, 17 experimental steps were examined. These included increasing the number of epochs, introducing a dynamic learning-rate schedule, removing dropout and max-pooling layers, reducing and increasing model depth, changing filter counts and kernel sizes, altering dense-layer structure, removing batch normalization, and finally constructing a weighted ensemble from selected models. The first group of experiments focused on training schedule. Because the baseline appeared undertrained at 25 epochs, the model was retrained for 50, 100, and 200 epochs. These changes produced a steady pattern of improvement, with test accuracy rising to 81.64%, 83.36%, and 84.60%, respectively. (a) Training and validation accuracy. (b) Training and validation loss. Figure 1 : Step 1 results after retraining the baseline CNN for 50 epochs. Panel (a) shows the accuracy curves for training and validation data, and panel (b) shows the corresponding loss curves. (a) Training and validation accuracy. (b) Training and validation loss. Figure 2 : Step 2 results after increasing training to 100 epochs. Panel (a) presents the accuracy curves and panel (b) presents the loss curves for training and validation data. (a) Training and validation accuracy.

Read the complete text