VeloxNet: Efficient Spatial Gating for Lightweight Embedded Image Classification

Md Meftahul Ferdaus, Elias Ioup, Mahdi Abdelguerfi, Anton Netchaev, Steven Sloan, Ken Pathak, and Kendall N. Niles · 2026

Read VeloxNet: Efficient Spatial Gating for Lightweight Embedded Image Classification by Md Meftahul Ferdaus, Elias Ioup, Mahdi Abdelguerfi, Anton Netchaev, Steven Sloan, Ken Pathak, and Kendall N. Niles (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19496.txt.

Introduction

Ferdaus and co-authors present VeloxNet, a light network for image classification on embedded devices. Such devices serve tasks like aerial disaster monitoring and levee inspection. VeloxNet swaps SqueezeNet fire modules for gated perceptron blocks with spatial gating. Each block models the full feature map at once with fewer weights. On three aerial data sets, weights drop 46.1 percent versus SqueezeNet. Weighted F1 scores rise by 6.32, 30.83, and 2.51 percent on the three sets.

Catalog record

CREATORMd Meftahul Ferdaus, Elias Ioup, Mahdi Abdelguerfi, Anton Netchaev, Steven Sloan, Ken Pathak, and Kendall N. Niles
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37134
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.19496

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VeloxNet: Efficient Spatial Gating for Lightweight Embedded Image Classification Md Meftahul Ferdaus, Elias Ioup, Mahdi Abdelguerfi, Anton Netchaev, Steven Sloan, Ken Pathak, and Kendall N. Niles Abstract Deploying deep learning models on embedded devices for tasks such as aerial disaster monitoring and infrastructure inspection requires architectures that balance accuracy with strict constraints on model size, memory, and latency. This paper introduces VeloxNet, a lightweight CNN architecture that replaces SqueezeNet’s fire modules with gated multi-layer perceptron (gMLP) blocks for embedded image classification. Each gMLP block uses a spatial gating unit (SGU) that applies learned spatial projections and multiplicative gating, enabling the network to capture spatial dependencies across the full feature map in a single layer. Unlike fire modules, which are limited to local receptive fields defined by small convolutional kernels, the SGU provides global spatial modeling at each layer with fewer parameters. We evaluate VeloxNet on three aerial image datasets: the Aerial Image Database for Emergency Response (AIDER), the Comprehensive Disaster Dataset (CDD), and the Levee Defect Dataset (LDD), comparing against eleven baselines including MobileNet variants, ShuffleNet, EfficientNet, and recent vision transformers. VeloxNet reduces the parameter count by 46.1% relative to SqueezeNet (from 740,970 to 399,366) while improving weighted F1 scores by 6.32% on AIDER, 30.83% on CDD, and 2.51% on LDD. These results demonstrate that substituting local convolutional modules with spatial gating blocks can improve both classification accuracy and parameter efficiency for resource-constrained deployment. The source code will be made publicly available upon acceptance of the paper. Nomenclature Acronyms AIDER Aerial Image Database for Emergency Response LDD Levee Defect Dataset CDD Comprehensive Disaster Dataset PTQ Post-Training Quantization CNN Convolutional Neural Network QAT Quantization-Aware Training ERM Empirical Risk Minimization SGU Spatial Gating Unit FLOPs Floating-Point Operations SVD Singular Value Decomposition FPS Frames Per Second NLP Natural Language Processing GeLU Gaussian Error Linear Unit gMLP Gated Multi-Layer Perceptron Symbols \mathcal{D} Labeled training set \{(\mathbf{x}_{i},y_{i})\}_{i=1}^{N} \mathcal{L} Cross-entropy loss function \mathbf{x}_{i} Input image \in\mathbb{R}^{H\times W\times C} \mathcal{F} Fire module operation y_{i} Class label \in\{1,\dots,K\} L Number of gMLP blocks H,W,C Image height, width, and channels n Number of spatial tokens K Number of target classes d Feature dimension (channel size) N Number of training samples \mathbf{Z} Intermediate representation \in\mathbb{R}^{n\times d} f_{\theta} Classifier parameterized by \theta \mathbf{Z}_{1},\mathbf{Z}_{2} Channel-wise splits of \mathbf{Z} \theta Learnable model parameters \mathbf{W}_{g} Spatial projection weights \in\mathbb{R}^{n\times n} P(\theta) Total trainable parameters \mathbf{b}_{g} Spatial projection bias \mathcal{C}(f_{\theta}) Computational cost (FLOPs) U,V Channel projection matrices \mathcal{M}(\theta) Model storage size (bytes) \sigma GeLU activation function \mathcal{T}(f_{\theta}) Inference latency \odot Hadamard (element-wise) product \mathcal{A}(f_{\theta}) Accuracy metric (e.g., F1 score) s Squeeze ratio in fire modules e_{1},e_{3} Expand dimensions in fire modules I Introduction Embedded devices used in applications such as drone-based disaster monitoring, mobile inspection systems, and infrastructure defect assessment [ 1 , 2 ] impose tight constraints on model size, memory, and inference latency. Convolutional neural networks (CNNs) are effective for image classification, but deploying them on such hardware requires careful trade-offs between model capacity and computational cost. In practice, reducing model size usually comes at the expense of accuracy, and this trade-off has motivated a large body of work on efficient architectures. Prior work on efficient deep learning falls broadly into two categories: post-training compression and efficient architecture design. Post-training compression techniques such as pruning, quantization, and Huffman encoding reduce the size and computational cost of pretrained networks. These methods induce sparsity or reduce numerical precision, making models smaller at the cost of some accuracy loss. Pruning involves removing less important weights or neurons from the network, which can be executed either in a structured manner (removing entire filters or channels) or in an unstructured manner (removing individual weights). This approach helps reduce model size and computational load but traditionally leads to accuracy degradation if not carefully managed. For instance, [ 3 ] introduced a method to prune networks by learning only the important connections, followed by retraining to fine-tune the remaining weights. Recent advancements demonstrate that pruning combined with quantization can achieve significant compression while minimizing accuracy loss [ 4 , 5 ] . Quantization reduces the number of bits required to represent each weight, typically transitioning from 32-bit floating-point to lower bit-width representations like 8-bit integers, thereby reducing model size and accelerating inference. Techniques such as quantization-aware training (QAT) and post-training quantization (PTQ) help maintain accuracy while reducing precision. [ 6 ] highlight the redundancy in CNN weight tensors that can be exploited to further improve compression ratios. [ 7 ] propose a structured representation for sparse quantized weights, achieving high compression ratios with efficient decoding. Huffman encoding, a lossless compression technique, further minimizes storage requirements by encoding frequently occurring weights with fewer bits, often used alongside pruning and quantization to maximize compression benefits. [ 8 ] propose several Huffman-based encoding techniques that outperform traditional methods in reducing encoded data volume during distributed deep learning. [ 9 ] enhance Huffman encoding through frequent pattern mining, achieving superior compression ratios compared to conventional approaches. Another line of work focuses on architectural techniques such as low-rank decomposition and lightweight model design. Low-rank decomposition factorizes weight matrices into lower-dimensional forms, reducing both parameter count and computation. [ 10 ] proposed a sparse low-rank method that sparsifies SVD matrices to achieve high compression with minimal accuracy loss. [ 11 ] introduced PARS, which uses Bayesian optimization to select optimal rank combinations for compression. [ 12 ] developed a stable low-rank tensor decomposition method for CNNs. An alternative approach is to design compact architectures from scratch. SqueezeNet achieves AlexNet-level accuracy on ImageNet with 50 \times fewer parameters by using fire modules that compress and expand channels [ 13 ] . MobileNet uses depthwise separable convolutions to reduce parameters and FLOPs while maintaining competitive accuracy [ 14 ] . Further architectural innovations include techniques such as the EfficientNet model, which scales network depth, width, and resolution systematically using a compound scaling method, achieving state-of-the-art performance with fewer parameters and computations [ 15 ] . Inception modules and residual connections have been explored to improve deep network optimization. Inception modules, introduced by Szegedy et al., enhance performance by increasing network width and capturing multi-scale features within the same layer [ 16 ] . Residual connections, introduced by He et al., enable training of much deeper networks by allowing gradients to flow without vanishing, significantly improving training efficiency and accuracy [ 17 ] . Combining these techniques, such as Inception-ResNet, has been shown to accelerate training and improve performance by leveraging the strengths of both architectures [ 18 ] . Recent work on efficient embedded vision has also explored widened atrous convolutions with attention mechanisms to improve accuracy under resource constraints [ 19 ] . More broadly, attention mechanisms have shown strong results for modeling long-range dependencies. The Transformer architecture [ 20 ] achieves strong performance across many tasks, but the quadratic cost of self-attention limits its use on embedded hardware. Liu et al. [ 21 ] introduced Gated MLPs (gMLPs), showing that spatial gating can replace attention in certain settings while reducing computational cost. Their work focused on large-scale vision and NLP tasks using models with millions of parameters. However, several gaps remain. The original gMLP work targeted large-scale settings (ImageNet, CIFAR) with substantial compute budgets and did

Read the complete text