ImageHD: Energy-Efficient On-Device Continual Learning of Visual Representations via Hyperdimensional Computing
Jebacyril Arockiaraj, Dhruv Parikh, Viktor Prasanna · 2026
Read ImageHD: Energy-Efficient On-Device Continual Learning of Visual Representations via Hyperdimensional Computing by Jebacyril Arockiaraj, Dhruv Parikh, Viktor Prasanna (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21280.txt.
Introduction
Jebacyril Arockiaraj, Dhruv Parikh, and Viktor Prasanna build ImageHD, a chip accelerator that lets edge devices keep learning from streaming images without backpropagation or bulky stored examples. Hyperdimensional computing updates fast and non-iteratively, and a compact convolutional front end supplies strong visual features. A hardware-aware method bounds stored exemplars and merges clusters efficiently on an AMD Zynq FPGA. On CORe50 it runs about 40 times faster and 383 times more energy-efficiently than tuned CPU baselines.
Catalog record
| CREATOR | Jebacyril Arockiaraj, Dhruv Parikh, Viktor Prasanna |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44343 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.21280
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ImageHD: Energy-Efficient On-Device Continual Learning of Visual Representations via Hyperdimensional Computing Jebacyril Arockiaraj, Dhruv Parikh, Viktor Prasanna Abstract On-device continual learning (CL) enables edge devices to adapt to non-stationary data streams without offline retraining, which is critical for real-time edge AI. However, most existing CL methods rely on backpropagation-based updates or exemplar-heavy classifiers, incurring high compute, memory, and latency overheads that hinder deployment on resource-constrained devices. Hyperdimensional Computing (HDC) offers an alternative by enabling fast, non-iterative online updates. When paired with a lightweight convolutional neural network (CNN) feature extractor, HDC supports efficient on-device adaptation with strong visual representations. Despite this progress, prior HDC-based continual learning systems typically employ multi-tier memory hierarchies and complex cluster management, which complicate deployment on resource-constrained hardware platforms. In this paper, we propose ImageHD, an FPGA accelerator for on-device continual learning of visual data based on HDC, designed to address these limitations. ImageHD targets streaming continual learning under strict latency and on-chip memory constraints, without expensive iterative optimization. At the algorithmic level, we introduce a hardware-optimized continual learning method that bounds total class exemplars through a unified exemplar memory and a hardware-efficient cluster merging strategy, while integrating a quantized CNN front-end to reduce edge deployment overhead without sacrificing accuracy. At the system level, ImageHD is realized as a streaming dataflow architecture on the AMD Zynq ZCU104 FPGA, integrating hyperdimensional encoding, similarity search, and bounded cluster management using word-packed binary hypervector representations to enable massively parallel bitwise operations within tight on-chip resource budgets. Experimental results on the CORe50 dataset demonstrate up to 40.4 × 40.4\times ( 4.84 × 4.84\times ) speedups and 383 × 383\times ( 105.1 × 105.1\times ) energy efficiency gains over optimized CPU (GPU) baselines, establishing HDC-enabled continual learning as a practical foundation for real-time, on-device lifelong learning systems. I Introduction Continual Learning (CL) [ 10 , 3 , 8 , 38 , 44 ] , is the ability of an artificial intelligence (AI) system to incrementally acquire knowledge from sequence of data streams, similar to biological learning systems, without training from scratch [ 17 ] . This capability is important for edge applications, where devices continuously encounter new visual data and must adapt on-device without reliance on cloud retraining. Scenarios include autonomous drones, robotic inspection systems, IoT sensors, etc. where network connectivity may be limited, and energy and latency budgets are constrained. Table I summarizes continual learning methods and highlights key trade-offs relevant to edge deployment. Approaches such as [ 19 ] and [ 20 ] rely on supervised learning and require labeled data. However, in realistic continual learning scenarios, manual annotation is often impractical, particularly when new data is generated on-the-fly [ 14 ] . Self-supervised methods, including [ 14 ] and [ 34 ] , eliminate the need for labels but depend on backpropagation-based training, resulting in high computational cost and making them unsuitable for resource-constrained edge devices. These limitations motivate learning paradigms that avoid both labeled supervision and gradient-based optimization while maintaining the capability for continual learning. TABLE I: Comparison of representative continual (visual) learning pipelines from the perspective of on-device deployability . Method Sup. Backbone Update Mechanism Bounded HW Deep SLDA [ 19 ] Sup. ResNet-18 Class statistics (SLDA) ✗ CPU/GPU AILIS [ 20 ] Sup. VGG-16 Exemplar-based KNN ✗ FPGA CaSSLe [ 14 ] SSL ResNet-18/50 Backprop + distill ✗ GPU LUMP [ 34 ] SSL ResNet-18 Backprop + replay ✗ GPU LifeHD [ 51 ] Unsup. MobileNetV2 HDC + spectral merge ✓ \dagger CPU/GPU ImageHD (Ours) Unsup. INT8 MobileNetV2 HDC + kMeans++ merge ✓ FPGA \dagger [ 51 ] bounds memory via multi-tier storage and spectral merging, introducing additional control state and cubic-time consolidation. Hyperdimensional Computing (HDC) is a brain-inspired computing paradigm that represents information using low-precision vectors, often comprising thousands of dimensions [ 22 ] . Unlike deep neural networks, HDC does not rely on complex iterative training and often generalizes well with minimal data and computation. Even though [ 46 ] prior work introduces a dynamic encoding technique to improve accuracy, HDC accuracy on complex visual tasks still lags behind that of deep neural networks [ 5 , 33 ] . [ 51 ] represents a pioneering effort that bridges this gap by combining a frozen convolutional neural network feature extractor with HDC-based encoding to enable continual learning on visual workloads. To mitigate catastrophic forgetting, where new data overwrites previously learned representations, [ 51 ] maintains short-term memory (STM) and long-term memory (LTM), where class hypervectors accumulated in STM are periodically merged into LTM using spectral graph clustering. While effective, this design introduces several limitations. First, the spectral graph clustering introduces a cubic-time complexity ( O(N^{3}) ) due to the eigen decomposition step. Second, separate short-term and long-term memory structures increase memory footprint and control complexity. Finally, existing implementations utilize general-purpose computing platforms, such as GPUs, which have been shown to be inefficient for HDC workloads due to their high dynamic power consumption, limited memory bandwidth utilization, and lack of hardware specialization [ 9 ] . To address these limitations, we present ImageHD, a domain-specific FPGA accelerator co-designed for efficient on-device continual learning. Our main contributions include: • Clustering: We exploit the quasi-orthogonality of high-dimensional representations to replace cubic-complexity spectral clustering with a hardware-optimized kMeans++ algorithm. This reduces the algorithmic complexity of memory updates from O(N^{3}) to linear time ( O(N) ), which results in a speedup of 8.6 \times . • Unified Exemplar Memory: We replace multi-tier short-term and long-term memory structures with a single, consolidated cluster memory that is explicitly bounded to fit on-chip resources, simplifying control logic and improving hardware efficiency. • Quantization: We leverage the inherent robustness of hyperdimensional representations to deploy an INT8-quantized CNN feature extractor. This design choice significantly reduces the memory footprint of the neural feature extractor, without compromising continual learning accuracy. • Streaming Architecture: We design a streaming dataflow FPGA architecture on the AMD Zynq ZCU104 that combines a layer-wise pipelined, tile-based quantized CNN front-end with a streaming hyperdimensional encoding and similarity engine operating on word-packed binary hypervectors, and a hardware-efficient cluster merge unit that bounds on-chip exemplar growth. • System Efficiency: We evaluate ImageHD on the Core-50 dataset, demonstrating 40.4\times and 4.84\times speedups and 383\times and 105.1\times energy efficiency gains over CPU and GPU baselines, respectively, validating its suitability for real-time edge adaptation. II On-Device Continual Learning: Motivation and Challenges II-A Definition On-device AI models refer to AI systems that are trained and deployed directly on edge devices. These models perform data processing and inference locally without the need to transmit data to the cloud [ 13 ] . On-Device Continual Learning (CL) extends this by requiring the model to not only run locally but to continuously update its knowledge using streaming data [ 18 ] . This allows the device to learn new tasks or adapt to changing environments in real-time. In the context of On-Device CL for images, the system processes a continuous input stream \mathcal{S}=\{(\mathbf{x}_{t},y_{t})\}_{t=1}^{\infty} , where \mathbf{x}_{t}\in\mathbb{R}^{H\times W\times C} represents raw pixel data. The objective is to map \mathbf{x}_{t} to a dynamic label set \mathcal{Y}_{t} while strictly adhering to hardware-imposed constraints [ 18 ] . II-B Motivation The growing volume of data motivates learning at the edge, as transmitting large volumes of