Accelerating HDC-CNN Hybrid Models Using Custom Instructions on RISC-V GPUs
Wakuto Matsumi 1st and Riaz-Ul-Haque Mian 2nd · 2025
Read Accelerating HDC-CNN Hybrid Models Using Custom Instructions on RISC-V GPUs by Wakuto Matsumi 1st and Riaz-Ul-Haque Mian 2nd (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-05053.txt.
Introduction
Wakuto Matsumi and Riaz-Ul-Haque Mian add custom commands to a RISC-V graphics chip to speed mixed models that join hyperdimensional computing with neural nets. Hyperdimensional methods are light and parallel but weak on hard image tasks, while neural nets are strong yet costly.
RISC-V chips are open and can be reshaped for niche work, unlike locked designs. The authors built four custom commands tuned for hyperdimensional steps and ran hybrid image workloads on them. Their tests check how far such tailored hardware can cut the cost of running these mixed models.
Catalog record
| CREATOR | Wakuto Matsumi 1st and Riaz-Ul-Haque Mian 2nd |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-05053
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Accelerating HDC-CNN Hybrid Models Using Custom Instructions on RISC-V GPUs Thanks: This study do not receive any particular fund Wakuto Matsumi 1st and Riaz-Ul-Haque Mian 2nd Abstract Accelerating HDC-CNN Hybrid Models on custom GPUs. Machine learning based on neural networks has been advancing rapidly. However, the energy consumption required for training and inference remains substantial, posing significant challenges in environments where energy efficiency is a critical concern. Hyperdimensional Computing (HDC) is an extremely lightweight machine learning approach designed to mimic the mechanisms of the human brain. While HDC exhibits a high degree of parallelism, its recognition accuracy for complex images is relatively low. To address this problem, accelerators that combine HDC and neural networks, such as Convolutional Neural Networks (CNN) have been proposed. However, their adoption is hindered by limitations in generalizability and programmability. Meanwhile, the emergence of RISC-V has accelerated the development of open-source accelerators specialized for specific domains . Traditionally, GPUs have been developed in a closed ecosystem, but there is a growing trend toward open-source GPU architectures, leading to the introduction of multiple RISC-V-based GPUs . Like RISC-V CPU RISC-V GPU also have the potential to serve flexibility in custom HDC where conventional GPU are not design. Just as RISC-V CPUs have become the foundation for domain-specific accelerators, RISC-V GPUs have the potential to serve as a flexible and programmable platform, enabling efficient high-speed computation even in areas where conventional GPUs are less effective. In this study, we implement custom instructions for GPU-based HDC operations , establishing a computational framework that efficiently handles processing tasks using HDC. This approach enhances the applicability of HDC-CNN hybrid models, addressing the challenges of energy-efficient, high-performance computing. Our experimental analysis with four different custom instructions for HDC operations shows that it can achieve a speedup of up to 56.2 times when using a microbenchmark for bound operations. I Introduction Fig. 1: Overview of HDC-CNN Hybrid Models Machine learning models based on neural networks have advanced significantly, but their high computational and energy costs pose challenges for efficient deployment. Hyperdimensional Computing (HDC) offers a lightweight, brain-inspired alternative that excels in parallelism and low power consumption. However, HDC alone struggles with complex image recognition tasks, leading to the development of HDC-CNN hybrid models that leverage Convolutional Neural Networks (CNNs) for improved accuracy. Despite these advancements, traditional GPUs face memory bottlenecks when processing HDC, limiting overall efficiency. In this work, we propose a custom GPU architecture based on RISC-V with specialized HDC instructions to accelerate HDC operations. The proposed approach enhances computational efficiency and enables seamless integration with CNNs, achieving significant speedup in hybrid model processing. Hyperdimensional Computing (HDC) is an emerging lightweight machine learning paradigm inspired by the human brain. It encodes data into high-dimensional vectors, enabling massively parallel operations while reducing computational complexity and power consumption [ 11 ] . Due to its efficiency, HDC is well-suited for edge computing applications. However, its recognition accuracy for complex datasets remains a challenge, leading to the development of HDC-CNN hybrid models that integrate Convolutional Neural Networks (CNNs) to enhance performance [ 4 ] . Despite these advancements, executing HDC efficiently on traditional CPUs and GPUs is difficult due to their inherent architectural limitations. This necessitates the design of custom hardware accelerators that can efficiently process both HDC and hybrid models, motivating further research into specialized architectures [ 10 ] . Figure 1 illustrates the processing flow of the HDC-CNN hybrid model. The processing is divided into feature extraction by CNN and feature classification by HDC. Existing CNNs such as VGG, ResNet, and MobileNet are used as feature extractors up to the first pooling layer. HDC is used as the feature classifier. The features output from the feature extractor are encoded into Hyper Vectors (HVs). A Bound operation is performed on the encoded HVs for each classification class to create an HV representing the average feature. The test data is also encoded into HV in the same way, and the class with the highest similarity to the HV representing the average feature is the classification result. This research aims to propose a GPU-based platform capable of efficiently processing HDC by implementing custom instructions to support HDC operations on GPUs. This research will enable the high-speed execution of hybrid models combining HDC with other machine learning techniques, making it easier to validate related studies. The key contributions of this paper are as follows: • We propose a custom GPU architecture based on RISC-V to efficiently accelerate Hyperdimensional Computing (HDC) operations, enabling seamless integration with Convolutional Neural Networks (CNNs) in HDC-CNN hybrid models. • We introduce four custom GPU instructions tailored for HDC operations, significantly reducing memory access overhead and improving computational efficiency. • We implement and evaluate the proposed architecture on the Vortex RISC-V-based GPU , demonstrating a maximum speedup of 56.2x for microbenchmark-bound operations. • We analyze the performance bottlenecks in HDC-CNN hybrid models and identify encoding as a major limiting factor, highlighting the need for further optimization in matrix operations. • We provide a comprehensive evaluation comparing execution cycles before and after implementing custom GPU instructions, offering insights into their impact on real-world machine learning workloads. The rest of the paper is organized as follows. Section II provides the necessary background and discusses related work on Hyperdimensional Computing (HDC) and its integration with Convolutional Neural Networks (CNNs). Section II-B introduces the fundamental concepts and computational techniques used in HDC, highlighting the key challenges in its hardware acceleration. Section IV presents the proposed custom GPU architecture, detailing the implementation of specialized HDC-specific instructions and optimization strategies. Section V describes the experimental setup, evaluation methodology, and benchmark results, demonstrating the efficiency of our approach. Finally, Section VI concludes the paper by summarizing the key findings and discussing potential future research directions. II Background To address the limitations of conventional neural network-based models in terms of computational efficiency and energy consumption, alternative learning paradigms and specialized architectures have been explored. The following sections discuss Hyperdimensional Computing (HDC) as a lightweight machine learning approach and the role of domain-specific architectures, particularly RISC-V-based GPUs, in accelerating such models. II-A Hyperdimensional Computing Neural network-based machine learning has been rapidly advancing. In particular, the progress in generative AI has been remarkable, and a development race fueled by massive computational resources is underway, based on scaling laws [ 13 ] . The forward and backward propagation used in neural network training and inference can be represented as matrix operations. Therefore, achieving fast matrix operations is essential for the rapid training and inference of neural networks. GPUs, which can efficiently execute matrix operations in parallel and are also capable of handling general-purpose tasks, are well-suited for neural network processing and are extensively used in large-scale AI development. While high-accuracy models using neural networks have been developed, these models often come with a large number of parameters, making them computationally intensive and challenging to deploy on edge devices. GPUs, which are capable of general-purpose parallel processing, suffer from low power efficiency, making them unsuitable for running high-accuracy, parameter-heavy models in power-constrained environments. To achieve higher power efficiency, many accelerators have been proposed, but their architectures differ from traditional processors, leaving challenges in terms of programmability. One proposed solution to these challenges is Hyperdimensional Computing