Tensor Program Optimization for the RISC-V Vector Extension Using Probabilistic Programs
Federico Nicolás Peccia, Frederik Haxel, Oliver Bringmann · 2025
Read Tensor Program Optimization for the RISC-V Vector Extension Using Probabilistic Programs by Federico Nicolás Peccia, Frederik Haxel, Oliver Bringmann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-01457.txt.
Introduction
Federico Nicolás Peccia, Frederik Haxel, and Oliver Bringmann speed up tensor programs on RISC-V vector chips, in a 2025 paper. The catalog title adds a funding note, so the head title governs. The open chip design scales from tiny devices to clusters. Its vector unit suits AI work, but few tuning tools target it.
Their flow plugs the vector unit into a compiler's probabilistic tuning frame instead of hand written libraries. On test chips built on a programmable board, tuned code ran about forty six percent faster than a standard compiler's auto vector output. It beat a hand library too, with smaller code size. A real commercial chip showed the same flow finds good mappings.
Catalog record
| CREATOR | Federico Nicolás Peccia, Frederik Haxel, Oliver Bringmann |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-01457
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Tensor Program Optimization for the RISC-V Vector Extension Using Probabilistic Programs Thanks: This research was funded by German Federal Ministry of Education and Research within the project ”GreenEdge-FuE”, funding no. 16ME0517K. Federico Nicolás Peccia, Frederik Haxel, Oliver Bringmann Abstract RISC-V provides a flexible and scalable platform for applications ranging from embedded devices to high-performance computing clusters. Particularly, its RISC-V Vector Extension (RVV) becomes of interest for the acceleration of AI workloads. But writing software that efficiently utilizes the vector units of RISC-V CPUs without expert knowledge requires the programmer to rely on the autovectorization features of compilers or hand-crafted libraries like muRISCV-NN. Smarter approaches, like autotuning frameworks, have been missing the integration with the RISC-V RVV extension, thus heavily limiting the efficient deployment of complex AI workloads. In this paper, we present a workflow based on the TVM compiler to efficiently map AI workloads onto RISC-V vector units. Instead of relying on hand-crafted libraries, we integrated the RVV extension into TVM’s MetaSchedule framework, a probabilistic program framework for tensor operation tuning. We implemented different RISC-V SoCs on an FPGA and tuned a wide range of AI workloads on them. We found that our proposal shows a mean improvement of 46% in execution latency when compared against the autovectorization feature of GCC, and 29% against muRISCV-NN. Moreover, the binary resulting from our proposal has a smaller code memory footprint, making it more suitable for embedded devices. Finally, we also evaluated our solution on a commercially available RISC-V SoC implementing the RVV 1.0 Vector Extension and found our solution is able to find mappings that are 35% faster on average than the ones proposed by LLVM. We open-sourced our proposal for the community to expand it to target other RISC-V extensions. I Introduction The execution of AI models is nowadays a task that permeates all computing levels, from High Performance Computing (HPC) servers to embedded devices. Given its open-source nature, scalability, and broadly ratified extensions, the RISC-V ISA presents itself as an ideal candidate to accelerate the execution of these AI workloads across this wide range of hardware platforms. Since its Vector Extension (RVV) has been ratified, numerous commercial [ 1 , 2 ] and research platforms [ 3 , 4 , 5 ] have added support for it. But although the hardware is now available, the software support to deploy AI workloads that efficiently use the vector unit is still lacking. Even though compilers like GCC and LLVM provide autovectorization features, these do not always use the vector unit as efficiently as manually programmed kernels [ 6 ] . Another option is to use libraries of hand-crafted kernels like muRISCV-NN [ 7 ] , which have been demonstrated to provide higher speedups than the compiler’s autovectorization. But these libraries do not adapt to changes in the hardware: for example, there is no guarantee that a kernel written for a RISC-V CPU with a 1 Mb L2 cache will still be the best option for one with a bigger L2 cache, where the scheduling of the AI workload could benefit from different data reuse. This gets even more complicated given that vector units from different vendors can expose different performances because of differences in their microarchitecture implementation. This is where tuning the AI workload for the particular target hardware using frameworks like AutoTVM [ 8 ] or MetaSchedule [ 9 ] becomes advantageous. But so far, the RISC-V RVV extension has been missing from these kinds of frameworks, heavily limiting the deployment of AI workloads on RISC-V vector units. In this paper, we extended the MetaSchedule framework of TVM with tensor intrinsics that make use of the RISC-V RVV extension. These intrinsic, together with the probabilistic sampling of the possible mappings provided by MetaSchedule enable an efficient exploration of the design space of possible schedule candidates of each tensor operation onto the RISC-V vector unit. To evaluate our proposal, instead of relying on simulators, we perform a broad study implementing different versions of a RISC-V System-on-Chip (SoC), together with a vector unit that supports the RVV 1.0 extension, on an AMD ZCU102 FPGA. We tuned multiple AI workloads on each hardware version and compared our work against muRISCV-NN and the autovectorization feature from GCC 14. We found that, for all the evaluated hardware configurations, our proposal outperforms both of them. We also analyzed instruction traces to verify that our schedules utilize the vector register file more efficiently than muRISCV-NN. We even evaluated vector units with different hardware parameters and confirmed that our solution still finds better schedules than the other approaches for each hardware version. Then, to demonstrate that our integration is also able to target commercially available boards, we tuned several AI workloads on the Banana Pi BPI-F3 board, which provides an octa-core RISC-V SoC with 256-bit RVV 1.0 compatible vector units. We used the full capabilities of the TVM runtime to execute these and found that our tuned workloads are better than the ones executed using plain TVM (enabling the autovectorization of LLVM 19). This work demonstrates the advantages of probabilistic program exploration for the acceleration of AI workloads using the RISCV RVV extension. The open-source nature of our proposal will allow other works to expand this approach using other RISCV extensions, for example, Packed SIMD. In addition, our integration is able to target both the microTVM runtime (for embedded devices running bare metal or an RTOS) and the full TVM runtime (for more capable embedded devices or servers). This will enable the deployment of AI workloads on a wide range of RISC-V platforms implementing the RVV extension. The rest of this paper is organized as follows. First, Section II presents the current options for developers to map AI workloads onto the vector units of RISC-V CPUs. Then, Section III presents our extension to the MetaSchedule framework to enable the exploration of different mapping possibilities of these AI workloads using the RISC-V RVV extension. In Section IV , we demonstrate that the mappings found by our proposal surpass the ones from previous works on a variety of hardware configurations. Finally, Section V concludes the work and discusses future research directions. II Related work The RISC-V RVV 1.0 Vector Extension introduces a flexible vector processing model in order to support a wide range of applications, from digital signal processing (DSP), graphics or AI workloads, and targets platforms ranging from embedded systems to high-performance computing (HPC). It provides 32 vector registers, each up to VLEN bits wide. Using a combination of SEW (Selected Element Width, the actual width of each element in the vector to be processed) and LMUL (Vector Register Group Multiplier, which allows the programmer to group multiple vector registers together to process longer sequences of elements) the program can define, during runtime, how many elements each vector instructions is actually going to operate on (VL, or Vector Length). In order to accelerate applications using this RISC-V extension, programmers can always write their program and insert the specific assembly instructions to offload computation to the vector unit. But this is cumbersome and requires a lot of expertise. This is why GCC provides C intrinsics to abstract the programmer from the actual RVV instructions call. But still, writing its own code