MXDOTP: A RISC-V ISA Extension for Enabling Microscaling (MX) Floating-Point Dot Products
Gamze İslamoğlu1, Luca Bertaccini1, Arpan Suravi Prasad1, Francesco Conti2, Angelo Garofalo2, Luca Benini12 · 2025
Read MXDOTP: A RISC-V ISA Extension for Enabling Microscaling (MX) Floating-Point Dot Products by Gamze İslamoğlu1, Luca Bertaccini1, Arpan Suravi Prasad1, Francesco Conti2, Angelo Garofalo2, Luca Benini12 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13159.txt.
Introduction
Gamze İslamoğlu and co-authors add a hardware instruction that lets a RISC-V processor multiply in the compact microscaling number format, in a 2025 paper. AI workloads favor tiny eight-bit numbers grouped under one shared scale per block, but general processors handle them clumsily. Their MXDOTP extension gives the open-source Snitch core a dedicated dot-product unit that consumes eight packed values at once.
Built in a modern 12-nanometer process, an eight-core cluster reached high efficiency on matrix work while using the streaming registers to keep the unit fed. Against a software workaround it ran twenty-five times faster and used far less energy, for a small area cost. The authors target energy-tight AI chips.
Catalog record
| CREATOR | Gamze İslamoğlu1, Luca Bertaccini1, Arpan Suravi Prasad1, Francesco Conti2, Angelo Garofalo2, Luca Benini12 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13159
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MXDOTP: A RISC-V ISA Extension for Enabling Microscaling (MX) Floating-Point Dot Products † † thanks: This work is supported in part by the NeuroSoC project funded under Horizon Europe Grant Agreement n° 101070634. Gamze İslamoğlu1, Luca Bertaccini1, Arpan Suravi Prasad1, Francesco Conti2, Angelo Garofalo2, Luca Benini12 Abstract Fast and energy-efficient low-bitwidth floating-point (FP) arithmetic is essential for Artificial Intelligence (AI) systems. Microscaling (MX) standardized formats have recently emerged as a promising alternative to baseline low-bitwidth FP formats, offering improved accuracy with a block-wise shared exponent scale combined with per-element values. However, efficiently executing the key linear algebra primitives for AI applications on MX formats requires specialized hardware support for the fundamental operators such as scaled dot product. In this work, we propose MXDOTP , the first RISC-V ISA extension for MX dot products, focusing on the 8-bit MXFP8 FP format. We extend the open-source Snitch RISC-V core with a dedicated MXFP8 dot product-accumulate unit, which fully consumes blocks of eight 8-bit operands packed into 64-bit inputs. To feed MXDOTP at full utilization with four operands per cycle, including block scales, we exploit Snitch’s Stream Semantic Registers (SSRs), achieving up to \qty [detect-all=true]80 utilization with minimal impact on the Snitch core’s architecture and no modification to the register file. Implemented in \qty [detect-all=true]12 \nano FinFET, a cluster with eight MXDOTP -extended cores reaches up to \qty [detect-all=true]356 \giga FLOPS \per when computing MXFP8 matrix multiplications at \qty [detect-all=true]0.8, \qty [detect-all=true]1 \giga . Compared to a software baseline, where MX dot products are computed by type casting FP8 inputs to FP32 for higher accumulation precision and applying explicit block scaling, the cluster achieves \qty [detect-all=true,mode=text]25× speedup and \qty [detect-all=true]12.5× better energy efficiency at a minimal \qty [detect-all=true]5.1 area increase. \AddToShipoutPictureFG * This paper has been accepted for publication by the 2025 IEEE 36th International Conference on Application-specific Systems, Architectures and Processors (ASAP). ©2025 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. I Introduction Compact floating-point (FP) arithmetic has gained significant attention for energy-efficient computing, particularly in machine learning workloads [ 1 , 2 , 3 , 4 ] . Deep learning models, like Large Language Models (LLMs) , demand massive computational and memory resources, making both compute efficiency and memory bandwidth key concerns. In low-bitwidth formats like FP8, increasing the exponent range improves dynamic range but reduces precision, while a larger mantissa improves precision at the cost of a smaller dynamic range. Block formats mitigate this hard tradeoff with a shared exponent, preserving both range and precision. While block floating-point (BFP) formats have been widely studied [ 1 ] , Microscaling (MX) formats offer greater flexibility with a two-level exponent representation, where groups of FP elements share a wide exponent while each low-bitwidth element retains a narrower individual exponent to achieve finer-grained scaling. This reduces precision loss while retaining the advantages of low-bitwidth FP . MX formats have demonstrated high accuracy in workloads such as Convolutional Neural Networks (CNNs) , LLMs , and Vision Transformers (ViTs) [ 2 ] , leading to their adoption in a standard specification proposal developed by the Open Compute Project (OCP) [ 5 ] . Low-bitwidth formats significantly reduce memory and bandwidth needs by representing model parameters with fewer bits. However, fully realizing their efficiency gains requires native hardware support for scaled dot products, which dominate execution time in deep learning models. In large-scale data centers, reducing memory footprint and bandwidth is often prioritized, even at the cost of decompression overhead. In contrast, embedded Artificial Intelligence (AI) systems face strict power and compute constraints, where the overhead of format conversion can outweigh its benefits. This necessitates direct execution on MX formats to maximize efficiency. NVIDIA’s Blackwell Tensor Cores introduce micro-tensor scaling, incorporating community-defined MX formats [ 6 ] . Similarly, AMD’s Versal AI Edge Series Gen 2 employs MX6 and MX9 formats in its AI engines [ 7 ] . In the RISC-V ecosystem, Tenstorrent’s Tensix cores support BFP formats [ 8 ] , but unlike MX , they rely solely on shared exponents per block, lacking the finer-grained scaling of MX (except for MXINT8). All these solutions are proprietary and tightly coupled to specialized tensor processing units. In contrast, our approach enables flexible and efficient execution of MX computations on RISC-V cores without dedicated tensor accelerators. Existing RISC-V architectures lack native support for efficient execution of mixed-precision MX dot products due to the need for concurrent access to multiple types of data—scales, vector elements, and accumulators. Handling these dependencies in software requires frequent format conversions and explicit scaling operations, causing significant latency and energy overhead. To address these limitations, we propose MXDOTP , a dedicated ISA extension that natively integrates scaling within a dot product-accumulate operation. By eliminating redundant data movement and format conversions, MXDOTP enables high-throughput, energy-efficient execution of MX matrix multiplications. The contributions of this paper are: • We design a dot product-accumulate unit to support MX FP8 (E5M2, E4M3) data formats with 8-bit scales and an FP32 accumulator to minimize overflows and precision losses. • We introduce MXDOTP , a RISC-V ISA extension that fuses scaling and dot product into a single four-operand instruction. To sustain four operand fetches per cycle, we exploit Stream Semantic Registers (SSRs) in the Snitch core [ 9 ] to supply block scales without modifying the register file, which is typically limited to three read ports. • We integrate MXDOTP into an 8-core Snitch cluster, reaching up to \qty [detect-all=true]80 utilization. Implemented in \qty [detect-all=true]12 \nano FinFET, it incurs only \qty [detect-all=true]11 core-level and \qty [detect-all=true]5.1 cluster-level area, and \qty [detect-all=true]1.9 power overhead when idle. • We demonstrate a \qty [detect-all=true,mode=text]25× speedup and \qty [detect-all=true]12.5× improvement in energy efficiency over software-based MX matrix multiplication on the baseline cluster. Figure 1 : (a) Simplified MXDOTP datapath (sign bits omitted). (b) Integration of MXDOTP into the Snitch core. (c) Execution of MXDOTP over multiple cycles with three pipeline stages. SSRs enable efficient operand streaming, ensuring full utilization without stalls. II Background II-A Microscaling (MX) Formats MX formats, proposed by OCP as a standard, are low-bitwidth data formats optimized for AI, co-developed by major industry players such as AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm. These formats aim to enhance computational efficiency and memory footprint while maintaining accuracy in deep learning models. Unlike traditional per-tensor scaling or BFP , which share a single exponent across large blocks, MX offers finer-grained scaling by allowing individual elements or smaller blocks to retain independent scales. This reduces numerical errors while preserving memory efficiency. MX serves as a drop-in replacement for FP32 in inference and enables sub-8-bit training for large generative models with negligible accuracy loss [ 2 ] . An MX -compliant format consists of three components: a scale factor , private elements , and a block size . The specification defines four concrete formats: MXFP8, MXFP6, MXFP4, and MXINT8. Each block of k elements shares a scale factor, while elements are encoded in