Leveraging Hardware-Aware Computation in Mixed-Precision Matrix Multiply: A Tile-Centric Approach

Qiao Zhang, Rabab Alomairy, Dali Wang, Zhuowei Gu, Qinglei Cao · 2025

Read Leveraging Hardware-Aware Computation in Mixed-Precision Matrix Multiply: A Tile-Centric Approach by Qiao Zhang, Rabab Alomairy, Dali Wang, Zhuowei Gu, Qinglei Cao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-14848.txt.

Introduction

Qiao Zhang and co-authors speed up a core math task, matrix multiplication, by mixing number precisions tile by tile, in a 2025 paper. New hardware favors low-precision math, which is faster and thriftier but less exact.

Their framework lets each small block of a big multiplication use the precision that suits it. A runtime system spreads the work across different machines and balances the load. The team ran it on three supercomputers in Japan and the United States, based on different chip families. Performance scaled well on all three. The authors aim to pair smart algorithms with modern hardware for gains in both speed and energy use.

Catalog record

CREATORQiao Zhang, Rabab Alomairy, Dali Wang, Zhuowei Gu, Qinglei Cao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-14848

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Leveraging Hardware-Aware Computation in Mixed-Precision Matrix Multiply: A Tile-Centric Approach Qiao Zhang, Rabab Alomairy, Dali Wang, Zhuowei Gu, Qinglei Cao Abstract General Matrix Multiplication (GEMM) is a critical operation underpinning a wide range of applications in high-performance computing (HPC) and artificial intelligence (AI). The emergence of hardware optimized for low-precision arithmetic necessitates a reevaluation of numerical algorithms to leverage mixed-precision computations, achieving improved performance and energy efficiency. This research introduces an adaptive mixed-precision GEMM framework that supports different precision formats at fine-grained tile/block levels. We utilize the PaRSEC runtime system to balance workloads across various architectures. The performance scales well on ARM CPU-based Fugaku supercomputer, Nvidia GPU-based A100 DGX, and AMD GPU-based Frontier supercomputer. This research aims to enhance computational efficiency and accuracy by bridging algorithmic advancements and hardware innovations, driving transformative progress in various applications. 1 Introduction General Matrix Multiplication (GEMM) lies at the core of linear algebra and acts as a critical kernel in numerous High-Performance Computing (HPC) applications—such as earthquake simulation [ 1 ] and weather and climate prediction [ 2 ] —and in Artificial Intelligence (AI) applications, including fully connected layers in traditional neural networks and natural language processing [ 3 ] . GEMM is a computationally expensive operation due to its high demands on both memory and computation, whose performance directly affects the efficiency of these applications, making it a focal point for optimization on diverse architectures, ranging from homogeneous CPU clusters to heterogeneous GPU systems. Over the last ten years, hardware designers have prioritized crafting processors that excel in high-speed, energy-efficient, and reduced-precision computations [ 4 ] . A notable example is the widespread use of Nvidia GPUs in both HPC and AI tasks, driven by their exceptional processing capabilities and efficiency. The latest Nvidia GPUs, including the H100 and the upcoming B100, incorporate a range of precision formats, each influencing peak theoretical throughput differently. The lowest precision levels (FP8/INT8 on H100 and FP4 on B100) achieve speedups of 58\times and 233\times , respectively, relative to FP64 computations, underscoring the dramatic acceleration possible when shifting from conventional double-precision arithmetic to Tensor Core-accelerated lower-precision execution. A similar evolution is observed in CPUs, where FP64 performance is typically double that of FP32 and quadruple that of FP16. This transition has motivated researchers to reassess classical numerical techniques, identifying areas where reduced precision is sufficient without compromising the overall quality of solutions. Additionally, leveraging lower precision not only diminishes memory demands—allowing more data to fit within high-speed memory and thereby improving execution rates [ 5 , 6 ] —but also cuts down on energy consumption, making it a compelling strategy for boosting energy efficiency in computational workflows [ 7 ] . The synergy between hardware advancements and algorithmic refinements plays a pivotal role in tackling complex computational challenges. Mixed-precision computations, which involve utilizing multiple numerical precisions within a single task, have emerged as a powerful strategy [ 8 , 9 ] . This methodology has been successfully integrated into numerous scientific and engineering disciplines, such as deep learning [ 10 ] , computational fluid dynamics [ 11 , 12 ] , astronomy [ 13 ] , climate modeling [ 14 ] , molecular dynamics [ 15 ] , and quantum mechanics [ 16 ] . The primary advantage of these techniques is their ability to harness lower-precision hardware capabilities to enhance computational speed while preserving necessary accuracy constraints. Nevertheless, most existing implementations rely on coarse-grained precision adjustments and are optimized for single-GPU or single-node setups, which limits their applicability to large-scale distributed environments. Efficiently handling mixed-precision computations in distributed multi-GPU architectures introduces significant hurdles due to disparities in data representation and precision consistency. Moreover, many existing solutions are tailored to specific hardware platforms, reducing their portability across diverse computing architectures. This paper introduces a mixed-precision GEMM framework (GEMM-MP) at fine-grained tile/block levels and integrates GEMM-MP into the runtime system PaRSEC. The contributions of this paper are as follows: • Introducing a tile-centric mixed-precision GEMM algorithm that carefully balances accuracy and performance, enabling a robust and trustworthy mixed-precision solution for HPC and AI applications. • Leveraging a dynamic task-based runtime system to efficiently orchestrate heterogeneous tasks, dynamically adapting to workload variations while minimizing scheduling overhead and maximizing resource utilization. • Demonstrating strong scalability and portability across a diverse range of homogeneous and heterogeneous computing platforms, ensuring broad applicability and efficiency. To the best of our knowledge, this is the first work to introduce a tile-centric mixed-precision GEMM algorithm. The structure of this paper is as follows. Section 2 discusses prior research relevant to our work. In Section 3 , we provide an overview of PaRSEC. The tile-centric mixed-precision GEMM framework is described in Section 4 . A performance evaluation is then conducted in Section 5 , and finally, Section 6 outlines our conclusions and future directions. 2 Related Work 2.1 Mixed-Precision Matrix Computations Mixed-precision approaches have garnered extensive interest [ 8 , 9 ] and are widely applied across multiple computational fields [ 10 , 11 , 12 , 13 , 14 , 15 , 16 ] . A well-known example is iterative refinement techniques for linear solvers, where the system is initially solved using lower precision, followed by higher precision corrections applied iteratively to refine the residual. This strategy has demonstrated significant reductions in computational time and energy consumption, making it an effective approach for enhancing efficiency in large-scale computations [ 7 , 17 ] . However, the effectiveness of such methods is highly dependent on the matrix condition number, and the requirement to store multiple precision copies of matrices can introduce substantial memory overhead. Ongoing research explores performance optimization while ensuring numerical reliability by selectively employing reduced precision in non-critical computations while preserving higher precision where necessary [ 18 , 2 , 19 ] . These methodologies primarily target linear system solutions. In deep learning, reduced precision is typically applied to forward and backward computations, whereas weight updates often require higher precision accumulation to maintain numerical stability. This trade-off, however, may lead to accuracy degradation due to precision loss in key computations of forward and backward propagation. In this work, we introduce an adaptive tile-centric mixed-precision GEMM framework that dynamically adjusts precision levels within a single computation. This fine-grained approach can provide another solution to ensure that accuracy remains uncompromised by recognizing that not all computational components require the same precision level in HPC and applications. 2.2 Runtime Systems Recent advancements in task-based programming models have driven the scalability of scientific applications on modern supercomputers. A variety of runtime systems have been developed to facilitate efficient task execution, optimizing resource utilization across diverse architectures. OpenMP [ 20 ] has emerged as a widely adopted standard for shared-memory parallelism, incorporating tasking constructs that provide dynamic scheduling and fine-grained control. By leveraging compiler directives and runtime support, OpenMP simplifies concurrency management and enhances performance in multi-threaded environments. Building upon OpenMP, OmpSs [ 21 ] extends capabilities for heterogeneous architectures, introducing advanced dependency tracking and asynchronous execution strategies. Similarly, COMPSs [ 22 ] provides a comprehensive programming environment, streamlining task parallelism across distributed computing infrastructures. For heterogeneous, distributed computing platforms, StarPU [ 23 ] delivers a flexible runtime system that integrates scheduling mechanisms with explicit kernel annotations, optimizing execution

Read the complete text