A Pilot Study on Tunable Precision Emulation via Automatic BLAS Offloading DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2025; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: General and reference Performance CCS:

Hang Liu, Junjie Li, Yinzhi Wang · 2025

Read A Pilot Study on Tunable Precision Emulation via Automatic BLAS Offloading DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2025; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: General and reference Performance CCS: by Hang Liu, Junjie Li, Yinzhi Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-22875.txt.

Introduction

Hang Liu, Junjie Li, and Yinzhi Wang pilot a way to speed heavy computing by faking high precision with fast low-precision hardware, in a 2025 study. Modern graphics chips excel at small-integer math, built for AI. Science codes, though, need double precision. The authors offload standard math library calls and emulate the precision on the fast units.

They tested the method on a materials science application with no code changes. Accuracy hinged on both the precision chosen and the traits of each operation. A tunable setting let them trade speed against exactness per operation. The approach keeps old algorithms intact while using new hardware well. It points to adaptive precision as a theme for future scientific computing.

Catalog record

CREATORHang Liu, Junjie Li, Yinzhi Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-22875

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Pilot Study on Tunable Precision Emulation via Automatic BLAS Offloading DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2025; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: General and reference Performance CCS: Mathematics of computing Mathematical software performance CCS: Software and its engineering Software performance Hang Liu, Junjie Li, Yinzhi Wang Abstract . This study explores the use of automatic BLAS offloading and INT8-based emulation for accelerating traditional HPC workloads on modern GPU architectures. Through the use of low-bitwidth integer units and cache-coherent Unified Memory Architecture, we emulate double-precision matrix multiplications in the MuST application without code changes. We find that accuracy depends on both arithmetic precision and the properties of the operator, which can be dealt with through tunable precision emulation. Unlike traditional mixed-precision approaches, this method preserves original algorithms while optimizing hardware utilization. We showcases the potential of improving accuracy and performance at the same time. This work highlights the potential of AI-driven hardware to transform HPC, advocating for adaptive precision strategies in future scientific computing. 1. Introduction Rapid advancement in artificial intelligence (AI) has triggered the development of specialized hardware, such as NVIDIA GPUs with Tensor Cores, AMD GPUs with Matrix Cores and various TPUs, to accelerate machine learning computations with high-throughput and low-precision arithmetic units ( Abdelfattah et al. 2021 ) . These processors take advantage of low/mixed-precision floating-point formats (e.g., FP16, BF16) and low-bitwidth integer units (e.g., INT8, INT4) to increase performance, reduce power consumption, and minimize memory footprint, as in the case of technologies like NVIDIA’s DP4A instruction and integer Tensor Cores ( Horowitz 2014 ) . While these architecture innovations have had a dramatic impact on AI training and inference performance, their potential to accelerate traditional HPC workloads that were historically based on CPUs and double-precision floating point (FP64) computations for scientific simulation, numerical modeling, and other compute-intensive tasks, remains vast yet not fully understood, requiring further research and development to unleash. FP64 has been the norm in traditional HPC, ensuring the numerical precision required for simulations like climate modeling, quantum chemistry simulation, and computational fluid dynamics. However, not all of the calculations in science demand that level of precision; lower precision may be enough in some cases without compromising result validity. Meanwhile, certain special computations, such as in quantum chemistry, can require higher precision than FP64. This level of precision needed provides an opportunity to rethink computational strategies in HPC. By offloading suitable workloads on GPUs and leveraging reduced-bit-width integer types like INT8, one can emulate FP64 computations at acceptable precision and utilize fewer computational resources while achieving a huge performance gain. For example, NVIDIA Tensor Cores are capable of performing INT8 matrix multiplications with INT32 accumulation at theoretical maximum performance rates 2 to 4 times higher than their floating point counterparts ( Ootomo et al. 2024 ) . Similarly, more recent processors such as Google TPUs and Intel AMX-INT8 demonstrate that integer acceleration for intensive matrix workloads is possible ( Jouppi et al. 2017 ) . The limitation of using GPU acceleration in traditional HPC workflows lies in the challenge of retrofitting legacy CPU-based applications, which are often deeply embedded within FP64-based numerical libraries and complex workflows. Manual porting is labor-intensive and not a viable choice for the majority of scientific communities, and existing automatic offloading capabilities have previously offered relatively modest performance improvement due to inefficiencies in traditional CPU-GPU memory architectures. However, recent innovations such as NVIDIA’s Grace-Hopper superchip’s Unified Memory Architecture (UMA) and AMD’s MI300A APU enable cache-coherent memory access between the GPU and CPU, ease data handling, and allow more transparent offloading strategies ( Li et al. 2024 ) . All these advances foreshadow a paradigm shift for HPC, away from CPU-centric FP64 computations toward adaptive precision strategies that leverage AI-driven hardware for both performance and accuracy. This crossroads of hardware innovation and modern memory architectures motivates our pilot study: examining whether automatically offloading Basic Linear Algebra Subprograms (BLAS) operations to GPUs, with tunable precision emulation using INT8, can maximize the efficiency of HPC workloads. By tuning the precision when needed, through the use of the integer representations, we can improve performance as well as the potential for better numerical precision for scientific computing. This work examines the application of automatic BLAS offloading ( Li et al. 2024 ) together with the Ozaki scheme on Integer Matrix Multiplication Units (IMMU) for the emulation of matrix multiplications in INT8 ( Ootomo et al. 2024 ; Uchino et al. 2025 ) . We experiment with a CPU application, MuST ( Wang et al. 1995 ) , through GPU offloading and emulation across various precision levels. The Ozaki scheme provides a general framework that we adapt here to exploit the accuracy of the application using the INT8 arithmetic units of GPUs. Our work demonstrates that MuST’s accuracy is not only determined by the arithmetic precisions, but also influenced by the operator’s analytic properties on its energy domain, revealing an accuracy sensitivity that can be addressed through tunable precision. This showcases the potential of improving accuracy without trading off performance by the automatic offloading and tunable precision emulation on modern GPU architectures. 2. Related Work 2.1. Previous Work on Automatic BLAS Offload Several attempts have been made to automatically offload BLAS calls from CPUs to GPUs with the rise of GPU adoption. Cray LIBSCI_ACC ( Hewlett Packard Enterprise 2024 ; Poxon 2013 ) pioneered the approach on the Titan supercomputer and supports offloading selected BLAS, LAPACK, and ScaLAPACK routines when the accelerated module is used. IBM’s ESSL ( IBM 2018 ) enables automatic offload of BLAS, LAPACK, and FFTW calls when binary is linked to libesslsmpcuda. NVIDIA’s NVBLAS ( NVIDIA 2024 ) used LD_PRELOAD to divert CPU BLAS calls to the GPU without relinking. These tools automate data movement and offload decisions. However, they are all designed for the conventional GPU architectures where data transfer between CPU and GPU is mandatory for every offloaded BLAS call, resulting in high overheads and heavy loss of performance in practical HPC applications ( Li et al. 2024 ; Li 2025 ) . A significant advancement comes from Li et al. ( Li et al. 2024 ; Li 2025 ) , who developed a tool for automatic BLAS offloading on NVIDIA Grace-Hopper superchip. Leveraging the cache coherency in UMA, the library intercepts compute intensive level-3 BLAS calls using a light-weight, trampoline-based Dynamic Binary Instrumentation (DBI) approach ( Wang and Li 2023 ) , and offers three data movement strategies tailored for the new GPU architecture: traditional CPU-GPU memory copies, cahce-coherent unified memory access, and an optimal migration scheme moving data upon GPU first use. Tested on real HPC applications, it achieves substantial performance gains without code changes, overcoming prior tools’ limitations and setting a new standard for BLAS offloading in HPC. 2.2. DGEMM Emulation on Integer Matrix Multiplication Unit The study of floating point emulation isn’t new. The most widely known emulation is emulating quad-precision using double-precision ( Hida et al. 2001 ) for those scientific applications that mandates precision beyond FP64. In recent years, a series of paper on the Ozaki scheme ( Ozaki et al. 2012 ; Ozaki et al. 2013 ; Mukunoki et al. 2020

Read the complete text