Hardware for converting floating-point to the microscaling (MX) format
Danila Gorodecky, Leonel Sousa · 2024
Read Hardware for converting floating-point to the microscaling (MX) format by Danila Gorodecky, Leonel Sousa (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-03149.txt.
Description
Abstract This paper proposes hardware converters for the microscaling format (MX-format), a reduced representation of floating-point numbers. We present an algorithm and a memory-free hardware model for converting 32 single-precision floating-point numbers to MX-format. The proposed model supports six different types of MX-format: E5M2, E4M3, E3M2, E2M3, E2M1, and INT8.
Catalog record
| CREATOR | Danila Gorodecky, Leonel Sousa |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-03149
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hardware for converting floating-point to the microscaling (MX) format Danila Gorodecky, Leonel Sousa Abstract This paper proposes hardware converters for the microscaling format (MX-format), a reduced representation of floating-point numbers. We present an algorithm and a memory-free hardware model for converting 32 single-precision floating-point numbers to MX-format. The proposed model supports six different types of MX-format: E5M2, E4M3, E3M2, E2M3, E2M1, and INT8. The conversion process consists of three steps: calculating the maximum absolute value among 32 inputs, generating a shared scale, and producing 32 outputs in the selected MX-format type. The hardware converters were implemented in FPGA, and experimental results demonstrate. I Introduction and Background The Microscaling (MX) number representation was proposed in 2023 [ 1 , 2 ] through a collaboration between AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm. This format aims to minimize hardware costs and storage requirements via reduced bit-width, while maintaining the efficiency and accuracy needed for machine learning training and inference tasks. MX-formats provide a unified representation for half-precision (FP16), single-precision (FP32), BFloat16, and others formats by using truncated binary formats with a sign bit, up to 5 bits for the exponent, and up to 6 bits for the mantissa. A key feature of the MX-format is its use of a shared scale factor that represents the exponent of the floating-point format. MX-format is the modified floating-point representation with w -bit shared scale X (represented in exponent format) and privet n element P_{i} , where i=1,2,\dots,n . Fig. 1 shows the mapping of 32-bits floating point numbers (FP32) V_{1},V_{2},\dots,V_{32} to MX-format X,P_{1},P_{2},\dots,P_{32} . Fig. 1: The ratio between FP32 and MX-format: S_{1},S_{2},\dots,S_{32} - sign bits, EV_{1},EV_{2},\dots,EV_{32} - exponent and MV_{1},MV_{2},\dots,MV_{32} - mantissa of V_{1},V_{2},\dots,V_{32} , X - shared scale, EK_{1},EK_{2},\dots,EK_{32} - exponent and MR_{1},MR_{2},\dots,MR_{32} - mantissa of P_{1},P_{2},\dots,P_{32} , where K and R - nimber of bits of exponent and mantissa, respectively, of MX-format MX-format is characterized by the w -bit shared scale X and private elements P_{1},P_{2},\dots,P_{n} . This paper considers six types of MX-format. The bit-with of the shared scale depends on the type of MX. The general form of a type is represented as EKMR, where ”K” and ”R” are the number of bits of exponent (EK on Fig. 1 ) and mantissa (MR on Fig. 1 ) respectively. The sign bit (S on Fig. 1 ) is mentioned by default. This paper considers the following types of MX-format: E5M2, E4M3, E3M2, E2M3, E2M1, and INT8, which are the ones currently considered for possible standardization. The bit range of the shared scale X for FP32 in the mentioned types of MX-format is 8, i.e. w=8 . For example, for n=2 and E5M2, V_{1},V_{2} in FP32 correspond to two private elements P_{1},P_{2} that consist of 8-bit, i.e. 1 sign bit, 5-bit exponent, and 2-bit mantissa each. The bit-width of the shared scale, sign bit, exponent, and mantissa for the considered formats are shown in Table I . TABLE I: Bit-width of the shared scale ( X ), sign bit S , exponent EK , and mantissa MR for E5M2, E4M3, E3M2, E2M3, E2M1, and INT8 X S EK MR E5M2 5 2 E4M3 4 3 E3M2 8 1 3 2 E2M3 2 3 E2M1 2 1 INT8 1 6 According to the original mechanism [ 1 , 2 ] , the converting consists of two steps: 1. calculating X ; 2. generating quantized outputs P_{1},P_{2},\dots,P_{n} . The shared scale X can represent NaN, but not infinity. X value is treated separately from the private elements. Even when X is NaN, the private elements can be anything (normal number, infinity, and NaN). However, in backward transformation, the resulting values will all be NaN if X is NaN, because multiplication of NaN to a value equals NaN. This paper proposes an algorithm and a hardware architecture for converting FP32 to MX formats. The proposed models were implemented on FPGA, with experimental results obtained after synthesis and place-and-route. In [ 3 ] , a converter is described that processes input into blocks of MX values using ’round to nearest even’ for FP32 and BFloat16 and focuses on a pipelined (memory-based) design. In contrast, the approach in this paper uses combinational logic (i.e., without memory) and applies standard IEEE 754 rounding. The rest of the paper is organized as follows. Section II will consider the converting algorithm from FP32 to MX-format and quantization of the considered MX-formats with the examples. Section III will represent the structure of the hardware converter. This will be followed by the results of the synthesis in section IV. Finally, the conclusions are presented in section V. II Converting algorithm FP32 to MX-formats The procedure of the transformation reminds the original one, but it splits the first step into two steps, where on the first step the largest power-of-two among all inputs is calculated, and then the largest value is tis defined, thus transformed to X (see Table II ). As a result, the procedure of the transformation is the following: 1. identifying the largest power of two, denoted as max(|V_{i}|) , among all inputs V_{1},V_{2},\dots,V_{n} . The largest power of two is determined by examining the exponents EV_{1},EV_{2},\dots,EV_{n} , so that max(|V_{i}|)=max|\{EV_{i}\}| . This maximum value is calculated by comparing pairs of values in a hierarchical manner: first, EV_{1} with EV_{2} , EV_{3} with EV_{4} , and so on, up to EV_{31} with EV_{32} . At each level, the larger value in each pair is compared with the larger value from the adjacent pair. This process continues until, on the final (fifth) step, the largest value among all 32 inputs is determined; 2. the transformation of max(|EV_{i}|) into X is achieved by converting max(|EV_{i}|) to the order O_{max} , then subtracting the value of the largest order of the considered data type EK in the MX-format from O_{max} . For example, if max(|V_{i}|)=0\underbrace{10101011}_{\text{8-bit exponent}}011\dots , then max(|EV_{i}|)=10101011 , which corresponds to EV_{i}=171 and O_{max}(EV)=44 . The value of the largest order of E5M2 is 15, with E5=11110 ; thus X=O_{max}-E5=44-15=29 . The ratio of exponent and order of numbers in format FP32 to the shared scale X for E5M2, E4M3, E3M2, E2M3, E2M1, and INT8 is shown in Table II ; 3. the calculation and quantization of P_{1},P_{2},\dots,P_{n} involve rounding r+1 input bits to r output bits. For example, the three most significant bits of the mantissa in FP32 are rounded to two bits of the mantissa in the E5M2 format. The quantization details for E5M2, E4M3, E3M2, E2M3, and E2M1 formats are presented in Tables III , IV , V , VI , and VII , respectively. The quantization of INT8 is not considered here due to the extensive 65-line table required. TABLE II: The ratio of exponent and order of numbers in format FP32 to the shared scale X of E5M2, E4M3, E3M2, E2M3 E2M1, and INT8 FP32 order 0 1 2 3 4 5 6 7 8 … 15 16 17 … 254 255 exponent -126 -126 -125 … 127 \infty / NaN E5M2 order 0 0 … 0 1 2 … 239 240 exponent 0 0 … 0 -126 -125 … 112 NaN E4M3 order 0 0 … 0 1 … 247 248 exponent 0 0