BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference
Wonsuk Jang, Thierry Tambe · 2025
Read BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference by Wonsuk Jang, Thierry Tambe (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-01144.txt.
Description
Wonsuk Jang and Thierry Tambe address the cost of running large language models in a 2025 paper. Bigger models need more memory and power, so engineers shrink the numbers inside them. Fine-grained shrinking helps, but one format rarely fits all blocks of data well. Outlying values make the fit worse.
Their method gives each block of data its own best number format, chosen from a small set of variants. A two-stage step handles new data as the model runs, while staying cheap in energy. Tests on two well-known models showed clear accuracy gains over a standard format, using fewer bits and staying close to full precision.
Catalog record
| CREATOR | Wonsuk Jang, Thierry Tambe |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-01144