Blockbuster, Part 1: Block-level AI Operator Fusion
Ofer Dekel, Microsoft April, 2025 · 2025
Read Blockbuster, Part 1: Block-level AI Operator Fusion by Ofer Dekel, Microsoft April, 2025 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07829.txt.
Description
Ofer Dekel presents Blockbuster, in a 2025 preprint, a framework for fusing AI operations during inference. Fusing steps into one kernel saves trips to memory, the true bottleneck on modern chips. Blockbuster works on any machine with tiered memory, from graphics cards to AI accelerators. Workloads are drawn as block programs, which track exactly how chunks of data move between memory levels. Fusion candidates are picked first, then each is fused by a rule-based algorithm. That split suits the huge programs now common.
The paper centers on the fusion algorithm itself. As a sanity check, it rediscovers the famous Flash Attention kernel on its own. It then fuses normalization with matrix multiplication into single kernels. One mega-kernel swallows three multiplications plus several smaller operations. Dekel argues that modeling data movement directly is what makes the fusions unusually powerful.
Catalog record
| CREATOR | Ofer Dekel, Microsoft April, 2025 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 100 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-07829