Blockbuster, Part 1: Block-level AI Operator Fusion

Ofer Dekel, Microsoft April, 2025 · 2025

Read Blockbuster, Part 1: Block-level AI Operator Fusion by Ofer Dekel, Microsoft April, 2025 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07829.txt.

Description

Ofer Dekel presents Blockbuster, in a 2025 preprint, a framework for fusing AI operations during inference. Fusing steps into one kernel saves trips to memory, the true bottleneck on modern chips. Blockbuster works on any machine with tiered memory, from graphics cards to AI accelerators. Workloads are drawn as block programs, which track exactly how chunks of data move between memory levels. Fusion candidates are picked first, then each is fused by a rule-based algorithm. That split suits the huge programs now common.

The paper centers on the fusion algorithm itself. As a sanity check, it rediscovers the famous Flash Attention kernel on its own. It then fuses normalization with matrix multiplication into single kernels. One mega-kernel swallows three multiplications plus several smaller operations. Dekel argues that modeling data movement directly is what makes the fusions unusually powerful.

Catalog record

CREATOROfer Dekel, Microsoft April, 2025
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE100 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-07829

Browse the full library · About the Tower of Babel