Nautilus: An Auto-Scheduling Tensor Compiler for Efficient Tiled GPU Kernels

Yifan Zhao, Yuchen Yang, Matei Budiu, Sasa Misailovic · 2026

Read Nautilus: An Auto-Scheduling Tensor Compiler for Efficient Tiled GPU Kernels by Yifan Zhao, Yuchen Yang, Matei Budiu, Sasa Misailovic (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14825.txt.

Description

Zhao, Yang, Budiu, and Misailovic present Nautilus, a tensor compiler turning algebraic specs of tensor operators into tiled GPU kernels. Its lowering design applies high-level, expression, and tile optimizations jointly. An auto-scheduler finds optimization chains while keeping tile-optimizer structure. Nautilus is the first full compiler to start from a math-like attention description and discover FlashAttention-3-like kernels. On five transformer models and 150 configs on GH200 and RTX 5090 hardware, it beat leading compilers by up to 23 and 42 percent, matching cuDNN kernels on many long-sequence setups.

Catalog record

CREATORYifan Zhao, Yuchen Yang, Matei Budiu, Sasa Misailovic
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57193
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.14825

Browse the full library · About the Tower of Babel