CuTe Layout Representation and Algebra

Cris Cecka · 2026

Read CuTe Layout Representation and Algebra by Cris Cecka (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-02298.txt.

Introduction

Cris Cecka presents a math framework for tensor layouts in this 2026 paper. Modern chips use special tensor instructions with fixed, complex data layouts. A layout says where each element of a multi-dimensional array sits in memory. Wrong layouts break correctness or waste the chip's speed. Layouts must be tracked through every step of a program.

His system, CuTe, extends flat shape-and-stride layouts into layered forms. A rich algebra joins, merges, splits, and inverts layouts with clear rules. Layouts cover both data placement and how threads share the work. The algebra enables checks at compile time, before code ever runs. Cecka argues these tools ease building fast GPU kernels and generic tensor code alike.

Catalog record

CREATORCris Cecka
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51772
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.02298

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CuTe Layout Representation and Algebra Cris Cecka Abstract Modern architectures for high-performance computing and deep learning increasingly incorporate specialized tensor instructions, including tensor cores for matrix multiplication and hardware-optimized copy operations for multi-dimensional data. These instructions prescribe fixed, often complex data layouts that must be correctly propagated through the entire execution pipeline to ensure both correctness and optimal performance. We present CuTe , a novel mathematical specification for representing and manipulating tensors. CuTe introduces two key innovations: (1) a hierarchical layout representation that directly extends traditional flat-shape and flat-stride tensor representations, enabling the representation of complex mappings required by modern hardware instructions, and (2) a rich algebra of layout operations – including concatenation, coalescence, composition, complementation, division, tiling, and inversion – that enables sophisticated layout manipulation, derivation, verification, and static analysis. CuTe layouts provide a framework for managing both data layouts and thread arrangements in GPU kernels, while the layout algebra enables powerful compile-time reasoning about layout properties and the expression of generic tensor transformations. In this work, we demonstrate that CuTe ’s abstractions significantly aid software development compared to traditional approaches, promote compile-time verification of architecturally prescribed layouts, facilitate the implementation of algorithmic primitives that generalize to a wide range of applications, and enable the concise expression of tiling and partitioning patterns required by modern specialized tensor instructions. CuTe has been successfully deployed in production systems, forming the foundation of NVIDIA’s CUTLASS library and a number of related efforts including CuTe DSL. 1 Introduction and Motivation Modern GPUs are increasingly optimized for tensor-centric computations, driven by the demands of deep learning and scientific computing. NVIDIA’s Volta architecture [ NVIDIA:Volta ] introduced Tensor Cores, enabling efficient small-matrix multiplications directly in hardware. This capability expanded in Turing [ NVIDIA:Turing ] and Ampere [ NVIDIA:Ampere ] with specialized instructions for structured matrix movement within the GPU memory hierarchy. The Hopper [ NVIDIA:Hopper ] and Blackwell [ NVIDIA:Blackwell ] architectures further advance this paradigm, introducing copy instructions for efficiently transferring rank-5 tensors between global and shared memory and further expanding tensor core capabilities. Fully harnessing these tensor-oriented hardware features is critical for peak GPU performance, motivating high-performance programming models that can represent and manipulate these tensors efficiently. The existing and emerging hardware depends critically on how multidimensional data is stored and accessed in multiple hierarchical memory spaces and in multiple hierarchical levels of parallelism. Data storage layouts have always affected performance by dictating how and when memory accesses occur, but as hardware instructions become larger and prescribe fixed layouts for inputs and outputs the effect on correctness also becomes vital. These layouts must be propagated through the entire execution pipeline to ensure correct invocation of hardware instructions and ensure optimized memory access patterns. In this work, we present the foundational concepts of CuTe , a specification for CU DA Te nsors, or C ompute U nified Te nsors, designed to provide building blocks for writing peak-performance linear algebra libraries. At its core, CuTe introduces two key innovations: • A novel representation for tensor layouts : CuTe shapes, layouts, and tensors are inherently hierarchical, constructed from smaller nested instances. This hierarchy provides the means to represent complex mappings required by modern tensor instructions, but remains a strict extension of existing flat-shape and flat-stride representations found in libraries like BLAS , torch.tensor , numpy.ndarray , and MATLAB. • A novel algebra of operations defined over layouts : CuTe layouts support a rich set of operations, including concatenation, coalescence, composition, complementation, division, tiling, and inversion, which all result in new CuTe layouts. These operations enable sophisticated partitioning, manipulation, verification, and derivation of tensor layouts demanded by modern tensor instructions. CuTe ’s layout representation provides an intuitive framework for managing threads and data in writing generic algorithms. CuTe ’s layout algebra provides an expressive approach for manipulating layouts and generating new layouts in the development of high-performance linear algebra kernels. These approaches enable: • Support for complex layouts and partitioning: CuTe facilitates the representation of intricate layouts required by application-specific data patterns and complex partitioning patterns required by specialized tensor instructions. • Separation of concerns: Data layouts are declared independently of algorithmic logic, promoting clarity and modularity. • Static analysis and optimization: Sophisticated algebraic techniques empower the inspection, reordering, and partitioning of tensor arguments according to architectural constraints. 1.1 Related Work CuTe is motivated by the need to support development of efficient tensor contractions, which are at the core of many scientific computing and machine learning applications. Conventional approaches for computing general tensor contractions rely on matricization, which involves logically or explicitly restructuring tensor data to perform computations using a sequence of calls to a Basic Linear Algebra Subprograms (BLAS) library. BLAS provides efficient and portable implementations of core linear algebra operations, with highly optimized versions available for a wide range of architectures [ blackford2002updated ] . Among BLAS primitives, the GEneral Matrix Multiply (GEMM) routine is easily the most optimized and widely used operation in all of scientific computing and machine learning. The BLAS-like Library Instantiation Software (BLIS) framework [ VanZee:BLIS ] extends GEMM by supporting non-unit strides in both row and column modes simultaneously, which addresses some challenges in handling irregular memory layouts without resorting to explicit memory copies. The strided-batched GEMM extension to BLAS further generalizes the primitive and allows its application to even more tensor contractions [ Shi:BLASContractions ] . Abstracting matrix layouts even further, a key insight motivating CuTe is the use of multi-indices in tensor notation to enable the transformation of arbitrary tensor contractions into a canonical batched-GEMM primitive. Many existing libraries rely on tensors and nearly all of them are based on the flat-shape and flat-stride representation. In Python, these include numpy.ndarray and torch.tensor ⬇ > > > a = numpy . ndarray ( [ 3 , 7 , 5 ] ) > > > a . dtype dtype ( ’float64’ ) > > > a . shape ( 3 , 7 , 5 ) > > > a . strides ( 280 , 40 , 8 ) ⬇ > > > a = torch . empty ( 3 , 7 , 5 ) > > > a . dtype torch . float32 > > > a . shape torch . Size ( [ 3 , 7 , 5 ] ) > > > a . stride () ( 35 , 5 , 1 ) and in C++, std::mdspan ⬇ std :: mdspan a = std :: mdspan ( data , 3, 7, 5); a . extent (0); // 3 a . extent (1); // 7 a . extent (2); // 5 a . stride (0); // 35 a . stride (1); // 5 a . stride (2); // 1 CuTe supports these representations and strictly expands on them with generalizations to hierarchical shapes and strides to represent more complex layouts, non-integral strides, and non-integral layout codomains. Independent generalizations of dense tensor representations include HeLayers [ Aharoni:HeLayers ] , ThunderKittens [ Spector:ThunderKittens ] , and the Linear Layouts [ Tillet:LinearLayouts ] approach used in OpenAI’s Triton compiler [ Tillet:TritonAI ] . Thunderkittens implements a wide variety of bespoke types for register memory, shared memory, row/column-major tiles, row/column-major tiles of row/column-major subtiles, and prescribed access patterns for warps and threads. These

Read the complete text