The Fascinating World of 2 × \bm{\times} 2 × \bm{\times} 2 Tensors: Its Geometry and Optimization Challenges

Gabriel H. Brown, Joe Kileel, Tamara G. Kolda · 2025

Read The Fascinating World of 2 × \bm{\times} 2 × \bm{\times} 2 Tensors: Its Geometry and Optimization Challenges by Gabriel H. Brown, Joe Kileel, Tamara G. Kolda (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03937.txt.

Introduction

Gabriel Brown, Joe Kileel, and Tamara Kolda write an educational article on the geometry of small tensors. Using real tensors of size two by two by two, they show how tensors break many familiar matrix rules. A tensor of that size can have rank three, above its smallest dimension, unlike any matrix.

A generic such tensor can have either of two typical ranks, and limits of rank two tensors can reach rank three. Some tensors have no best low rank approximation at all. Tools from algebraic geometry and optimization unify the matrix and tensor stories and yield clean derivations. An interactive visualization tool lets students explore rank constrained sets and build intuition for these surprising facts.

Catalog record

CREATORGabriel H. Brown, Joe Kileel, Tamara G. Kolda
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-03937

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Fascinating World of 2 × \bm{\times} 2 × \bm{\times} 2 Tensors: Its Geometry and Optimization Challenges Thanks: \funding G.B. was supported in part by a CSEM fellowship from the Oden Insitute. J.K. and G.B. were supported in part by NSF DMS 2309782, NSF DMS 2436499, NSF CISE-IIS 2312746 and DE SC0025312. T.K. was supported in part by the JTO visitor program at the Oden Institute. Gabriel H. Brown, Joe Kileel, Tamara G. Kolda Abstract This educational article highlights the geometric and algebraic complexities that distinguish tensors from matrices, to supplement coverage in advanced courses on linear algebra, matrix analysis, and tensor decompositions. Using the case of real-valued 2 × 2 × 2 2\times 2\times 2 tensors, we show how tensors violate many well-known properties of matrices: – The rank of a matrix is bounded by its smallest dimension, but a 2 × 2 × 2 2\times 2\times 2 tensor can be rank 3. – Matrices have a single typical rank, but the rank of a generic 2 × 2 × 2 2\times 2\times 2 tensor can be 2 or 3—it has two typical ranks. – Any limit point of a sequence of matrices of rank r r is at most rank r r , but a limit point of a sequence of 2 × 2 × 2 2\times 2\times 2 tensors of rank 2 can be rank 3 (a higher rank). – Matrices always have a best rank- r r approximation, but no rank-3 tensor of size 2 × 2 × 2 2\times 2\times 2 has a best rank-2 approximation. We unify the analysis of the matrix and tensor cases using tools from algebraic geometry and optimization, providing derivations of these surprising facts. To build intuition for the geometry of rank-constrained sets, students and educators can explore the geometry of matrix and tensor ranks via our interactive visualization tool. \newneutralremark remarkRemark \newneutralremark hypothesisHypothesis \newneutralthm claimClaim \headers The Fascinating World of 2\times 2\times 2 TensorsG. H. Brown, J. Kileel, and T. G. Kolda \newneutralremark exampleExample \newneutralremark exerciseExercise keywords tensors, matrices, rank, geometry, hyperdeterminant, group action, ill-posedness, canonical forms, semi-algebraic sets, geometry, border rank, visualization {MSCcodes} 15A69, 65F99, 14P10, 97N30, 97H60 1 Introduction Tensors have emerged as powerful tools for modeling complex relationships across multiple dimensions. In this educational article, we aim to highlight key characteristics that distinguish tensor analysis from matrix analysis, with a focus the geometry of sets of matrices and tensors of particular ranks. We have created an interactive tool that students and educators can use to visualize and build intuition for the results in this paper. This article provides material suitable for a numerical linear algebra, data science, or first algebraic geometry course. It is peppered with helpful examples and exercises designed to deepen understanding. We assume some knowledge of linear algebra and use examples based on linear algebra to motivate the discussion. One characterization of a rank-1 matrix is as the outer product of two vectors, which we denote here as \mathbf{a}\circ\mathbf{b} , with (i,j) -entry equal to a_{i}b_{j} . The rank of a matrix \bm{{\mathbf{{M}}}}\in\mathbb{R}^{m\times n} is the minimum r such that \bm{{\mathbf{{M}}}} can be written as the sum of r rank-1 matrices: \bm{{\mathbf{{M}}}}=\bm{{\mathbf{{A}}}}\bm{{\mathbf{{B}}}}^{\intercal}\equiv\sum_{\ell=1}^{r}\mathbf{a}_{\ell}\circ\mathbf{b}_{\ell}. Here, \bm{{\mathbf{{A}}}}\in\mathbb{R}^{m\times r} and \bm{{\mathbf{{B}}}}\in\mathbb{R}^{n\times r} are the factor matrices and \mathbf{a}_{\ell} and \mathbf{b}_{\ell} are the factors representing the \ell th columns of \bm{{\mathbf{{A}}}} and \bm{{\mathbf{{B}}}} , respectively. This is illustrated in Fig. 1(a) . Analogous definitions exist for tensors [ kolda_tensor_2025 ] . A rank-1 tensor is the outer product of three vectors, which we denote as \mathbf{a}\circ\mathbf{b}\circ\mathbf{c} and whose (i,j,k) entry is a_{i}b_{j}c_{k} . The rank of a tensor \bm{{\mathscr{{T}}}}\in\mathbb{R}^{m\times n\times p} is the minimum r such that \bm{{\mathscr{{T}}}} can be written as the sum of r three-way vector outer products: \displaystyle\bm{{\mathscr{{T}}}}=\llbracket\bm{{\mathbf{{A}}}},\bm{{\mathbf{{B}}}},\bm{{\mathbf{{C}}}}\rrbracket\equiv\sum_{\ell=1}^{r}\mathbf{a}_{\ell}\circ\mathbf{b}_{\ell}\circ\mathbf{c}_{\ell}. (1) Here, \bm{{\mathbf{{A}}}}\in\mathbb{R}^{m\times r} , \bm{{\mathbf{{B}}}}\in\mathbb{R}^{n\times r} , and \bm{{\mathbf{{C}}}}\in\mathbb{R}^{p\times r} are the factor matrices and \mathbf{a}_{\ell} , \mathbf{b}_{\ell} , and \mathbf{c}_{\ell} are the factors representing the \ell th columns of \bm{{\mathbf{{A}}}} , \bm{{\mathbf{{B}}}} , and \bm{{\mathbf{{C}}}} , respectively. See Fig. 1(b) for a visualization of this decomposition. (a) Rank- r matrix decomposition: \bm{{\mathbf{{M}}}}=\sum_{\ell=1}^{r}\mathbf{a}_{\ell}\circ\mathbf{b}_{\ell} . (b) Rank- r tensor decomposition: \bm{{\mathscr{{T}}}}=\sum_{\ell=1}^{r}\mathbf{a}_{\ell}\circ\mathbf{b}_{\ell}\circ\mathbf{c}_{\ell} . Figure 1 : Matrix and tensor rank decompositions. In contrast to matrix rank, there is no efficient algorithm for computing tensor rank, which is known to be NP-hard in general [ hillar_most_2013 ] . However, the rank of any 2\times 2\times 2 tensor can be determined [ kruskal_rank_1983 , kruskal_rank_1989 , ten_berge_kruskals_1991 , de_silva_tensor_2008 ] as we discuss in Section 2 , and so we focus on 2\times 2\times 2 tensors to give a glimpse of the geometric and algebraic structure of tensors. We compare the properties of matrix and tensor rank, and delve into why tensors have the properties that they do. We call a rank typical if it occurs on a set of positive (Lebesgue) measure. One interesting property of tensors is that there is not necessarily a single typical rank for a tensor of a given size. For example, the typical rank of a 2\times 2 matrix is 2; in other words, every 2\times 2 matrix has rank 2 except for a set of measure zero. In contrast, the typical rank of a 2\times 2\times 2 tensor is 2 or 3; that is, both ranks 2 and 3 occur on sets of positive measure. (Additionally, the reader may have observed that we are implying the tensor rank can be larger than the maximum dimension of the tensor. For tensors of size 2\times 2\times 2 , the maximum possible rank is 3.) A special polynomial in tensor entries, called the hyperdeterminant and discussed in detail in Section 2 , can be used to determine if a 2\times 2\times 2 tensor has one of the two typical ranks. A tensor is rank-2 if its hyperdeterminant is positive and rank-3 if its hyperdeterminant is negative. The hyperdeterminant is zero on a 7-dimensional hypersurface which contains 2\times 2\times 2 tensors of all possible ranks (0, 1, 2, and 3). A three-dimensional slice of the surface is visualized in Fig. 2 using https://ghbrown.net/deter/hdet which contains a button to generate this figure. Figure 2 : A spherically-clipped 3-dimensional projection of the 8-dimensional space of 2\times 2\times 2 tensors showing the 7-dimensional zero set of the hyperdeterminant. A matrix or tensor is said to have a given canonical form if it can be transformed into that form by an invertible transformation; see Section 3 . There are eight canonical forms for 2\times 2\times 2 tensors, even though there are only four possible ranks (0, 1, 2, and 3). In contrast, the number of canonical forms of a matrix is equal to the number of possible ranks. An orbit of a canonical form is a set of matrices or tensors that are equivalent under invertible transformations. The simplicity of the matrix case is evident from the topology and tesselation of its orbits in the ambient space. The set of rank-2 matrices of size 2\times 2 is a single orbit of dimension 4. Its closure contains the set of rank-1 matrices, which is a single orbit of dimension 3. The closure of rank-1 matrices contains the rank-0 (all zero entries) matrix. This leads to a simple hierarchical structure on the orbits.

Read the complete text