Contrastive independent component analysis for salient patterns and dimensionality reduction Authors: Kexin Wang, Aida Maraj, Anna Seigal Published: 2025 ABSTRACT SignificanceVisualizing data and finding patterns in data are ubiquitous problems in the sciences. Increasingly, applications seek signal and structure in a contrastive setting: a foreground dataset relative to a background dataset. The goal is to learn patterns and visualize the foreground after “subtracting off” the effect of the background. For this purpose, we propose contrastive independent component analysis (cICA). We investigate cICA theoretically and computationally. We find that, relative to other approaches, cICA is more expressive; that is, able to model a broader range of settings, while simultaneously being identifiable, able to recover patterns uniquely. Finding and understanding patterns in data is fundamental in various scientific fields. Often, data have been collected under two different settings, such as a group of patients receiving treatment and a control group, or a group of patients with a certain disease and a group without the disease. The goal is to understand the effect of the treatment or to understand the genetic changes that describe the disease. While standard data analysis methods can be used, which restrict attention to one of the datasets or combine them together, an alternate view is offered by contrastive methods. Contrastive methods view the two settings as a foreground and a background. They seek to learn patterns in the foreground after accounting for (or, “subtracting off”) the background. The hope is that such patterns encode useful structures and offer a good basis for dimensionality reduction and visualization of the data, to identify fine-grained structures and clusters particular to the foreground. Back in the 1980s, Flury initiated the idea of comparing covariance matrices and finding principal components across multiple datasets (1–3). The contrastive viewpoint was then addressed and formalized in ref. 4, where the authors discussed contrastive topic modeling and contrastive hidden Markov models. Principal component analysis (PCA) was generalized to contrastive PCA (cPCA) in refs. 5 and 6. A latent variable model perspective is taken in refs. 7 and 8. The present work extends such methods, specifically cPCA, to a more expressive and identifiable setting. Specifically, it removes simplifying assumptions that the amount of each background signal present in the foreground is the same (4–8), that the latent variables are Gaussians (5–8), and that the salient patterns in the foreground data are orthogonal (5, 6). The greater expressivity and identifiability are achieved using the higher-order cumulant tensors of the foreground and background data, which encode more fine-grained structure than the covariance matrices. We call the method contrastive independent component analysis (cICA). Independent component analysis (ICA) is a blind source separation method, which seeks to recover latent sources and unknown mixing from observations of mixtures of signals (9). ICA assumes that latent sources are independent. In extending ICA to the contrastive setting, the idea is that background data is generated by mixing of independent sources while foreground data are generated by the background mixing together with a foreground mixing of independent sources. We show using connections to classical algebraic geometry that cICA has strong identifiability properties. This enables the contribution of each background pattern to the foreground to be found uniquely, avoiding the need for a sweep of hyperparameters to find the best multiple of the background to subtract from the foreground and even avoids the assumption that the background contribution to the foreground is via a single scalar multiple, both of which are required in refs. 4–7. To implement cICA, we devise a hierarchical tensor decomposition based on recursive eigendecompositions. The decomposition encourages (rather than imposes) orthogonality between the rank-one summands. We show that it recovers accurate patterns for synthetic data. We turn cICA into a dimensionality reduction tool and investigate its performance on real-world data, comparing the plots to those obtained with other contrastive methods to see its competitiveness. The paper is organized as follows. We define cICA in Section 1. We introduce the hierarchical tensor decomposition in Section 2. We study identifiability and algorithms for cICA in Section 3. Numerical results are in Section 4. From ICA to cICA Blind source separation seeks to recover latent sources and unknown mixing from observations of mixtures of signals (9). A special case is ICA, which assumes that the latent sources are independent. ICA was introduced in 1985 (10) and popularized by Comon in his paper (11). ICA can be viewed as a generalization of PCA, where instead of finding uncorrelated components, it goes a step further by aiming to make the components statistically independent and instead of decomposing second-order information (covariance matrices), it decomposes higher-order statistics (via the cumulant tensors). ICA studies observations that are a linear mixture of independent source variables. Applications include recovering speech and brain signals (12, 13), causal discovery (14), and image decomposition (15). We write the ICA model as[1]y=Az, where z is a vector of r independent latent random variables, the mixing matrix is A∈Rp×r, and y is a vector of p observed variables. The i-th column of A records a pattern in the data: the contribution of variable zi to each of the p observed variables. The identifiability of ICA refers to the uniqueness of the mixing matrix A and sometimes also of the variables z; see refs. 11, 16, and 17. Many algorithms for ICA proceed via tensor decomposition; see, e.g., refs. 9 and 18–20. The cumulants of a distribution are symmetric tensors that encode it. The d-th cumulant κd(y) of y is a symmetric order d tensor of format p×⋯×p whose entry at position (j1,…,jd) is[2]∑i=1rλi(ai)j1⋯(ai)jd, where the scalar λi is the d-th cumulant of zi and the vector ai∈Rp is the i-th column of A. We denote this by[3]κd(y)=∑i=1rλiai⊗d. This decomposition Eq. 3 follows from the multilinear properties of cumulants and the fact that cumulant tensors of independent variables are diagonal, see (21, Chapter 2). The matrix A can be recovered using tensor decomposition of the cumulant tensor Eq. 2. If the tensor decomposition is identifiable, then the columns ai with λi≠0 can be recovered uniquely up to permutation and scaling of columns. Thus tensor decomposition of higher-order cumulant tensors gives an algorithm for ICA, provided no source variable is Gaussian (this is required for nonzero higher-order cumulants). In this paper, we extend ICA, and tensor decomposition for ICA, to the comparison of two distributions. We call this contrastive ICA (cICA), by analogy with cPCA (6). We have two observed distributions, a foreground, and a background. Both are assumed to be linear mixtures of independent source variables. Our cICA model expresses the background y and foreground x as[4]y=Azandx=Az′+Bs. The background distribution y is a linear mixture of a random vector z of r independent random variables, as in Eq. 1. The foreground x is a mixture of r+ℓ independent variables z′=(z1′,…,zr′) and s=(s1,…,sℓ). The columns of A are the patterns in the background: Column ai∈Rp records how source variable zi appears among the p background variables as well as how source variable zi′ appears among the p foreground variables. The columns of B are patterns that appear only in the foreground. They correspond to the variables si, referred to as the salient variables in ref. 22. We propose a tensor decomposition algorithm to recover mixing matrices A and B from Eq. 4. These matrices record the patterns that encode our background and foreground distributions. We apply the algorithm to empirical cumulant tensors of x and y obtained from sample data. We order the columns of matrix B to obtain a dimensionality reduction tool. We work under the assumption that z,z′,s are non-Gaussian, an assumption that also appears for usual ICA. This can likely be relaxed to that at most one source is Gaussian, cf. (11, 17). Under the model Eq. 4, the d-th cumulants of the background and foreground data are, respectively,[5]κd(y)=∑i=1rλiai⊗d,κd(x)=∑i=1rλi′ai⊗d+∑j=1ℓνjbj⊗d, where λi is the d-th cumulant of zi, λi′ is the d-th cumulant of zi′, and νj is the d-th cumulant of sj. This follows from the multilinearity of cumulants and that cumulant tensors of independent sources are diagonal, as for usual ICA. See Fig. 1 for an illustration of κ3(x)=∑i=1rλi′ai⊗d+∑j=1ℓνjbj⊗d when d=3. Tensor decomposition for κ3(x)=∑i=1rλi′ai⊗d+∑j=1ℓνjbj⊗d when d=3 and r=ℓ=2. The central 4×4×4 diagonal tensor is multiplied along each index by a matrix with four columns, whose first two columns (blue) are the background patterns and last two columns (red) are the foreground patterns. We have the following optimization problem to recover A and B: find a joint decomposition of cumulant tensors κd(y) and κd(x) of the form in Eq. 5. Our approach is Compute a symmetric tensor decomposition of κd(y) to learn A.Find the coefficients λi′ of each ai⊗d in κd(x) to obtain ∑j=1ℓνjbj⊗d.Compute a symmetric tensor decomposition of ∑j=1ℓνjbj⊗d to learn B. Compute a symmetric tensor decomposition of κd(y) to learn A. Compute a symmetric tensor decomposition of κd(y) to learn A. Find the coefficients λi′ of each ai⊗d in κd(x) to obtain ∑j=1ℓνjbj⊗d. Find the coefficients λi′ of each ai⊗d in κd(x) to obtain ∑j=1ℓνjbj⊗d. Compute a symmetric tensor decomposition of ∑j=1ℓνjbj⊗d to learn B. Compute a symmetric tensor decomposition of ∑j=1ℓνjbj⊗d to learn B. We work with the fourth-order cumulants d=4, since the tensor decomposition we use works better for an even order symmetric tensor. For the third step of our approach, we require a tensor decomposition method that is efficient and promotes orthogonality among the rank-1 components, which aids interpretability and improves visualizations. To address this, we propose a hierarchical eigendecomposition based algorithm, which we describe in more detail in the next section. The algorithm uses linear algebra and can handle tensors of rank up to p2 [unlike p in other linear algebra-based methods (23, 24)]. Related Work. We relate cICA to other contrastive models. In cPCA, the contrastive patterns are principal components of the foreground covariance matrix minus a scalar multiple of the background covariance matrix (5, 6). We can specialize cICA to cPCA by setting z′=γz and studying observed distributions x and y via their covariance matrices (d=2). Probabilistic contrastive PCA (PCPCA) is introduced in ref. (7), where foreground patterns are inferred by maximizing a likelihood ratio of linear Gaussian mixtures. Contrastive ICA also relates to PCPCA (7) but we do not impose distributional assumptions, beyond independence and non-Gaussianity, on the variables z and (z′,s). The paper (8) studies a linear contrastive latent variable model. The contrastive ICA model aligns with the framework of the contrastive latent variable model proposed in ref. 8, but it does not assume any relationship between z and z′ while the contrastive latent variable model assumes z=z′. The setting of cICA relates to usual ICA, with block structure on the mixing matrix:ifz′,z,sare independent,xy=0ABA00zz′s;ifz′=γz,xy=γABA0zs. Identifiability can be characterized using (11), or using (16, 17) if the model is overcomplete (i.e., the number of sources exceeds the number of observations, which occurs for 2r+ℓ>2p). However, learning parameters via usual ICA requires access to the joint distribution of (x,y), which is generally unavailable because the data from the two datasets are unpaired. For example, single-cell RNA data for patients with a disease (foreground) and a control group (background) have each person assigned to either the foreground set or the background. In ref. 25, the authors study multimodal linear ICA. They recover the mixing matrices from each mode via usual linear ICA and use a hypothesis test to decide which latent variables are shared across modes. Our method differs from this as we seek patterns unique to the foreground rather than shared patterns. Nonlinear contrastive methods have been explored in the literature. Nonlinear ICA is studied using contrastive learning (26–28). Here contrastive is used in a different context: It describes a method to train a network to distinguish two datasets. A nonlinear contrastive method called a contrastive variational autoencoder (cVAE) is introduced in refs. 8 and 22. The paper (29) presents a method for cVAE using maximum mean discrepancy to prevent leakage of information between the two sets of latent variables. Identifiability of cVAE is studied using connections to nonlinear ICA in ref. 30. These works produce a nonlinear latent encoding of data, whereas our focus is on pattern vectors to describe observed variables. Hierarchical Tensor Decomposition ICA has seen limited application in data visualization, one notable exception being (31). Existing algorithms to compute a symmetric tensor decomposition usually have randomness due to initialization and the details of the optimization process, such as the step size in gradient-based optimization. Another challenge is that the resulting vectors may be nearly parallel (32), which yields a suboptimal basis for projecting the data and hinders its interpretability. We overcome these difficulties with our proposed hierarchical tensor decomposition (HTD). Its output is deterministic and the components learned are almost orthogonal. HTD decomposes an order four tensor via recursive eigendecompositions. The idea is to find a low-rank approximation of a tensor, whose rank-one summands offer an interpretable basis on which to project data. Later, we use the decomposition for cICA. In this section, we define the decomposition and study its properties. HTD for a tensor in (Rp)⊗4 uses linear structure in the space (Rp)⊗2 rather than Rp, so it handles tensors of rank up to p2 [unlike p in other linear algebra-based methods (23, 24)]. The detailed comparison with other tensor decomposition methods is in SI Appendix, section 1. The HTD Algorithm. Consider a symmetric tensor T of format p×p×p×p. We compute a rank r approximation,[6]T≈∑i=1rνibi⊗4, as follows. Let Mat(T) be the flattening of T that rearranges its p4 entries into a matrix of size p2×p2. The entries of Mat(T) are indexed ((i1,i2),(j1,j2)), where i1,i2,j1,j2∈[p]:={1,…,p}. We compute the approximation Eq. 6 by first computing the eigendecomposition of Mat(T), whose eigenvectors lie in Rp2, and then by reshaping these eigenvectors into p×p matrices and computing their top eigenvalue and corresponding eigenvector. By top eigenvalue we mean those of highest magnitude. This decomposition has not to our knowledge been studied before but has connections to the hierarchical tensor representations of (33, Chapter 11) and the PARATREE model in ref. 34; see SI Appendix, section 1. See Fig. 2 for an illustration of the steps of HTD on a 2×2×2×2 tensor. Here is the HTD algorithm. Compute unit vectors b1,…,br such that T≈∑i=1rvibi⊗4 Steps in the HTD algorithm: input tensor T, matrix flattening Mat(T), best rank r approximation Mat(T)≈∑i=1rμivi⊗2, best rank-one approximation of each Mat(vi) and the output rank r approximation for T. We record some observations pertaining to Algorithm 1. The matrix Mat(T)∈Rp2×p2 is symmetric since T is symmetric. The matrices M1,…,Mr∈Rp×p are also symmetric, because the vectors v1,…,vr are in the column space of Mat(T), whose (i1,i2)-th row coincides with its (i2,i1)-th row. Although the output vectors bi are in general not orthogonal, as each is an eigenvector of a distinct matrix, they can be nearly orthogonal in practice, see Section 2.2. This is because they are the leading eigenvectors of matrices that have been reshaped from orthogonal vectors vi. [2 × 2 × 2 × 2 example] Let r=2. FixT=210⊗4+0.09980.995⊗4. ThenMat(T)=2.00010.00100.00100.00990.00100.00990.00990.09830.00100.00990.00990.09830.00990.09830.09830.9801 with eigenvalues μ1=2.00019,μ2=0.99977 and associated eigenvectorsv1T≈0.999950.000980.000980.00985,v2T≈−0.009950.09930.09930.99003. Their corresponding matrices M1,M2∈R2×2 are symmetric with top eigenvalues β1=0.99995 and β2=0.9998, respectively, with associated eigenvectors b1T=0.999990.00099 and b2T=0.097870.99519. The HTD algorithm with input T and r=2 thus outputs ∑i=12(μiβi2)bi⊗4=1.999990.999990.00099⊗4+0.999370.097870.99519⊗4. We note the similarity to the input tensor T. Properties of the Decomposition. The HTD algorithm outputs a rank r approximation of a tensor. In certain cases, the output closely approximates the input tensor, as in Example 2.1. We bound the distance between the HTD approximation and the input tensor. We give a bound that applies to all tensors in Proposition 2.2. We show that the input and output coincide for orthogonally decomposable tensors in Proposition 2.3. Our main result is Theorem 2.4, which bounds the distance between an input and output tensor for a tensor decomposition involving vectors that are close to orthogonal. The norm ‖·‖F refers to the Frobenius norm for matrices and tensors and the 2-norm for vectors; i.e., the square root of the sum of the squares of the entries. The 2-norm of a matrix is denoted by ‖·‖2. Let T be a symmetric tensor of format p×p×p×p. Let T′=∑i=1r(μiβi2)bi⊗4 be the rank r HTD approximation of T. Then‖T′−T‖F≤∑i=r+1qμi212+∑i=1r|μi|(1+|βi|)∑j=2ri(βi(j))212, where q is the rank of Mat(T), ri is the rank of Mi, the numbers μ1,…,μr are the eigenvalues of Mat(T) in descending order of magnitude, and βi:=βi(1) is the highest magnitude eigenvalue of Mi with βi(2),…,βi(ri) the other eigenvalues. Proof We use the notation from Algorithm 1. We have‖Mat(T)−∑i=1rμivi⊗2‖F2=∑i=r+1qμi2,‖Mi−βibi⊗2‖F2=∑j=2ri(βi(j))2, from the properties of the eigendecomposition of a symmetric matrix and the Frobenius norm. Let T″ be the p×p×p×p tensor obtained from reshaping the truncated eigendecomposition ∑i=1rμivi⊗2 of Mat(T). Then ‖T−T″‖F2=∑i=r+1qμi2. Let Bi∈Rp2 be the vectorization of bi⊗2∈Rp×p. Then‖T″−T′‖F=‖∑i=1rμi(vi⊗2−βi2Bi⊗2)‖F≤∑i=1r|μi|‖vi⊗2−βi2Bi⊗2‖F≤∑i=1r|μi|(‖vi⊗2−βiBi⊗vi‖F+‖βi2Bi⊗2−βiBi⊗vi‖F)=∑i=1r|μi|(‖vi‖+|βi|‖Bi‖)‖vi−βiBi‖=∑i=1r|μi|(1+|βi|)∑j=2ri(βi(j))212, where the penultimate equality follows from ‖x⊗y‖=‖x‖·‖y‖ and the last equality uses ‖vi‖=‖Bi‖=1. We conclude with the triangle inequality ‖T−T′‖F≤‖T−T″‖F+‖T″−T′‖F. The quantity in Proposition 2.2 is small if Mat(T) is well approximated by a matrix of rank r, and each Mi is well approximated by a matrix of rank one. Orthogonally decomposable tensors are those with a decomposition into orthogonal rank-one terms; that is, a decomposition T=∑i=1rνibi⊗4, where b1,…,br are orthonormal (35). For orthogonally decomposable tensors, HTD recovers the exact decomposition. Let T=∑i=1rνibi⊗4, where the vectors b1,…,br are orthonormal and the coefficients ν1,…,νr are distinct. Then the rank r HTD approximation is the tensor T. Proof The flattening Mat(T) has decomposition ∑i=1rνiBi⊗2, where Bi∈Rp2 is the vectorization of bi⊗2∈Rp×p. We have the orthogonality ⟨Bi,Bj⟩=⟨bi,bj⟩2=0 for all i≠j, since the vectors bi,bj are orthogonal. Hence this expression for Mat(T) is a sum of outer products of orthogonal vectors, so it is the eigendecomposition of Mat(T). The matrix reshaped from the eigenvector Bi is Mi=bi⊗2. It has top eigenvalue 1 with corresponding eigenvector bi. Hence, the output of HTD is ∑i=1rνibi⊗4. We extend Proposition 2.3 to decompositions where the vectors bi are close to orthogonal and the input tensor is noisy. The condition that the matrices b1⊗2,…,br⊗2 are linearly independent ensures that Mat(T) has rank r. This condition holds for generic vectors bi, provided r≤p+12. The quantity min‖bi−bi′‖,‖bi+bi′‖ arises because of the sign indeterminacy in the vectors in the decompositions, due to the equality (−bi)⊗d=bi⊗d for d even. We provide a sketch proof of Theorem 2.4. The full proof can be found in SI Appendix, section 2. Fix vectors b1,…,bℓ∈Rp with |⟨bi,bj⟩|≤ϵ for all i≠j. LetT=∑i=1ℓνibi⊗4, where ν1>⋯>νℓ, ℓ≤p, and b1⊗2,…,bℓ⊗2 are linearly independent. Fix T^ with ‖T^−T‖F≤δ. Let ci be the output patterns of the HTD algorithm with input tensor T^ and μi the corresponding recovered scalars ordered so that μ1>⋯>μℓ. Then for any i∈[ℓ],|νi−μi|=O(ϵ2)+O(δ),andmin‖bi−ci‖,‖bi+ci‖=O(ϵ2)+O(δ). Sketch Proof: Fix M=Mat(T). Then M=∑i=1rνiBi⊗2, where Bi=Vect(bi⊗2). Using Gram–Schmidt orthogonalization, we can construct a matrix M′ in Rp2×p2 with eigendecomposition ∑i=1ℓνi(Bi′)⊗2 such that[7]‖Bi′−Bi‖≤2(ℓ−1)ϵ2+O(ϵ4),‖M−M′‖F≤Kϵ2+O(ϵ4), where K=8∑i=1ℓ|νi|(i−1). Suppose M^=Mat(T^) has eigendecomposition M^=∑i=1ℓν^iB^i⊗2. The difference between M^ and M′ is bounded by‖M^−M′‖F≤‖M^−M‖F+‖M−M′‖F≤Kϵ2+δ+O(ϵ4) using the triangle inequality. We thus obtain[8]|νi−ν^i|≤δ+Kϵ2+O(ϵ4),[9]‖B^i−Bi′‖≤232ν(δ+Kϵ2+O(ϵ4)), by Weyl’s Theorem and the variant of Davis-Kahan Theorem in ref. (36), where ν=mini≠j{|νi−νj|,|νi|}. We bound the difference between Bi and B^i using Eqs. 7 and 9:[10]‖Bi−B^i‖≤‖Bi′−Bi‖+‖Bi′−B^i‖≤Lϵ2+232νδ+O(ϵ4), where L=23/2Kν+2ℓ−2. Then, by Weyl’s theorem,[11]|α−1|≤‖Bi−B^i‖, where α is the top eigenvalue of Mat(B^i). HTD implies μi=α2ν^i. The bound on |μi−νi| then follows from Eq. 8 and Eq. 11. The bound of min‖bi−ci‖,‖bi+ci‖ follows from Eq. 10 and (36), since ci is the top eigenvector of B^i. Tensor Decompositions for cICA Our cICA model assumes y=Az and x=Az′+Bs, for A∈Rp×r and B∈Rp×ℓ, see Eq. 4. This leads to the cICA tensor decompositions Eq. 5. One does not assume a relationship between z and z′. We discuss the algorithm and identifiability of cICA in Section 3.1. We explain how to use cICA for dimensionality reduction in Section 3.2. This projects data onto a subspace given by certain columns of the foreground mixing B. We bound the end-to-end error of our algorithm in Section 3.3. When z′=γz for some scalar γ, we discuss an alternative algorithm in SI Appendix, section 4 and its performance for various datasets in SI Appendix, section 6. cICA Algorithm and Identifiability. We present Algorithm 2 for cICA. Steps 1 and 3 both decompose a symmetric order four tensor. We use the subspace power method (37) in Step 1 to prioritize the accuracy of the tensor decomposition. We use Algorithm 1 in Step 3 to prioritize interpretability and efficiency. We provide numerical experiments to justify these choices of algorithm in Section 4.1. Recover A and B from the cumulants of the background and foreground We study the identifiability of the algorithm, that is, the uniqueness of the vectors and scalars it outputs, assuming genericity. Our genericity assumption holds almost surely in the space of parameters. We use the following lemma. Let vectors ai∈Rp and scalars λi∈R be generic. Then the decomposition T=∑i=1qλiai⊗d of a symmetric p×p×p×p tensor T is unique forq≤⌈1pp+34−1⌉forp∉{3,4,5},⌈1pp+34⌉forp∈{3,5},9forp=4,providedq≠8. Proof The rank of a generic p×p×p×p symmetric tensor is ⌈1pp+34⌉ for p∉{3,4,5} and ⌈1pp+34⌉+1 for p∈{3,4,5}, by the Alexander-Hirschowitz theorem (38). Generic rank q tensors in this space, with q strictly below the generic rank, have unique symmetric tensor decomposition for (p,q)≠(4,8) and two tensor decompositions for p=4,q=8 by (39, Theorem 1.1). The joint decomposition κ4(y)=∑i=1rλiai⊗4,κ4(x)=∑i=1rλi′ai⊗4+∑j=1ℓνjbj⊗4, is unique for generic ai,bj,λi,λi′,νj, where i∈[r] and j∈[ℓ], when r+ℓ<⌈1pp+34⌉ for p≠3,4,5, r+ℓ≤⌈1pp+34⌉ for p=3,5, and when r+ℓ≤9,r+ℓ≠8 for p=4. Proof The tensor decomposition for cICA in the statement is identifiable when the symmetric tensor decomposition of κ4(x) is unique, as follows. The tensor decomposition of κ4(x), gives vectors ai,bj up to permutation and scaling. Then we can solve a linear system to find the decomposition κ4(y)=∑i=1rλiai⊗4. It therefore remains to study the identifiability of the decomposition of κ4(x). It is a symmetric p×p×p×p tensor of rank r+ℓ. Hence, the uniqueness follows from Lemma 3.1, setting q=r+ℓ. When (λ1,…,λr) and (λ1′,…,λr′) are proportional, as vectors in Rr, we have a stronger identifiability result than the one for two separate tensor decompositions in Proposition 3.2. Consider the joint decomposition of κ4(y)=∑i=1rλiai⊗4,κ4(x)=∑i=1rλi′ai⊗4+∑j=1ℓνjbj⊗4. Suppose that (λ1′,⋯,λr′)=μ(λ1,⋯,λr) for some μ∈R∖{0}. Suppose further that ai,bj,λi,μ,νj for i∈[r] and j∈[ℓ] are generic. Then, the joint decomposition of κ4(y) and κ4(x) is unique providedmax1prℓ+ℓ,r<1pp+34. Proof We can assume that r is a multiple of ℓ: If the joint decomposition is unique with r replaced by the possibly larger number ⌈r/ℓ⌉ℓ, then the original joint decomposition with r terms is also unique. Let k=rℓ and define the tensors T1,…,Tk by taking a subset of ℓ consecutive terms from κ4(y): Tj=∑i=(j−1)ℓ+1jℓνibi⊗4. DefineW=Spanκ4(x),T1,⋯,Tk. Then ∑i=1ℓνibi⊗4∈W, since the difference between it and κ4(x) is μ(T1+⋯+Tk). Let X∈PN be the variety of symmetric border rank at most ℓ tensors in (Rp)⊗4, where N=p+34−1. The tensors[12]∑j=1ℓνjbj⊗4,T1,⋯,Tk are generic points on X, since ai,bj,λi,νj are generic for i∈[r],j∈[ℓ]. We have projective dimensions dimX≤ℓp−1 and dimW=k. When k+ℓp