Optimization on the Oblique Manifold for Sparse Simplex Constraints via Multiplicative Updates
Flavia Esposito, Andersen Ang · 2025
Read Optimization on the Oblique Manifold for Sparse Simplex Constraints via Multiplicative Updates by Flavia Esposito, Andersen Ang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-24075.txt.
Introduction
Flavia Esposito and Andersen Ang solve a stiff tuning task by walking on a curved shape in a 2025 paper. The task packs three demands at once: parts may not go below zero, most must be zero, and the rest must sum to one, all in a low-rank frame. Such tasks show up in signals, ecology, and gene work.
The team recasts the task on a slanted shape where the sum rule holds by build. A slope walk on that shape keeps each step legal. Tests on fake and real data beat both flat and curved rivals, the team reports.
Catalog record
| CREATOR | Flavia Esposito, Andersen Ang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-24075
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Optimization on the Oblique Manifold for Sparse Simplex Constraints via Multiplicative Updates Flavia Esposito, Andersen Ang Abstract Low-rank optimization problems with sparse simplex constraints involve variables that must satisfy nonnegativity, sparsity, and sum-to-1 conditions, making their optimization particularly challenging due to the interplay between low-rank structures and constraints. These problems arise in various applications, including machine learning, signal processing, environmental fields, and computational biology. In this work, we propose a novel manifold optimization approach to efficiently tackle these problems. Our method leverages the geometry of oblique manifolds to reformulate the problem and introduces a new Riemannian optimization method based on Riemannian gradient descent that strictly maintains the simplex constraints. By exploiting the underlying manifold structure, our approach improves optimization efficiency. Experiments on synthetic and real datasets demonstrate the effectiveness of the proposed method compared to standard Euclidean and Riemannian methods, paving the way for broader applications. MSC Classification]15A23, 65K10, 49Q99, 90C26, 90C30 keywords manifold, optimization, simplex, nonnegativity, sparsity pacs [ 1 Introduction Low-rank decomposition techniques, such as Nonnegative Matrix Factorization (NMF), exploit the fact that high-dimensional matrices can be well-approximated by a sum of rank-one components Gillis (2020) . This reduces the computational complexity of the operations in the optimization while preserving the essential structure of the data Udell and Townsend (2019) . In NMF, a nonnegative matrix \bm{X}\!\in\!\mathbb{R}^{m\times n}_{+} is approximated as the product of two low-rank matrices \bm{W}\!\in\!\mathbb{R}^{m\times r}_{+},\bm{H}\!\in\!\mathbb{R}^{r\times n}_{+} , where r\!\leq\!\min(m,n) is a pre-determined rank parameter. The factor matrices from NMF can be obtained by solving the following minimization problem \min_{\bm{W}\geq 0,\bm{H}\geq 0}\!\frac{1}{2}\|\bm{X}\!-\!\bm{W}\bm{H}\|_{F}^{2} . Additional constraints, such as sparsity or smoothing (to enforce temporal continuity or spatial coherence), are often imposed on the factors \bm{W} or \bm{H} to improve interpretability and performance Del Buono et al. (2025) . In many applications, the matrix \bm{W} is assumed to be known or precomputed, for instance as a dictionary derived from prior information. In this setting, the problem reduces to estimating the coefficient matrix \bm{H} , which typically encodes quantities such as abundances (in hyperspectral unmixing) or activations (in source separation and topic modeling) Gillis (2020) ; Ang and Gillis (2019) ; Yang et al. (2010) ; Casalino et al. (2017) . This subproblem is therefore of independent interest and often involves additional structural constraints. In this work, we focus on the subproblem in \bm{H} while keeping \bm{W} fixed, proposing a novel formulation that differs from standard NMF approaches. This choice is motivated by the fact that, in most applications, the interpretability of the factorization depends primarily on \bm{H} , where the relevant constraints are imposed. We focus on combining three conditions: nonnegativity , sparsity , and sum-to-1 into the subproblem in \bm{H} . By enforcing these constraints on \bm{H} , the factorization acquires a probabilistic interpretation, enabling the evaluation of the relative contribution of each column of \bm{X} (i.e., each sample or element) to the latent factors represented by \bm{H} . These conditions on \bm{H} finds applications: • In biomedical contexts where \bm{X} represents gene expression data (genes as rows and samples as columns), and \bm{H} quantifies the relative contribution or abundance of latent biological factors such as metagenes across samples Brunet et al. (2004) ; Boccarelli et al. (2023) . • In remote sensing, such as hyperspectral unmixing, where columns of \bm{X} contain spectral signatures of each pixel and \bm{H} represents the fractional abundances of endmember materials; enforcing the constraints on \bm{H} ensures physically meaningful interpretations of the contribution of each material within a pixel Ang and Gillis (2019) ; Gillis (2020) ; Settembre et al. (2025) . Specifically, we reformulate the optimization in \bm{H} and solve it directly as a quartic optimization problem, where the nonnegativity and sum-to-1 constraints are intrinsically handled by a Hadamard parametrization on an oblique manifold. As discussed later, this leads to an \ell_{1} -norm regularized quartic formulation. To provide context, we first introduce the classical formulation of the subproblem in \bm{H} , which corresponds to the task of estimating the abundance or activation matrix \bm{H} given a fixed, pre-computed dictionary matrix \bm{W} and the observation matrix \bm{X} . We consider the following Nonconvex-sparse simplex least squares ( NSSls ) problem \argmin_{\bm{H}}\tfrac{1}{2}\|\bm{X}-\bm{W}\bm{H}\|_{F}^{2}+\lambda\|\bm{H}\|_{1/2}~~~~\,\textrm{s.t.}\,~\underbrace{\bm{H}\geq\bm{0},~\bm{H}^{\top}\bm{1}_{r}=\bm{1}_{n},}_{\text{columns of $\bm{H}$ in unit simplex}} (NSSls) where \|\cdot\|_{F} is the Frobenius norm, \lambda\!\in\!\mathbb{R}_{+} is a penalty hyperparameter 1 1 1 Hyperparameters are user-set parameters that affect models’ behavior and performance, such as learning rate, architecture, regularization coefficients, and batch size, and must be tuned for optimal results Esposito et al. (2023) ; here, \lambda controls the trade-off between the data fidelity term and the sparsity-inducing penalty. , \bm{1}_{r}\!\in\!\mathbb{R}^{r} is vector-of-1, and the nonconvex \ell_{1/2} -quasi-norm Xu et al. (2012) , for promoting sparsity 2 2 2 \ell_{1/2} is \ell_{p} -norm with p=1/2 . This is a quasi-norm because it is not homogeneous. , is \textstyle\|\bm{H}\|_{1/2}=\sum_{i=1}^{r}{\sum_{j=1}^{n}{\sqrt{H_{ij}}}}. This latter, \|\bm{H}\|_{1/2} , represents the \ell_{1/2} quasi-norm, which evaluates the sum of the square roots of the absolute values of the entries. The \ell_{1/2} quasi-norm can be seen as a non-convex approximation of the \ell_{0} pseudo-norm that combines the sparsity-promoting effect of the element-wise \ell_{1} -norm with the smoothing properties of the \ell_{2} -norm. By penalizing small coefficients more strongly and large ones more mildly, this quasi-norm drives weaker components toward zero more effectively, yielding sparser and more interpretable solutions. This balance between the \ell_{1} and \ell_{2} behaviors makes the \ell_{1/2} quasi-norm Zeng et al. (2014) particularly useful in applications such as compressive sensing and sparse regression, where both sparsity and stability are desired Chartrand (2007a) ; Chartrand (2007b) . This quasi-norm with this specific power is used as it offsets the squaring effect of the Hadamard parametrization that will be introduced later. As shown, model ( NSSls ) encourages the columns in \bm{H} to be concentrated on the boundary of a simplex (defined by the nonnegativity and sum-to-1 constraints on each column), which promotes sparsity, while maintaining nonnegativity. See Fig. 1 . Figure 1 : The plot of the (nonconvex) \ell_{1/2}- quasi-norm and the simplex constraint. Left: The simplex constraint is represented by the diagonal line intersecting the axes at 1 , visible as the dotted line in the left panel and the diagonal segment in the right panel. Right: the level set plot. Minimizing the \ell_{1/2}- quasi-norm on the simplex gives the solution sitting on the corner of the simplex, leading to sparsity. Problem ( NSSls ) has been addressed in Qian et al. (2011) based on Euclidean geometry. We reformulate the problem within a Riemannian framework, incorporating the constraints directly into the optimization process through fast and efficient updates. Contribution. In this work, we introduce a novel Riemannian approach for solving ( NSSls ). The proposed approach consists of reformulating ( NSSls ) as a manifold optimization problem on the oblique manifold and solving the resulting problem using an adaptation of Riemannian Multiplicative Update (RMU), an optimization method introduced in Esposito and Ang (2025) . Thus, throughout the paper, RMU denotes the optimization algorithm, whereas the proposed approach refers to the combination of the manifold reformulation and the RMU solver. RMU is based on an approximate Riemannian gradient descent and employs a projection-free strategy that maintains the smoothness requirements on the manifold, namely the differentiability of the retraction and the continuity