Sparse Hyperparametric Itakura-Saito NMF via Bi-Level Optimization Journal: Journal of Computational and Applied Mathematics

Laura Selicato, Flavia Esposito, Andersen Ang, Nicoletta Del Buono, Rafał Zdunek · 2025

Read Sparse Hyperparametric Itakura-Saito NMF via Bi-Level Optimization Journal: Journal of Computational and Applied Mathematics by Laura Selicato, Flavia Esposito, Andersen Ang, Nicoletta Del Buono, Rafał Zdunek (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-17123.txt.

Introduction

Laura Selicato and co-authors automate a tuning headache in a 2025 signal processing paper. Splitting a mixed signal into clean parts with nonnegative factorization works well for sound spectra, but penalty settings steer the balance between fit and sparsity, and hand tuning is fiddly. Their algorithm tunes those settings by itself.

A two-level scheme treats the penalties as row-wise values learned in an outer loop while the factorization runs inside. The method shines on sparse, repeating signals buried in noise. A standout use is spotting bearing faults from vibration data, where faint high tones hide under broad noise. Tests beat prior approaches on synthetic and real data.

Catalog record

CREATORLaura Selicato, Flavia Esposito, Andersen Ang, Nicoletta Del Buono, Rafał Zdunek
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-17123

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sparse Hyperparametric Itakura-Saito NMF via Bi-Level Optimization Journal: Journal of Computational and Applied Mathematics Laura Selicato, Flavia Esposito, Andersen Ang, Nicoletta Del Buono, Rafał Zdunek Abstract The selection of penalty hyperparameters is a critical aspect in Nonnegative Matrix Factorization (NMF), since these values control the trade-off between the reconstruction accuracy and the adherence to desired constraints. In this work, we focus on an NMF problem involving the Itakura-Saito (IS) divergence, effective for extracting low spectral density components from spectrograms of mixed signals, enhanced with sparsity constraints. We propose a new algorithm called SHINBO, which introduces a bi-level optimization framework to automatically and adaptively tune the row-dependent penalty hyperparameters, enhancing the ability of IS-NMF to isolate sparse, periodic signals against noise. Experimental results showed SHINBO ensures precise spectral decomposition and demonstrates superior performance in both synthetic and real-world applications. For the latter, SHINBO is particularly useful, as noninvasive vibration-based fault detection in rolling bearings, where the desired signal components often reside in high-frequency subbands but are obscured by stronger, spectrally broader noise. By addressing the critical issue of hyperparameter selection, SHINBO advances the state-of-the-art in signal recovery for complex, noise-dominated environments. 1 Introduction Nonnegative Matrix Factorization (NMF) is a dimensionality reduction technique that approximates a nonnegative data matrix as the product of two (lower-dimensional) nonnegative matrices. A key challenge in NMF is setting penalty coefficients when additional constraints, such as sparsity or smoothness, are imposed [ 18 ] . These coefficients control the trade-off between reconstruction accuracy and constraint adherence, but their optimal values are highly dataset- and application-dependent, making the selection process non-trivial. For example, [ 6 ] proposes a variant of NMF that incorporates data-dependent penalties and introduces auxiliary constraints to enhance performance in tasks such as face recognition. Additionally, [ 20 ] presents multiplicative algorithms for NMF that enforce non-negativity and flow preservation constraints while introducing regularizations to ensure smoothness or sparsity. Finally, [ 11 ] adapts a minimization scheme for functions with non-differentiable constraints, known as PALM, to solve NMF problems, yielding solutions that can be both smooth and sparse—two highly desirable properties. In this work, we rely on prior studies [ 8 , 9 ] in which the penalized problem is reformulated in a general form, and a strategy is proposed to tune the penalty coefficient automatically. Formally, let \bm{X}\in\mathbb{R}^{m\times n}_{+} with m,n\in\mathbb{N} be a data matrix, NMF aims to approximate it as the product of \bm{W}\in\mathbb{R}^{m\times r}_{+} and \bm{H}\in\mathbb{R}^{r\times n}_{+} with r\leq\min\{m,n\} , so that \bm{X}\approx\bm{W}\bm{H} . In our problem, we want to solve (\bm{W}^{*},\bm{H}^{*})~\in\displaystyle\argmin_{\begin{subarray}{c}\bm{W}\geq\bm{0},\bm{H}\geq\bm{0}\end{subarray}}D_{\beta}(\bm{X},\bm{W}\bm{H})+\mathcal{P}\big(\textrm{Diag}(\bm{\lambda})\bm{H}\big) (1) with the objective function D_{\beta}(\cdot,\cdot) being the \beta -divergence [ 13 , 19 ] , assessing the quality of the reconstruction \bm{W}\bm{H} in fitting \bm{X} . We remind that the \beta -divergence for matrices is defined as D_{\beta}(\bm{A},\bm{B})=\sum_{i}{\sum_{j}{d_{\beta}(a_{ij},b_{ij})}} , where the function d_{\beta} for each x,y\in\mathbb{R} is defined as d_{\beta}(x,y)=\left\{\begin{array}[]{lc}\frac{1}{\beta(\beta-1)}(x^{\beta}+(\beta-1)y^{\beta}-\beta xy^{\beta-1})&\beta\in\mathbb{R}\setminus\{0,1\};\\ x\log(\frac{x}{y})-x+y&\beta=1;\\ \frac{x}{y}-\log(\frac{x}{y})-1&\beta=0.\end{array}\right. The function \mathcal{P}:\mathbb{R}^{r\times n}\to\mathbb{R} is a penalty term on \bm{H} that enforces a particular constraint. The vector \bm{\lambda}\in\mathbb{R}^{r}_{+} in ( 1 ) contains a nonnegative penalty hyperparameters [\lambda_{1},\dots,\lambda_{r}] associated to each row of \bm{H} . In this study, we are interested in a special case of the \beta -divergence. When \beta=0 , the \beta -divergence boils down to the Itakura-Saito (IS) divergence which is a scale-invariant measure of dissimilarity, useful for extracting low spectral density components from spectrograms of mixed signals. In many practical applications, the desired components are located in a high frequency subband and are often masked by much stronger and spectrally wider noisy disturbances. This is the case of a noninvasive vibration-based fault detection in rolling bearings [ 23 , 24 , 25 ] . The vibrational diagnostic signals measured from faulty bearings on the laboratory test rig [ 16 ] are used in our study to validate the proposed algorithm. The signal of interest (SOI) in this application is represented by a periodic and impulsive signal. The observed signal should contain the SOI and other perturbations, usually expressed by a strong independent and identically distributed (i.i.d.) Gaussian noise. Applying NMF to the spectrogram of the measured signal, we expect that the representative of the SOI will be expressed by one of the temporal components of NMF, i.e., one row of matrix \bm{H} . This component, which presents a periodic spiky signal, is obviously very sparse. The other components, which represent the noisy perturbations, should not be sparse. To enforce NMF to search for the desired component, we introduced the penalty for the rows of \bm{H} using the term \mathcal{P}(\textrm{Diag}(\bm{\lambda})\bm{H}) in ( 1 ). The proposed bi-level approach should perform data-driven adaptation of the hyperparameters (vector \bm{\lambda} ) to the desired nature of the estimated components. Contribution The contribution of this work is two folded. 1. New model . In this work, we present a new model for minimizing the Itakura-Saito divergence (Problem ( 1 ) with \beta=0 ) while penalizing rows of \bm{H} . In particular (a) The penalty hyperparameter is not known in advance. This parameter is formulated as an optimization variable (in form of bi-level optimization model) and is solved by a bi-level optimization method. (b) The penalty hyperparameter is row-dependent. Note that Problem ( 1 ) is not the standard penalized NMF [ 18 ] , which applies the same penalty coefficient to all rows in the matrix \bm{H} . In Problem ( 1 ) each row of \bm{H} is penalized by its own penalty parameter. 2. New algorithm . For Problem ( 1 ), we present a new multiplicative update (see Equation ( 12 )), and a way to automatically tune the penalty hyperparameter based on bi-level strategy (see details in Section 3 ). Paper Organization We introduce the problem and overall algorithm framework in Section 2 . In Section 3 we first review the bi-level optimization, then we discuss the details of the bi-level approach proposed in this work for solving the Problem 1 in Section 4 . Experimental results on synthetic and real datasets are presented in Section 5 . We conclude the paper in Section 6 , giving an outline of possible future directions. Notation The symbol \bm{0} denote the zero matrix, the symbol \bm{E}_{a\times b} is all-one matrix sizing a\times b with a,b\in\mathbb{N} . The notation \bm{v} denotes a column vector and the notation \underline{\bm{v}} means \bm{v} is a row vector. On matrix, \bm{A}^{\top} is the transpose of \bm{A} , and \bm{A}^{2}=\bm{A}\bm{A} . The symbol \bm{A}\odot\bm{B} refers to the Hadamard (element-wise) product between \bm{A} and \bm{B} of conformal dimensions, and the symbol \bm{A}\oslash\bm{B} with \bm{B}\neq\bm{0} refers to the Hadamard division, and we denote \bm{A}^{\odot k} as the Hadamard power- k of \bm{A} . Given n\in\mathbb{N} , we denote [n]\coloneqq\{1,2,\ldots,n\} . The symbols k,t\in\mathbb{N} indicate iteration counters. The symbol \bm{A}^{k} refers to the variable \bm{A} at the iteration k , A_{ij} or a_{ij} is the (i,j) -th element of \bm{A} . Lastly \bm{A}_{i:} and \bm{A}_{:j} are the i -th and j -th row and column of \bm{A} , respectively. 2 The overall optimization framework of \AlgoName In this section, we discuss the main focus of the paper, Problem ( 1 ) and the overall framework of the proposed algorithm. The optimization problem We focus on Problem ( 1 ) with \beta=0 . As \mathcal{P} we

Read the complete text