Heaviside Low-Rank Support Matrix Machine

Xian-Chao Xiu, Sheng-Hao Sun, *, Ji-Yuan Tao · 2026

Read Heaviside Low-Rank Support Matrix Machine by Xian-Chao Xiu, Sheng-Hao Sun, *, Ji-Yuan Tao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-00491.txt.

Introduction

Xiu and three colleagues make matrix classifiers tougher against noise by swapping surrogate losses for a Heaviside loss in a low rank support matrix machine. The low rank constraint captures global structure in matrix inputs, which vectorising would scramble. Theory pins down Karush-Kuhn-Tucker points with sufficient and necessary conditions, and a proximal alternating scheme solves every subproblem in closed form. Benchmark tests show the new model beats leading methods on accuracy and robustness.

Catalog record

CREATORXian-Chao Xiu, Sheng-Hao Sun, *, Ji-Yuan Tao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46432
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.00491

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Heaviside Low-Rank Support Matrix Machine Xian-Chao Xiu, Sheng-Hao Sun, *, Ji-Yuan Tao Abstract Support matrix machine (SMM) is an emerging classification framework that directly handles matrix-structured observations, thereby avoiding the spatial correlations destroyed by vectorization. However, most existing SMM variants rely on convex or nonconvex surrogate loss functions, which may lead to high sensitivity to noise. To address this issue, we propose a novel Heaviside low-rank SMM model called HL-SMM, which leverages the Heaviside loss instead of the common hinge or ramp losses for robustness. Moreover, the low-rank constraint is adopted to accurately characterize the inherent global structure. In theory, we analyze the Karush-Kuhn-Tucker (KKT) points and rigorously prove the sufficient and necessary conditions. In algorithms, we develop an effective proximal alternating minimization (PAM) scheme, where all subproblems have closed-form solutions. Extensive experiments on benchmark datasets validate that the proposed HL-SMM achieves superior classification accuracy and robustness compared to state-of-the-art methods. 2023 [2]Xin-Rong Li 1]School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, 200400, China [2]National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Northeastern University, Shenyang, 110819, China 3]Department of Mathematics and Statistics, Loyola University Maryland, Baltimore, MD 21210, USA This paper was supported in part by the National Natural Science Foundation of China under Grant Nos. 12371306, 12501420. Mathematics Subject Classification]90C90, 90C26, 90C30 keywords Classification, Support matrix machine, Heaviside loss, Proximal alternating minimization pacs [ 1 Introduction Classification is a fundamental task in machine learning and pattern recognition, with extensive applications in signal and image processing. For example, in the medical field, classification has been shown to remarkably enhance the accuracy and efficiency of diagnosing various diseases Yan et al. 2023 . Traditional classification methods mainly include support vector machine (SVM) Cortes and Vapnik 1995 , k-nearest neighbors (KNN) Cover and Hart 1967 , and logistic regression (LR) LaValley 2008 . Among these, SVM stands out as a supervised learning method with rigorous mathematical foundations and statistical learning theory, see, e.g., Cervantes et al. 2020 ; Tanveer et al. 2024 for recent surveys. Over the past few decades, SVM has evolved into a series of variants, which can be categorized into linear SVM Chen et al. 2026 , kernel SVM Zhang and Sen 2025 , and deep SVM Okwuashi and Ndehedehe 2020 . By maximizing the separation margin between different classes, SVM exhibits strong generalization ability and excellent performance in processing high-dimensional feature spaces. However, these SVM variants deal with input data represented in the vector form. In reality, most data in the world exists in the matrix form, such as medical images and facial images. As described in Kumari et al. 2025 , converting matrices to vectors not only destroys the inherent spatial correlations but also significantly increases the computational complexity. Recently, the support matrix machine (SMM) was proposed and studied Luo et al. 2015 . It directly processed matrix data with the help of the hinge loss and nuclear norm, ensuring the effectiveness of classification while making full use of the data structure. After that, numerous variants have been developed to enhance its efficiency and generalization. For example, Liang et al. Liang et al. 2022 constructed the least-squares SMM (LS-SMM) by replacing the hinge loss with the least-squares loss for matrix-structured EEG classification. Li et al. Li et al. 2022 introduced non-parallel hyperplanes and used small infrared thermal images for fault diagnosis. Li et al. Li et al. 2023 improved the generalization performance of SMM by extending the input matrix through an auto-correlation function transformation. In fact, the hinge loss function used in SMM is a convex approximation of the Heaviside loss. While this simplifies computation, it makes the method more sensitive to noise. To address this limitation, Feng et al. Feng and Xu 2022 developed a new method with the pinball loss (Pin-SMM) that better captured the underlying data distribution and significantly enhanced its resistance to noise near the decision boundary. Other variants, such as incorporating the truncated hinge loss Qian et al. 2019 and the truncated pinball loss Li and Xu 2024 , are also introduced. Fig. 1 visualizes the coefficient matrices obtained by different loss functions. The more localized and cleaner patterns indicate that the loss can better suppress noise and outliers. It can be seen that the Heaviside loss performs better than others. (a) Hinge loss (b) Pinball loss (c) Truncated pinball loss (d) Heaviside loss Figure 1 : Coefficient matrices obtained by different loss functions. Although these aforementioned SMM methods have obtained excellent noise resistance, the inherent sparsity and low-rank characteristics of the input data were not fully considered, potentially leading to poor performance when dealing with complex intrinsic structures and data volumes Xiu et al. 2024 ; Xiu et al. 2025 . Zheng et al. Zheng et al. 2018 integrated the \ell_{1} -norm with SMM to filter out redundant features, thus achieving more interpretable modeling. Wang et al. Wang et al. 2022 employed a twin non-parallel hyperplane structure with the \ell_{1} -norm regularization to ensure sparsity and discriminativeness. Unlike the previous two methods, Gu et al. Gu et al. 2021 introduced the nonconvex ramp loss function to SMM called Ramp-SMM. Note that the rank in the objective function is often replaced with the nuclear norm for overcoming computational difficulties Li and Luo 2023 . However, this often tends to over-shrink singular values, and thus distort the underlying low-rank structure, especially when the true intrinsic dimension is small Huang et al. 2018 . More importantly, as stated in Zhang et al. 2013 , the solution to the nuclear norm minimization problem is not necessarily the solution to the corresponding rank minimization problem. A natural question arises: can we construct a new SMM variant that combines the Heaviside loss for robustness and the rank constraint for structure preservation? This article provides a definitive answer and proposes a rank-constrained SMM with the Heaviside loss, termed HL-SMM. Specifically, the Heaviside loss improves robustness to noise and outliers, while the rank constraint explicitly enforces the low-dimensional representation. In fact, to our best knowledge, this is the first SMM variant that introducing the Heaviside loss. However, the resulting problem is nonconvex and nonsmooth. To this end, we establish the necessary and sufficient conditions for KKT points and develop an effective PAM algorithm with closed-form updates for each subproblem. Furthermore, we conduct extensive experiments on six benchmark datasets, including robustness evaluation and parameter sensitivity analysis, to evaluate the performance of our proposed method. The remainder of this paper is organized as follows. Section 2 introduces the preliminaries. Section 3 presents the proposed model and optimization algorithm. Section 4 reports the experimental results and discussions. Finally, Section 5 concludes this paper. 2 Preliminaries In this paper, matrices are represented by bold uppercase letters, vectors by bold lowercase letters, and scalars by lowercase letters. Let \mathbb{R}^{p\times q} and \mathbb{R}^{p} be the sets of p\times q -dimensional matrices and p -dimensional vectors, respectively. For a matrix \mathbf{W}\in\mathbb{R}^{p\times q} , denote its transpose as \mathbf{W}^{\top} . The inner product of two matrices \mathbf{W},\mathbf{Y}\in\mathbb{R}^{p\times q} is \langle\mathbf{W},\mathbf{Y}\rangle=\operatorname{tr}(\mathbf{W}^{\top}\mathbf{Y}), where \operatorname{tr}(\cdot) denotes the trace. The Frobenius norm of \mathbf{W} is \|\mathbf{W}\|_{\mathrm{F}}=\sqrt{\langle\mathbf{W},\mathbf{W}\rangle} , and the nuclear norm is \|\mathbf{W}\|_{*} . In addition, the singular value decomposition (SVD) is denoted as \mathbf{W}=\mathbf{U}\,\Sigma(\mathbf{W})\,\mathbf{V}^{\top} , where \mathbf{U}\in\mathcal{O}^{p}

Read the complete text