A Nonparallel Support Tensor Machine for Binary Classification based Large Margin Distribution and Iterative Optimization

Zhuolin Du, Yisheng Song · 2025

Read A Nonparallel Support Tensor Machine for Binary Classification based Large Margin Distribution and Iterative Optimization by Zhuolin Du, Yisheng Song (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-13012.txt.

Introduction

Zhuolin Du and Yisheng Song build a new classifier for binary choices that works straight on tensor data. Support vector machines flatten data into vectors, which throws structure away. Their classifier trains on tensors instead. It keeps the built-in structure of high-dimensional data, which should lift accuracy.

The new machine builds on past nonparallel tensor machines. Past those risks, it adds the marginal distribution of the samples. That is the spread of data around the surface that splits the classes, and it lifts scores. The values of that surface form a tensor that must be a sum of rank-one tensors. The team solves the task step by step with an alternating projection method. Each step is a standard support vector machine task along one tensor mode. Tests confirm the model's speed and scores.

Catalog record

CREATORZhuolin Du, Yisheng Song
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-13012

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Nonparallel Support Tensor Machine for Binary Classification based Large Margin Distribution and Iterative Optimization. Thanks: The first author’s work was supported by the National Natural Science Foundation of P.R. China (Grant No.12171064), by The team project of innovation leading talent in chongqing (No.CQYC20210309536) and by the Foundation of Chongqing Normal university (20XLB009) Zhuolin Du, Yisheng Song Abstract Based on the tensor-based large margin distribution and the nonparallel support tensor machine, we establish a novel classifier for binary classification problem in this paper, termed the Large Margin Distribution based NonParallel Support Tensor Machine (LDM-NPSTM). The proposed classifier has the following advantages: First, it utilizes tensor data as training samples, which helps to comprehensively preserve the inherent structural information of high-dimensional data, thereby improving classification accuracy. Second, this classifier not only considers traditional empirical risk and structural risk but also incorporates the marginal distribution information of the samples, further enhancing its classification performance. To solve this classifier, we use alternative projection algorithm. Specifically, building on the formulation where in the proposed LDM-NPSTM, the parameters defining the separating hyperplane form a tensor (tensorplane) constrained to be the sum of rank-one tensors, the corresponding optimization problem is solved iteratively using alternative projection algorithm. In each iteration, the parameters related to the projections along a single tensor mode are estimated by solving a typical Support Vector Machine-type optimization problem. Finally, the efficiency and performance of the proposed model and algorithm are verified through theoretical analysis and some numerical examples. Keywords. Nonparallel support tensor machine; margin distribution; CANDECOMP/PARAFAC (CP) decomposition. AMS subject classifications. 62H30, 15A63, 90C55. 1 Introduction A significant number of real-world datasets, especially those involving image data, are frequently represented in tensor format. For instance, a grayscale face image [ 10 ] can be modeled as a second-order tensor (or matrix), while color images [ 12 , 26 ] , grayscale video sequences [ 13 ] , gait contour sequences [ 22 ] , and hyper spectral cubes [ 28 ] are typically expressed as third-order tensors. Additionally, color video sequences are often represented as fourth-order tensors [ 18 , 38 ] . The tensor-based data representation, with its multi-dimensional structure that complicates the capture of spatial and temporal relationships, high dimensionality prone to the curse of dimensionality and overfitting, and substantial storage and computational requirements, uniquely complicates feature extraction and representation, thereby rendering it a fundamental challenge in the design of classifier models. One of the most representative and successful classification algorithms is the Support Vector Machines (SVM) [ 6 , 30 , 31 ] , which have been successfully applied to a variety of real-world pattern recognition problems, such as text classification [ 15 , 1 ] , image classification [ 29 , 27 ] , feature extraction [ 21 , 33 , 9 ] , web mining [ 3 ] and function estimation [ 4 , 25 ] . The central idea of SVM is to find the optimal separating hyperplane between the positive and negative examples. The optimal hyperplane is defined as the one giving maximum margin between the training examples that are closest to the hyperplane. Different from the traditional SVM, in 2007, Jayadeva et al. [ 16 ] proposed Twin SVM (TWSVM), which also aims at generating two nonparallel planes such that each plane is closer to one of the two classes and is as far as possible from the other. Notably, the formulation of TWSVMs is very much in line with standard SVMs. However, TWSVMs seek to solve two dual QPPs of smaller size rather than solving single dual QPP with large number of parameters in conventional SVM. As a result, the algorithm achieves a processing speed roughly fourfold faster compared to the traditional SVM [ 16 ] . While the aforementioned classification methods focus on maximizing the minimum margin, researchs by Gao et al. [ 11 , 39 ] have indicated that doing so does not necessarily guarantee improved generalization performance. Instead, the distribution of margins has been shown to play a more critical role. Here, the margin distribution is defined as the margin mean and the margin variance. Therefore, in order to improve the generalization performance of SVM, Zhou et al. [ 41 ] characterized margin distribution through its mean and variance, leading to the development of the Large Margin Distribution Machine (LDM), which builds upon the SVM framework. The effectiveness of LDM has been proved in theory and experiments. In recent years, there has been a growing interest in extending traditional vector or matrix-based machine learning algorithms to better handle tensor data [ 5 , 14 ] . This shift is motivated by the need to effectively process high-dimensional datasets, such as those encountered in image and video analysis. In 2005, Tao et al. [ 35 ] proposed a Supervised Tensor Learning (STL) scheme by replacing the vector inputs with tensor inputs and decomposing the corresponding weight vector into a rank-1 tensor, which is trained by the alternating projection optimization method. Based on this learning scheme, in 2007, Tao et al. [ 36 ] further extended the standard linear SVM to a tensorial format known as the Support Tensor Machine (STM). This adaptation allows for more effective classification of tensor data by leveraging its inherent structure. Following this development, Zhang et al. [ 42 ] generalized the vector-based learning algorithm TWSVM to the tensor-based method Twin STM (TWSTM), and implemented the classifier for microcalcification clusters detection. By comparison with TWSVM, the tensor version reduces the overfitting problem significantly. Additionally, Khemchandani et al. developed a least squares variant of STM, termed as Proximal STM (PSTM) [ 17 ] , where the classifier is obtained by solving a system of linear equations rather than a quadratic programming problem at each iteration of PSTM algorithm as compared to STM algorithm. This modification enhances computational efficiency while maintaining classification performance. Tensor-based algorithms on the other hand decompose the whole problem into several smaller and simpler subproblems, each defined over specific tensor modes and characterized by lower dimensionality. This decomposition has been shown to reduce the degree of overfitting that appears in vector-based learning techniques, particularly when few training samples are available [ 36 ] . In this paper, we propose a novel framework, termed Nonparallel Support Tensor Machine based on Large Margin Distribution (LDM-NPSTM), aimed at further enhancing the generalization performance of the Twin Support Tensor Machine (TWSTM). Drawing on the strengths of Large Margin Distribution theory [ 41 ] and TWSTM [ 42 ] , our approach integrates their core principles to address classification challenges more effectively. Specifically, we characterize the margin distribution using first-order (margin mean) and second-order (margin variance) statistics, with the core objective of maximizing the margin mean while minimizing the margin variance to improve classification robustness. To ensure a more rigorous model structure, we incorporate a regularization term into the LDM-NPSTM framework, balancing empirical risk and structural complexity. For model optimization, we adopt an iterative solution based on CANDECOMP/PARAFAC (CP) decomposition. In each iteration, parameters corresponding to projections along a single tensor mode are estimated by solving a typical SVM-type optimization problem. Notably, the inverse matrix involved in the dual problem is inherently nonsingular, eliminating the need for additional assumptions and simplifying

Read the complete text