Generalized Reference Kernel With Negative Samples For Support Vector One-class Classification
Jenni Raitoharju · 2025
Read Generalized Reference Kernel With Negative Samples For Support Vector One-class Classification by Jenni Raitoharju (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-14895.txt.
Introduction
Jenni Raitoharju published this study in 2025. It tackles one-class classification, where a model learns what normal looks like from mostly normal examples. The twist here is that a few negative samples, cases of the other kind, are also at hand. Her method, GRKneg, uses those negatives only to improve the kernel, the similarity measure inside a standard one-class support vector machine. The training itself uses no labels at all.
She studies ways to pick or build the reference points the kernel needs, and recommends one for this setting. Against the standard one-class method, hers won consistently. A full two-class machine did better once negatives were plentiful. But with only a few negatives, the new method clearly beat it. That scarce-negative zone is where the method earns its keep.
Catalog record
| CREATOR | Jenni Raitoharju |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-14895
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generalized Reference Kernel With Negative Samples For Support Vector One-class Classification Jenni Raitoharju Abstract This paper focuses on small-scale one-class classification with some negative samples available. We propose GRKN ( GRKN ) for OCSVM ( OCSVM ). We study different ways to select/generate the reference vectors and recommend an approach for the problem at hand. It is worth noting that the proposed method does not use any labels in the model optimization but uses the original OCSVM implementation. Only the kernel used in the process is improved using the negative data. We compare our method with the standard OCSVM and with the binary SVM ( SVM ) using different amounts of negative samples. Our approach consistently outperforms the standard OCSVM using RBF kernel. When there are plenty of negative samples, the binary SVM outperforms the one-class approaches as expected, but we show that for the lowest numbers of negative samples the proposed approach clearly outperforms the binary SVM . I Introduction One-class classification techniques are used in situations where samples only from a single (positive) class are available, but it is necessary to create a model for recognizing outliers (negative samples) [ 1 ] . In recent years, deep learning-based techniques, e.g., [ 2 ] , have been proposed, but also the traditional techniques, such as OCSVM [ 3 ] and SVDD ( SVDD ) [ 4 ] , are still commonly used in small-scale problems [ 5 ] , [ 6 ] and new variants and extensions [ 7 , 8 ] are being proposed. If information about outliers is available either via expert knowledge or via existence of some negative samples, it should be possible to produce better one-class classification models. This scenario has been extensively considered in large-scale deep learning-based outlier detection [ 9 , 10 ] , but it has received surprisingly little attention in connection to traditional one-class classification techniques. An explanation for this lack of attention could be that these techniques typically perform poorly with negative data. Therefore, when any negative data are available, either binary classifiers or their variants for imbalanced classification, e.g., [ 11 ] , are used instead. SVDD variant considering negative samples was proposed already along the original SVDD [ 4 ] , but the authors concluded that using negative samples in the model optimization often leads to worse results than using only positive samples. More recent SVDD variants using negative samples [ 12 , 13 ] have been able to boost the performance of the standard SVDD , but not to the level of binary SVM . In [ 14 ] , SVM and SVDD were combined to tackle the problem of one-class classification with negative samples. Extensions of OCSVM to consider negative samples have been also proposed [ 15 , 16 ] , but the implementations are not publicly available. A larger number of one-class classification studies, such as [ 17 , 7 ] , have used negative samples for model validation (hyperparameter selection), while they are not used in the core model optimization. In such works, it is typically not evaluated how critical the availability of representative negative samples is for the model performance. Finally, some works have compared one-class and binary classification methods in different settings [ 18 , 19 ] . In this paper, we propose a new method for small-scale one-class classification with negative samples. Our method is based on OCSVM and GRK ( GRK ) [ 20 ] . We propose a novel approach for selecting GRK reference vectors based on negative training samples in a manner that can significantly and consistently improve the method performance. II Generalized Reference Kernel with Negative Samples for OCSVM GRK ( GRK ) [ 20 ] has methodological similarities with approximate kernel approaches, such as random sampling methods [ 21 , 22 ] , random projection methods [ 23 ] , and random Fourier features [ 24 , 25 ] . However, the main idea of GRK is significantly different. The approximate kernel approaches are designed for large datasets, where the original kernel approaches are computationally too expensive. They aim at finding a computationally lighter way to approximate the original kernel matrix or function. GRK , on the other hand, was proposed for one-class classification tasks where the amount of data is low. Instead of trying to approximate the original kernel, GRK aims at providing a better kernel using implicit data augmentation via reference vectors used in the kernel computation. As the reference vectors do not need to be labeled, the problems of data augmentation in the absence of negative data can be avoided. In [ 20 ] , it was shown that selecting the reference vectors in GRK as (positive) training samples augmented with random vectors generated from the training data distribution (assuming standard normal distribution for the standardized training set) could improve the results for SVDD . For OCSVM , data augmentation by sampling reference vectors from the distribution of the positive data was not equally successful, but the initial results suggested that selecting negative samples as reference vectors could improve the results more. However, the potential of using GRK with negative samples was not further studied. In this paper, we propose GRKN . The proposed kernel can be used instead of the regular, usually RBF ( RBF ), kernel in the standard kernel OCSVM , when some negative training samples are available. Our hypothesis is that if we have some negative samples, we can use them to generate more samples from the approximated negative class distribution and improve the OCSVM performance by setting them as reference vectors for GRKN . Indeed, our experiments presented in Section IV show that by setting the reference vectors as negative samples augmented by vectors sampled from the approximated negative distribution, we can significantly boost the OCSVM performance. The full algorithm for computing the GRKN matrix for OCSVM with the proposed reference vector selection approach is provided in Algorithm 1. As in [ 20 ] , we use a tilde to denote the base kernel and related terms used in the process. Derivations for the basic GRK operations, such as centering of the matrices, can be found in [ 20 ] . It is worth emphasizing that the final GRKN kernel size is equivalent to the RBF kernel size, i.e., P\times P , where P is the number of positive training samples, irrespective of the number of reference vectors used in computing the kernel. The negative and generated samples are not used directly in the OCSVM model optimization, but only to form a more discriminative representation for the positive training samples. After forming the GRKN kernel matrix, the original kernel OCSVM implementation can be used for one-class classification. As it is a well-known method, we skip its details here. TABLE I : GRK / GRKN variants used in the experiments Variant Reference vectors 1 P positive training samples 2 P positive + N negative training samples 3 N negative training samples 4 P positive and N negative generated samples 5 P+N negative generated samples 6 P non-positive and N negative generated samples 7 P negative generated samples and N negative training samples (Proposed) 8 2P negative generated samples and