This is an Author’s Original Manuscript of an article whose final and definitive form, the Version of Record, has been published in the Proceedings of the Third International Conference on Advances in Computing Research (ACR’25) , available online at: https://link.springer.com/chapter/10.1007/978-3-
Markus Haug [0009-0006-9581-6003]*, Gissel Velarde [0000-0001-5392-9540] · 2025
Read This is an Author’s Original Manuscript of an article whose final and definitive form, the Version of Record, has been published in the Proceedings of the Third International Conference on Advances in Computing Research (ACR’25) , available online at: https://link.springer.com/chapter/10.1007/978-3- by Markus Haug [0009-0006-9581-6003]*, Gissel Velarde [0000-0001-5392-9540] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18771.txt.
Introduction
Markus Haug and Gissel Velarde test machine learning classifiers for spotting anomalies in cyber security data. Their paper from 2025 uses two public data sets with strong class imbalance, a network attack set and a credit card fraud set. Odd cases are rare by nature in both fields, which makes them a hard test for classifiers.
Models ran from gradient boosting and a neural network to three generative types. Some were paired with over sampling or ensemble tricks to fight the imbalance. Data were split eighty to twenty for training and tests, with cross checks and three ways to fill missing values. The boosting and neural models beat the generative ones. A costly filling method matched simple ones but is not backed for large sets.
Catalog record
| CREATOR | Markus Haug [0009-0006-9581-6003]*, Gissel Velarde [0000-0001-5392-9540] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-18771
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
This is an Author’s Original Manuscript of an article whose final and definitive form, the Version of Record, has been published in the Proceedings of the Third International Conference on Advances in Computing Research (ACR’25) , available online at: https://link.springer.com/chapter/10.1007/978-3-031-87647-9_25 Performance of Machine Learning Classifiers for Anomaly Detection in Cyber Security Applications Markus Haug [0009-0006-9581-6003]*, Gissel Velarde [0000-0001-5392-9540] Abstract This work empirically evaluates machine learning models on two imbalanced public datasets (KDDCUP99 and Credit Card Fraud 2013). The method includes data preparation, model training, and evaluation, using an 80/20 (train/test) split. Models tested include eXtreme Gradient Boosting (XGB), Multi Layer Perceptron (MLP), Generative Adversarial Network (GAN), Variational Autoencoder (VAE), and Multiple-Objective Generative Adversarial Active Learning (MO-GAAL), with XGB and MLP further combined with Random-Over-Sampling (ROS) and Self-Paced-Ensemble (SPE). Evaluation involves 5-fold cross-validation and imputation techniques (mean, median, and IterativeImputer) with 10, 20, 30, and 50 % missing data. Findings show XGB and MLP outperform generative models. IterativeImputer results are comparable to mean and median, but not recommended for large datasets due to increased complexity and execution time. The code used is publicly available on GitHub ( github.com/markushaug/acr-25 ). † † institute: IU International University of Applied Sciences, Erfurt, 99084, Germany 1 Introduction In this work, we evaluate the performance of several machine learning algorithms for binary classification on tabular data. We compare the performance of supervised and unsupervised learning approaches, and study the case of fraud detection for cyber security. Still, our findings and lessons learned are relevant to any business application and domain where the solution can be framed as a binary classification problem including fraud detection, credit approval, medical diagnosis, or online advertising, to mention a few applications. Indeed, very few business use cases would have balanced data in terms of negative and positive ratios. Generally, the positive class samples appear less frequently than those of the negative class. However, the positive samples are of high interest when diagnosing sicknesses, identifying fraudulent activities or predicting if a customer will click on an ad. Binary classification of imbalance data has been a relevant topic for the machine learning community given that it is challenging to learn statistics from the under represented samples JMLR:v18:16-365 . Approaches to imbalanced learning include hyperparameter tuning wang2020imbalance ; VELARDE2024200354 ; li2023imbalanced , sampling the data before training kim2022 ; hajek2022fraud ; VELARDE2024200354 , and ensembling yang2021progressive ; JMLR:v18:16-365 . Cyber security applications are becoming increasingly important given that cyber crime is on the rise with cost estimations that reach the 400 billion USD globally sarker2020cybersecurity . For this experimental study we systematically evaluated several algorithms on fraud detection datasets, one of the applications of cyber security, however, the main take aways and findings of this study, apply to any application that can be framed as a binary classification problem. The motivation to compare supervised learning with unsupervised learning algorithms is justified since fraud patterns might change over time, fraudsters are very creative and try new possibilities to bypass any system, and therefore, some patterns may never appear during training or might be recognised in a larger time window when it is too late VELARDE2024200354 . Therefore, an approach that can discriminate between legit and fraud activity without the need of collecting labels, is very appealing. In our experiments, we focus first on the evaluation of different approaches and techniques to deal with imbalanced learning. Here we compare the performance of tree based and deep learning approaches in supervised and unsupervised fashion, and test the effect of sampling and ensembling. Then, we investigate the robustness of the best classifier when dealing with missing data, a problem that occurs either because certain features might not be available at the time of data collection, either because of the nature of the process or due to a system error. In the next section we describe the method. 2 The Method The method can be seen in Figure 1 . Each dataset is initially preprocessed in a standardized way so that they can be reused for the various models and sampling methods. The respective properties of the datasets can be seen in Table 1 . Categorical features are first converted into numerical features and then all numerical features are normalized using a Standard Scaler scikit-learn . Model training and evaluation follows a 80/20 (train/test) split. The partitioning is not purely stochastic, but is carried out as part of stratified sampling, so that the respective imbalance ratio (IR) is retained in all created partitions. Models tested include XGB chenXGBoostScalableTree2016 , MLP rumelhartLearningRepresentationsBackpropagating1986 , GAN goodfellowGenerativeAdversarialNets2014 , VAE bankAutoencoders2021 , and MO-GAAL winstonliSourceCodePyod2022 , with XGB and MLP further combined with ROS and SPE. The experiment was executed in a 5-fold cross-validation on the training set with scikit-learn pipelines scikit-learn . After evaluation, XGB was selected to study the effect of imputation techniques (mean, median, and IterativeImputer) with 10, 20, 30, and 50 % missing data. This last step is not represented in Figure 1 . The evaluation is based on the number of True positive (TP), True negative (TN), False positive (FP), and False negative (FN), considering Precision = TP/(TP+FP) , Recall = TP/(TP+FN) and F_{1} - Score = 2\times Recall\times Precision/(Recall+Precision) . Finally, significance tests are performed between imputation techniques. All experiments were conducted on a 2021 MacBook Pro with the Apple M1 Max and 32 GB of RAM. Figure 1: Graphical representation of the Method. Each dataset goes through a Pre-Processing stage before Model Training and Evaluation. Models are trained and tested independently. XGB and MLP are selected and combined with ROS and SPE. 2.1 Datasets We used two highly imbalanced datasets KDDCUP99 and Credit Card Fraud 2013 , see Table 1 . A scikit-learn pipeline was used to intially preprocess the datasets in a standardized way to be reused for the various models and sampling methods. The categorical features were first converted into numerical features and then all numerical features were scaled. Finally, a training and test data partition was created. The partitioning was not purely stochastic, but was carried out as part of stratified sampling, so that the respective IR was retained in all created partitions. Table 1: Characteristics of the datasets used. Data from pozzoloCalibratingProbabilityUndersampling2015 ; KDDCup19991999 . Dataset Negative Samples Positive Samples IR NaN Values Numerical Features Categorical Features Credit Card 284\,315 492 577.88 0 29 0 KDDCUP99 60\,593 250\,436 4.13 0 34 7 2.2 Hyperparameters Table 2: Hyperparameters found for the Credit Card dataset. Model Parameters XGB (Normal) Vanilla XGB with default parameters from xgboostdevelopersXGBoostParameters2022 XGB + ROS objective = Vanilla XGB with default parameters from xgboostdevelopersXGBoostParameters2022 XGB + SPE Vanilla XGB with default parameters from xgboostdevelopersXGBoostParameters2022 MLP (Normal) batch_size = 2048, epochs = 50, learning_rate = 0.01, optimizer = Adam, layers = [256, 256, Dropout(0.3), 256, Dropout(0.3), 1], activation = [ReLU, ReLU, ReLU, sigmoid], metrics = [f1, fn, fp, tn, tp, precision, recall], loss = binary_crossentropy MLP + ROS MLP (Normal), sampling_strategy = 1 MLP + SPE estimator = MLP (Normal), n_estimators = 50 VAE epochs = 10, batch_size = 32, lr=0.001, dropout_rate = 0.2 GAN epochs = 10, latent_dim = 29, batch_size = 32, d_optimizer = SGD with learning rate = 0.0002, g_optimizer = Adam with learning rate = 0.00001 &