SOUL: A S emi-supervised O pen-world contin U al L earning method for Network Intrusion Detection

Suresh Kumar Amalapuram, Shreya Kumar, Bheemarjuna Reddy Tamma · 2024

Read SOUL: A S emi-supervised O pen-world contin U al L earning method for Network Intrusion Detection by Suresh Kumar Amalapuram, Shreya Kumar, Bheemarjuna Reddy Tamma (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-00911.txt.

Introduction

Suresh Kumar Amalapuram and co-authors present SOUL, a continual learning guard for network intrusion in 2024. Attack traffic shifts over time, and labeled data are scarce in security work. A classifier trained on two days of traffic saw attack scores halve within three fresh days in their tests.

SOUL learns from partly labeled streams in an open world, where new attack types appear. A gradient projection memory built from buffer samples guards old knowledge. Confidence scores plus the buffer let the system label new tasks that resemble known ones. Replay of buffer samples fights forgetting. With at most a fifth of labels, scores neared fully supervised rivals.

Catalog record

CREATORSuresh Kumar Amalapuram, Shreya Kumar, Bheemarjuna Reddy Tamma
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-00911

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SOUL: A S emi-supervised O pen-world contin U al L earning method for Network Intrusion Detection Suresh Kumar Amalapuram, Shreya Kumar, Bheemarjuna Reddy Tamma Abstract Fully supervised continual learning methods have shown improved attack traffic detection in a closed-world learning setting. However, obtaining fully annotated data is an arduous task in the security domain. Further, our research finds that after training a classifier on two days of network traffic, the performance decay of attack class detection over time (computed using the area under the time on precision-recall AUC of the attack class) drops from 0.985 to 0.506 on testing with three days of new test samples. In this work, we focus on label scarcity and open-world learning (OWL) settings to improve the attack class detection of the continual learning-based network intrusion detection (NID). We formulate OWL for NID as a semi-supervised continual learning-based method, dubbed SOUL, to achieve the classifier performance on par with fully supervised models while using limited annotated data. The proposed method is motivated by our empirical observation that using gradient projection memory (constructed using buffer memory samples) can significantly improve the detection performance of the attack (minority) class when trained using partially labeled data. Further, using the classifier’s confidence in conjunction with buffer memory, SOUL generates high-confidence labels whenever it encounters OWL tasks closer to seen tasks, thus acting as a label generator. Interestingly, SOUL efficiently utilizes samples in the buffer memory for sample replay to avoid catastrophic forgetting, construct the projection memory, and assist in generating labels for unseen tasks. The proposed method is evaluated on four standard network intrusion detection datasets, and the performance results are closer to the fully supervised baselines using at most 20% labeled data while reducing the data annotation effort in the range of 11 to 45% for unseen data. I Introduction In its cybersecurity readiness index [ 1 ] , Cisco categorizes the organization’s preparedness for cyber threats and identifies that securing the network from malicious actors is critical to the viability of an organization. Network Intrusion Detection Systems (NIDS) [ 2 , 3 ] are an important cybersecurity system/tool to secure a network from cyber threats. Existing NIDS must evolve continuously to combat the ever-evolving cyber attacks [ 4 , 5 , 6 ] . The continuous evolution of such systems requires to meet two desiderata (i) the ability to detect known threats and (ii) leverage the learned knowledge to identify and adapt to novel/unseen attacks . Specifically, existing systems fail to satisfy the first desideratum, which is said to exhibit catastrophic forgetting (CF) [ 7 ] of the learned knowledge. The second desideratum is also known as open-world learning . To reduce the effects of CF, the paradigm of continual learning (CL) [ 8 , 9 ] can be employed. The direct application of CL to security applications is under-explored. Existing CL literature [ 10 , 11 , 9 , 12 , 13 , 14 ] focuses on supervised learning settings requiring annotated data. However, obtaining labeled data is difficult in cybersecurity applications. The complexity of the data annotation in cybersecurity comes from different facets: sample labelling complexity, labelling budget [ 15 ] , and implementation gap [ 16 ] . The sample labelling complexity is the effort required to provide the ground truth labels for each sample. For NID datasets, the sample complexity is in the range of millions. Thanks to automation today, the security research community has paid less attention to the most relevant problem of verifying the correctness [ 17 ] of the auto-labeling process. The labelling budget also affects data annotation. It is difficult for security practitioners to fix the labelling budget due to the variations in the collected data and subsequent labelling efforts, which depend on the specific environment. Eventually, the gap between academic research and real-world security practices is the reason for a lack of established systematic guidelines for the data annotation process in the cybersecurity domain. One way to reduce the reliance on labeled data is to employ the semi-supervised continual learning (SSCL) methods due to their ability to work with limited annotated data. Similar to prior works [ 6 ] , our work also considers the NID problem as a binary classification problem in a domain incremental learning setting. Open-world learning in CL-based NIDS: One of the long-standing goals of the security domain is to identify the unseen attacks ( a.k.a. zero-day attacks) with high confidence by leveraging the learned knowledge. Specifically, the goal is to distinguish zero-day attacks from normality shift detection. However, existing CL-based NIDS detection performance degrades while identifying unseen attacks. This degradation occurs whenever the distribution of the unseen data during inference differs from the training data distribution. While continual learning strategies strive to reduce the effect of CF on the seen tasks, their performance effectiveness remains questionable on unseen tasks. TABLE I : The performance decay of the classifier on the attack class detection on the CICIDS 2017 and CSE-CICIDS-2018 datasets. The reported values are area under time (AUT) computed on the precision-recall area under the curve values of the attack class (AUT (A)). AUT (A) Dataset Seen traffic Unseen traffic CICIDS-2017 [ 17 ] 0.985 0.506 CSE-CICIDS-2018 [ 17 ] 0.999 0.157 We demonstrate this empirically for network intrusion detection tasks using two representative datasets, CICIDS-2017 [ 17 ] and CSE-CICIDS-2018 [ 17 ] . The CICIDS-2017 and CSE-CICIDS-2018 datasets consist of traffic spread over five and ten days, respectively. Initially, a classifier (multilayer perception) is trained on the first two days of traffic in a fully supervised fashion. This model is then tested on the remaining days of traffic. The performance evaluation is measured using the area under the time (AUT) computed on the precision-recall area under the curve (PR-AUC) for attack class data. From the results shown in Table I , we observe that the detection performance of the attack class data on the unseen traffic deteriorates over time. Motivated by the label scarcity and performance decay of the classifier on the unseen traffic, we formulate the open-world learning problem as a semi-supervised continual learning method that requires limited annotated data. The key contributions of this work can be summarized as follows: 1. The formulation of an open-world learning problem for security applications as a semi-supervised CL problem, dubbed SOUL, incorporates test-time distributions with scarce label data for unseen network traffic. 2. Operating under the assumption of no prior knowledge about these unseen tasks, the method produces high-confidence labels in conjunction with buffer memory and model confidence when the unseen traffic bear similarities to the seen traffic. (Section V-C2 ). 3. An evaluation of the proposed method on various benchmarks (Section VI ) which demonstrates that its performance is on par with the fully supervised continual learning baselines using a maximum of 20% labeled data and reduces the data annotation effort in the range of 11 to 45% on the unseen data. The rest of the paper is organized as follows. Section II describes the related work. Section III describes the background details and introduces the notion of orthogonal projections and how to compute them. Section IV describes the SSCL notations for the NID problem. Section V explains the proposed methodology,

Read the complete text