NetFlowGen: Leveraging Generative Pre-training for Network Traffic Dynamics
Jiawei Zhou, Woojeong Kim, Zhiying Xu, Alexander M. Rush, Minlan Yu · 2024
Read NetFlowGen: Leveraging Generative Pre-training for Network Traffic Dynamics by Jiawei Zhou, Woojeong Kim, Zhiying Xu, Alexander M. Rush, Minlan Yu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-20635.txt.
Description
Abstract . Understanding the traffic dynamics in networks is a core capability for automated systems to monitor and analyze networking behaviors, reducing expensive human efforts and economic risks through tasks such as traffic classification, congestion prediction, and attack detection. However, it is still challenging to accurately model network traffic with machine learning approaches in an efficient and broadly applicable manner.
Catalog record
| CREATOR | Jiawei Zhou, Woojeong Kim, Zhiying Xu, Alexander M. Rush, Minlan Yu |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-20635
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
NetFlowGen: Leveraging Generative Pre-training for Network Traffic Dynamics Jiawei Zhou, Woojeong Kim, Zhiying Xu, Alexander M. Rush, Minlan Yu Abstract . Understanding the traffic dynamics in networks is a core capability for automated systems to monitor and analyze networking behaviors, reducing expensive human efforts and economic risks through tasks such as traffic classification, congestion prediction, and attack detection. However, it is still challenging to accurately model network traffic with machine learning approaches in an efficient and broadly applicable manner. Task-specific models trained from scratch are used for different networking applications, which limits the efficiency of model development and generalization of model deployment. Furthermore, while networking data is abundant, high-quality task-specific labels are often insufficient for training individual models. Large-scale self-supervised learning on unlabeled data provides a natural pathway for tackling these challenges. We propose to pre-train a general-purpose machine learning model to capture traffic dynamics with only traffic data from NetFlow records, with the goal of fine-tuning for different downstream tasks with small amount of labels. Our presented NetFlowGen framework goes beyond a proof-of-concept for network traffic pre-training and addresses specific challenges such as unifying network feature representations, learning from large unlabeled traffic data volume, and testing on real downstream tasks in DDoS attack detection. Experiments demonstrate promising results of our pre-training framework on capturing traffic dynamics and adapting to different networking tasks. DOI: XXXXXXX.XXXXXXX 1. Introduction Comprehensive understanding and rigorous modeling of traffic dynamics are important for achieving accurate and reliable network service and management. With the capture of rich network data, recent years have seen a surge in applying machine learning (ML) techniques in a wide range of networking tasks, including congestion prediction ( Poularakis et al., 2021 ) and control ( Abbasloo et al., 2020 ; Jay et al., 2019 ; Nie et al., 2019 ; Winstein and Balakrishnan, 2013 ) , packet ( Liang et al., 2019 ; Zhang et al., 2021b ) and traffic classification ( Casino et al., 2019 ; Meng et al., 2022 ; Shen et al., 2019 ; Zhang et al., 2020 ; Zheng et al., 2020 ) , traffic optimization ( Chen et al., 2018 ) , performance prediction and estimation ( Manousis et al., 2020 ; Zhang et al., 2021a ) , network traffic generation ( Ring et al., 2019 ; Yin et al., 2022 ) , and attack detection ( Fu et al., 2018 ; Kharraz et al., 2018 ; Wu et al., 2020b ; Zhou et al., 2020 ) . As a paradigm, ML models achieve strong performance for network applications ( Boutaba et al., 2018 ) , usually through supervised learning from in-task labeled datasets. Nevertheless, while individual ML models are effective for specific applications, it is challenging to transfer learned knowledge and network representations from task to task to achieve better efficiency and generalization. Apart from the repetitive cost of data collection and model development customized for different tasks, the inadequacy of high-quality ground truth labels poses a unique challenge in obtaining reliable models ( Rathore and Park, 2018 ) . Although there naturally exists an ample amount of networking data ( Hong et al., 2009 ; Kornexl et al., 2005 ; Wang et al., 2021 ) , most raw data are unlabeled. Acquiring task-specific labels with different learning purposes is a very expensive process. For example, flow-based traffic data might contain billions of flows and packets, for training a reliable attack detection ML model, manually labeling all the data with high quality is extremely time-consuming even for experts in security ( Ring et al., 2019 ) . The recent success of self-supervised learning from large-scale unlabeled data in natural language processing and related fields motivates us to explore its applications in network traffic dynamics modeling. The self-supervised learning framework contains two phases: a pre-training phase that learns general feature representations from a large amount of unannotated data, and a fine-tuning phase that adapts the general model for particular purposes with a much smaller set of task-specific labels. This transfer learning process uses knowledge encapsulated in the general pre-trained model to transfer to a variety of downstream tasks without redesigning the backbone model architectures and training model parameters from scratch. For network applications, the expensive data labeling issues along with the existence of massive unlabeled networking data make the pre-training paradigm a natural fit as a potential solution for efficient and generalizable ML systems ( Le et al., 2022 ) . In this work, we propose NetFlowGen, a framework for generative pre-training on NetFlow ( Claise, 2004 ) traffic data. Although pre-training frameworks offer potential opportunities for networks, only a handful of works have explored them, primarily discussing benefits ( Le et al., 2022 ) or providing proof of concepts under small-scale simulation setups ( Dietmüller et al., 2022 ) . Challenges persist in developing operational systems with pre-training techniques applied to real-world network traffic data. These include handling the complexities of diverse traffic features, designing effective training objectives with proper model architectures, assembling real-world traffic pre-training datasets at scale, and establishing benchmarks for testing on realistic downstream tasks ( Le et al., 2022 ) . In response to these challenges, we focus on pre-training on flow-based traffic from ISPs (Internet Service Providers), which are in a unique position with access to all or majority of incoming traffic from customers served, employing opportunities for larger data collection and broader usability. While traffic dynamics superficially resemble sequence modeling problems in other fields, there are substantial differences in the raw elements of traffic streams from other data such as language. We design a general feature embedding pipeline that incorporates rich and heterogeneous traffic features from NetFlow data, including IPs, timestamps, ports, transport protocols, packets, bytes, etc., and can be extended for new features. All features are cast into a common space within the pre-trained model, encompassing both incoming and outgoing traffic of an IP node. For model and learning, we use a pre-training task of predicting the next time-step based on the history of traffic information. This generative process relies solely on raw traffic data extracted from NetFlow records ( Claise, 2004 ; Claise, 2008 ) , without any labels. We adopt the Transformer architecture ( Vaswani et al., 2017 ) , commonly used in NLP applications, specifically a decoder-only architecture like OpenAI GPT ( Radford et al., 2018 ) . The resulting model learns to simultaneously generate various traffic features, leveraging diverse signals and feature interactions to acquire comprehensive knowledge representations for downstream tasks. For dataset construction, we, for the first time, utilize a large-scale network traffic corpus collected from a large ISP over three consecutive months for pre-training. We then fine-tune the model on a real task of DDoS early detection to showcase the advantages of large-scale pre-training. Our experiments consider 86 traffic features during the pre-training stage. The resulting models exhibit improved performance in various attack detection tasks for network security, even with limited labeled data. Our preliminary results with NetFlowGen demonstrate the promising benefits of incorporating self-supervised pre-training frameworks into networking research. The general design of our approach can be further extended and tested in diverse environments and applications. 2. Challenges and opportunities Figure 1 . A desirable vision of a network foundation model