Revisiting Network Traffic Analysis: Compatible network flows for ML models
João Vitorino, Daniela Pinto, Eva Maia, Ivone Amorim, Isabel Praça · 2025
Read Revisiting Network Traffic Analysis: Compatible network flows for ML models by João Vitorino, Daniela Pinto, Eva Maia, Ivone Amorim, Isabel Praça (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08345.txt.
Introduction
Joao Vitorino, Daniela Pinto, Eva Maia, and Ivone Amorim revisit network traffic analysis by rebuilding data set flows with one consistent tool. Machine learning detectors for cyberattacks need high quality traffic traits, yet popular data sets were processed by varied exporters whose similar looking traits behave differently. Internet of things attack patterns are above all hard to represent faithfully in prebuilt spreadsheet files.
The team reanalyzed raw packet files from Bot-IoT, IoT-23, and CICIoT23 with the HERA tool, producing new labeled flows and traits. Models tuned on the rebuilt versions, above all bagging and gradient boosting tree ensembles, generalized better than those using original files. Consistent extraction makes cross data set comparison more trustworthy.
Catalog record
| CREATOR | João Vitorino, Daniela Pinto, Eva Maia, Ivone Amorim, Isabel Praça |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-08345
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Revisiting Network Traffic Analysis: Compatible network flows for ML models João Vitorino, Daniela Pinto, Eva Maia, Ivone Amorim, Isabel Praça Abstract To ensure that Machine Learning (ML) models can perform a robust detection and classification of cyberattacks, it is essential to train them with high-quality datasets with relevant features. However, it can be difficult to accurately represent the complex traffic patterns of an attack, especially in Internet-of-Things (IoT) networks. This paper studies the impact that seemingly similar features created by different network traffic flow exporters can have on the generalization and robustness of ML models. In addition to the original CSV files of the Bot-IoT, IoT-23, and CICIoT23 datasets, the raw network packets of their PCAP files were analysed with the HERA tool, generating new labelled flows and extracting consistent features for new CSV versions. To assess the usefulness of these new flows for intrusion detection, they were compared with the original versions and were used to fine-tune multiple models. Overall, the results indicate that directly analysing and preprocessing PCAP files, instead of just using the commonly available CSV files, enables the computation of more relevant features to train bagging and gradient boosting decision tree ensembles. It is important to continue improving feature extraction and feature selection processes to make different datasets more compatible and enable a trustworthy evaluation and comparison of the ML models used in cybersecurity solutions. 1 Introduction Securing the Internet-of-Things (IoT) is a never-ending challenge. The complexity of IoT networks, including distributed network topologies, vulnerable devices, and diverse network traffic patterns, makes it difficult to deploy traditional intrusion detection systems [ 16 ] . Instead, modern cybersecurity solutions are trying to adapt to evolving cyber threats by integrating artificial intelligence, using Machine Learning (ML) models that are trained to identify suspicious network activity and detect possible zero-day attacks [ 2 ] . To ensure that an ML model is robust against abnormal data, it is essential to train it with high-quality and representative data of an IoT network. Since ML models commonly require the datasets to be in a simple tabular format, it is necessary to prepare Network Traffic Analysis (NTA) datasets by converting network traffic into features that ML models can process [ 3 ] . Flow exporter tools can preprocess the raw network packets of an IoT network, aggregating them to generate network traffic flows with statistical features that summarise the main characteristics of recent network activity [ 1 ] . However, it can be difficult to accurately convert and represent the complex traffic patterns of the evolving cyberattacks targeting IoT networks [ 19 ] . Different flow exporters may implement varying approaches for traffic aggregation, flow filtering, and feature computation, leading to discrepancies in the generated network flows [ 10 ] . These discrepancies can cause some portions of a dataset to contain incorrect or biased data, which affects both the generalization and the robustness of ML models, especially in an imbalanced multiclass classification task, which is the case of IoT intrusion detection [ 18 ] . To address this concern, this paper studies the impact that the features created by different flow exporters can have on the robustness of IoT cyberattack classification models. In addition to the original Comma-Separated Values (CSV) files of the Bot-IoT [ 8 ] , IoT-23 [ 4 ] , and CICIoT23 [ 9 ] datasets, their raw packet capture (PCAP) files were analyzed with a different flow exporter. The developed Holistic nEtwork featuRes Aggregator (HERA) tool [ 10 ] was used to generate new labelled flows and extract consistent features, resulting in new CSV versions that were compared with the original versions. To assess the usefulness of the new flows for the training of ML models for NTA and intrusion detection, multiple Random Forest (RF), Extreme Gradient Boosting (XGB), and Light Gradient Boosting Machine (LGBM) models were fine-tuned and evaluated. This paper is organized into multiple sections. Section 2 provides an overview of the recent scientific developments. Section 3 details the utilized tool, datasets, and ML models. Section 4 provides an analysis and discussion of the obtained results. Finally, Section 5 presents the main conclusions and future work. 2 Related Work In recent years, there has been a growing focus on the reliability and usability of NTA datasets for the training and validation of ML models. As the cyberattacks targeting modern IoT networks started becoming more complex, researchers started identifying limitations in how existing datasets represented that complex behaviour in a tabular data format [ 17 ] . Therefore, it is important to understand the conclusions and recommendations of previous work. Some well-established studies have carefully analyzed the NTA datasets that were publicly available at the time. Kenyon et al. [ 7 ] and Ring et al. [ 12 ] considered several factors, such as the quality and relevance of the captured cyberattacks, the methodology used to create and capture the traffic patterns, and the maintenance to keep it up-to-date. These studies revealed that most datasets suffer from over-summarization and have an unclear representation of benign and malicious traffic characteristics. These issues, combined with a lack of standardized methodologies for network traffic simulation, often cause the generated network flows to be incomplete and have inconsistent labels [ 1 ] . Furthermore, most public NTA datasets provide a preprocessed version that loses relevant information. Even though the raw traffic is captured in PCAP files with the same format, the features made available by the original authors of each dataset are often specific to that work, and it is not specified how they were computed [ 13 ] . This results in CSV files with seemingly similar features, but that actually represent different characteristics and have missing or incompatible data, which limits the ability to combine data from different origins [ 5 ] . Despite ongoing efforts to systematise the feature sets used for NTA, it is still difficult to identify which features best represent complex attack scenarios. In more recent studies, Sarhan et al. [ 14 ] and Silva et al. [ 15 ] , evaluated different NTA datasets, including Bot-IoT and IoT-23 focused on IoT networks. Several experiments were performed to select the most relevant features for ML models to keep a good generalization while providing a more computationally efficient detection. The resulting feature sets contained mostly time-related features of active transmission rates and interpacket arrival times, but some complex cyberattack classes require the use of quantity-related features, such as sent and received bytes and packets [ 20 ] . Overall, the recent literature highlights the need to perform feature engineering to better represent the characteristics and traffic patterns of more complex cyberattacks. To the best of our knowledge, no previous work has analyzed the effect that extracting new network traffic flow features from the raw network packets of the Bot-IoT, IoT-23, and CICIoT23 datasets has on the performance of ML models for multiclass cyberattack classification in IoT networks. 3 Methodology This section details the traffic analysis tool, IoT datasets, and ML models used in the study which was carried out on a relatively lightweight machine equipped with a 6-core central processing unit and 16 gigabytes of random access memory. The main programming language was Python, using