LLMs’ Suitability for Network Security: A Case Study of STRIDE Threat Modeling Author’s draft for soliciting feedback - May 6, 2025
AbdulAziz AbdulGhaffar, Ashraf Matrawy · 2025
Read LLMs’ Suitability for Network Security: A Case Study of STRIDE Threat Modeling Author’s draft for soliciting feedback - May 6, 2025 by AbdulAziz AbdulGhaffar, Ashraf Matrawy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04101.txt.
Introduction
AbdulAziz AbdulGhaffar and Ashraf Matrawy test whether language models can do structured threat modeling for networks, in a 2025 paper. The STRIDE method sorts threats into six classic classes. Fifth generation network threats supplied the test cases.
Five language models were tried with four prompting styles. Each model classified threats under the STRIDE headings. Results varied a lot by threat type and prompt. The authors trace the quirks to how the models were trained. Their verdict: raw models are not yet fit for security work without tuning. The study is an early map of where the failures cluster.
Catalog record
| CREATOR | AbdulAziz AbdulGhaffar, Ashraf Matrawy |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-04101
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLMs’ Suitability for Network Security: A Case Study of STRIDE Threat Modeling Author’s draft for soliciting feedback - May 6, 2025 AbdulAziz AbdulGhaffar, Ashraf Matrawy Abstract Artificial Intelligence (AI) is expected to be an integral part of next-generation AI -native 6G networks. With the prevalence of AI , researchers have identified numerous use cases of AI in network security. However, there are almost nonexistent studies that analyze the suitability of Large Language Models (LLMs) in network security. To fill this gap, we examine the suitability of LLMs in network security, particularly with the case study of STRIDE threat modeling. We utilize four prompting techniques with five LLMs to perform STRIDE classification of 5G threats. From our evaluation results, we point out key findings and detailed insights along with the explanation of the possible underlying factors influencing the behavior of LLMs in the modeling of certain threats. The numerical results and the insights support the necessity for adjusting and fine-tuning LLMs for network security use cases. I Introduction Future networks, such as Sixth Generation (6G) networks, are envisioned to integrate Artificial Intelligence (AI) into their networks to be AI -Native networks [ 1 ] to improve performance, efficiency, and scalability [ 2 ] . Ericsson’s report [ 3 ] indicates that deploying AI in telecom networks will not only reduce the Operational Expenditure (OPEX) of the network but also provide a 5\% to 10\% return on investment. On the other hand, with the increasing popularity of AI and Large Language Models (LLMs) , researchers are identifying potential applications and use cases of AI and LLMs in networks [ 4 , 5 , 6 ] . These potential use cases include, but are not limited to, network optimization [ 4 ] , automation of security [ 5 ] , and threat classification [ 6 ] . Upon examining the literature, we notice a significant gap where there is a lack of work analyzing and investigating the suitability of LLMs in the proposed network security use cases. This motivated us to investigate the suitability of LLMs in network security use cases. Due to the importance of threat modeling as a starting point in any security exercise, we focus on the “ Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege (STRIDE) ” threat model [ 7 , 8 ] . We have extensive experience with Fifth Generation (5G) threat modeling using STRIDE [ 9 , 10 ] . Hence, in this work, we select a case study of STRIDE threat modeling to perform LLM -based classification of 5G threats. We perform the experiments using four prompting techniques with five different LLMs . This work is important as it provides insights on using LLMs for threat classification in next-generation ‘ AI -Native ’ telecom networks. The main contributions of this work are as follows: 1. We investigate the suitability of Large Language Models (LLMs) in network security use cases. For this purpose, we select the case study of STRIDE threat modeling of 5G threats and vulnerabilities. We perform experiments by selecting six 5G threats and utilizing four different prompting techniques with five LLMs . 2. We provide detailed insights based on the evaluation results of LLM -based STRIDE classification. We provide detailed discussions on potential underlying factors that influence the behavior of LLMs in modeling certain threats, including: incorrect threat perspective, failure to identify second-order threats, and insights on Few-Shot (FS) prompting positively impacting performance. 3. We analyze the suitability of LLMs using numerical testing and various performance metrics, including accuracy, precision, recall, and F1 score. Our results indicate that the performance of the selected LLMs is comparable, highlighting the need for enhancements in these models for STRIDE threat modeling in 5G networks. The paper is organized as follows: Section II provides the motivation of our work in light of the examined related works. Section III explains the evaluation methodology we use for the STRIDE threat modeling case study. The detailed results and insights of the evaluation are presented in Section IV . Finally, Section V provides the discussion and conclusion of our study. II Motivation and Examination of Related Work With the advent of LLMs , many researchers have identified various potential use cases of LLMs in telecom networks and cybersecurity. We present the most relevant papers in this section. LLM for Networking: The white paper by Shahid et al. [ 4 ] presents the concept of “ Large Telecom Model (LTM) ” for use cases of telecom networks. Some of the potential use cases they mention include the use of LTM at the network edge, LTM for network optimization, using LTM for network automation tasks, and performing intent-based management using LTMs , etc. Similarly, Zhou et al. [ 5 ] identify four different areas of telecom networks that may benefit from LLMs . These areas include generation, optimization, classification, and prediction problems in telecom networks. Wu et al. [ 11 ] present the NetLLM method to utilize LLMs for three different networking problems, namely, “viewport prediction”, “adaptive bitrate streaming”, and “cluster job scheduling”. The authors extensively evaluate the performance of their proposed framework within these problems. LLM for Security: Ferrag et al. [ 6 ] propose multiple applications of LLMs in cybersecurity, including detection and analysis of threats, incident response, automation of security tasks, verification of security protocols, etc. The work by Guthula et al. [ 12 ] proposes a foundation model for security that takes into account the distinct nature of network traffic. The aim of the authors is to consider the general applicability of the model. Sędkowski [ 13 ] studies the efficacy of LLMs in recognizing potential threats in the network and recommending countermeasures. Their methodology includes using Nmap reports to classify threats using STRIDE threat modeling with three different LLMs . The author concludes that the performance of LLMs in threat detection is comparable to humans. The scope of their work is focused on testing the application of AI in threat modeling, as opposed to our aim of studying the suitability of LLMs for network security. Yang et al. [ 14 ] present “ThreatModeling-LLM”, which is an LLM -based method to perform threat modeling of the banking system using the STRIDE model. Their approach includes various steps to improve the performance of the LLMs . Their scope is limited to banking systems. The main objective of their work is to improve and automate threat modeling using LLMs , instead of investigating the suitability of LLMs for network security. Motivation: After examining the related works, we identify that the suitability of the potential use cases of LLMs needs to be investigated. However, to the best of our knowledge, there are almost nonexistent or very scarce existing studies that are actually testing the suitability of LLMs for these use cases. This motivated us to study the suitability of the LLMs in telecom network security using a case study of STRIDE threat modeling. III Evaluation Methodology for A Case Study of STRIDE Threat Modeling The aim of this case study is to employ LLMs to categorize the threats and vulnerabilities on 5G interfaces using the STRIDE model. The main objective is to evaluate various prompting and search techniques