LLM-based Multi-class Attack Analysis and Mitigation Framework in IoT/IIoT Networks

Seif Ikbarieh, Maanak Gupta, Elmahedi Mahalal · 2025

Read LLM-based Multi-class Attack Analysis and Mitigation Framework in IoT/IIoT Networks by Seif Ikbarieh, Maanak Gupta, Elmahedi Mahalal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-26941.txt.

Introduction

Seif Ikbarieh, Maanak Gupta, and Elmahedi Mahalal build a hybrid framework to analyse attacks on internet-of-things networks. Connected devices have spread fast and widened the attack surface. AI helps detect attacks, analyse their behaviour, and suggest mitigations, but evaluations of that analysis have stayed purely qualitative, with no standard benchmark.

Ikbarieh and his two co-authors pair machine learning for multi-class attack detection with large language models for behaviour analysis and mitigation advice. They benchmark classifiers on two IoT security data sets, finding random forest best for detection. Structured role-play prompts with retrieval augmentation then guide two leading models, and an ensemble of eight judge models scores their answers with new quantitative metrics. ChatGPT-o3 beat DeepSeek-R1 on analysis and mitigation in their tests.

Catalog record

CREATORSeif Ikbarieh, Maanak Gupta, Elmahedi Mahalal
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-26941

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LLM-based Multi-class Attack Analysis and Mitigation Framework in IoT/IIoT Networks Seif Ikbarieh, Maanak Gupta, Elmahedi Mahalal Abstract The Internet of Things has expanded rapidly, transforming communication and operations across industries but also increasing the attack surface and security breaches. Artificial Intelligence plays a key role in securing IoT, enabling attack detection, attack behavior analysis, and mitigation suggestion. Despite advancements, evaluations remain purely qualitative, and the lack of a standardized, objective benchmark for quantitatively measuring AI-based attack analysis and mitigation hinders consistent assessment of model effectiveness. In this work, we propose a hybrid framework combining Machine Learning (ML) for multi-class attack detection with Large Language Models (LLMs) for attack behavior analysis and mitigation suggestion. After benchmarking several ML and Deep Learning (DL) classifiers on the Edge-IIoTset and CICIoT2023 datasets, we applied structured role-play prompt engineering with Retrieval-Augmented Generation (RAG) to guide ChatGPT-o3 and DeepSeek-R1 in producing detailed, context-aware responses. We introduce novel evaluation metrics for quantitative assessment to guide us and an ensemble of judge LLMs, namely ChatGPT-4o, DeepSeek-V3, Mixtral 8x7B Instruct, Gemini 2.5 Flash, Meta Llama 4, TII Falcon H1 34B Instruct, xAI Grok 3, and Claude 4 Sonnet, to independently evaluate the responses. Results show that Random Forest has the best detection model, and ChatGPT-o3 outperformed DeepSeek-R1 in attack analysis and mitigation. I Introduction The Internet of Things (IoT) is a large network of interconnected devices that communicate over the internet to deliver valuable data. The adoption of IoT has surged over the past decade, transforming operations across sectors. In 2024, the global IoT market was valued at about $714.48 billion and is projected to reach $4.06 trillion by 2032, growing at a 24.3% Compound Annual Growth Rate (CAGR) [ 1 ] . Within the realm of IoT, Industrial IoT (IIoT) integrates IoT into industrial processes, boosting efficiency in manufacturing, energy, healthcare, transportation, and many other industries. The growth of IoT also expands the attack surface for cybercriminals. IoT devices often handle sensitive data, and their interconnected nature means a single vulnerability can compromise an entire network. Attacks on IoT have increased by 124% in 2024 alone [ 2 ] , underscoring the need for robust security measures to protect such environments, supporting the efficient management and secure growth of connected technologies. Clearly, securing these IoT-driven ecosystems is critical as our society embraces digital transformation, which is the core contribution of this work. Artificial Intelligence (AI) has become important in modern cybersecurity, enhancing defense mechanisms and reducing financial impact. Organizations using AI-based security prevention save an average of $2.22 million over the organizations that did not deploy such technologies [ 3 ] . Machine Learning (ML) detects attacks by analyzing manually selected features using algorithms such as Random Forest (RF) and Support Vector Machines (SVM). Deep Learning (DL) learns to extract hierarchical features from raw data using neural networks such as Convolutional Neural Network (CNN) for spatial analysis and Long Short-Term Memory (LSTM) for classifying sequential patterns. Both ML [ 4 ] and DL [ 5 , 6 ] have been widely used for IoT attack detection. Despite significant advances in AI for IoT security, there exists several critical challenges and research gaps . Firstly, current ML and DL models excel at attack detection but cannot analyze attack behaviors or provide actionable mitigation suggestions, limiting their operational practicality. Secondly, while existing cybersecurity applications using Large Language Models (LLMs) have shown potential for attack behavior analysis and mitigation suggestion [ 7 , 8 ] , they have relied solely on qualitative evaluation. This highlights the need for a benchmark that introduces standardized, objective metrics to quantitatively assess LLM-generated analyses and mitigation suggestions, driving the development of more effective AI-driven IoT security solutions. Motivated by those gaps, our contributions in this work are as follows: • We propose a novel hybrid framework combining ML-based attack detection with LLM-based reasoning to analyze attack behavior and generate tailored mitigations for 13 different attack types in IoT/IIoT networks. • We benchmark multiple ML and DL classifiers on the Edge-IIoTset [ 9 ] and CICIoT2023 [ 10 ] datasets, integrating the top performer into our framework. • We leverage structured role-play prompt engineering with Retrieval-Augmented Generation (RAG) to guide ChatGPT-o3 [ 11 ] and DeepSeek-R1 [ 12 ] in detailed attack analysis and tailored mitigation suggestion. • We introduce a novel set of scoring metrics, eight judge LLMs and human expert use to independently assess and compare the LLM-generated attack behavior analyses and mitigation suggestions, enabling quantitative performance measurement and objective effectiveness comparison between ChatGPT-o3 and DeepSeek-R1. The remaining sections of this paper are structured as follows. Section II reviews related work on AI-based IoT attack detection, LLM prompting for attack analysis and mitigation, and the use of LLMs as judges. Section III describes the Edge-IIoTset and CICIoT2023 datasets. Section IV outlines our framework, including attack detection, RAG pipeline, and prompt engineering approach. Section V presents our evaluation metrics and results. Finally, section VI concludes the paper and discusses future research directions. II Related works ML and DL classifiers have shown promising results for attack detection on IoT networks. The authors in [ 4 ] benchmark seven different ML models for IoT attack detection on the Bot-IoT dataset, where RF achieved 99% accuracy for binary classification, and K-Nearest Neighbor (KNN) achieved 99% accuracy for multi-class classification. Taşcı in [ 5 ] proposes an optimized 1D CNN model for IoT attack and malware detection. The model achieves an accuracy of 98.36% on the CICIoT2023 dataset, 99.90% on CIC-MalMem-2022 dataset, and 99.99% on CIC-IDS2017 dataset. The authors in [ 6 ] propose an intrusion detection system (IDS) for IIoT environments using a hybrid LSTM-CNN-Attention model. Their approach integrates LSTM for temporal dependency modeling, CNN for spatial feature extraction, and an attention mechanism to enhance feature selection. The model is evaluated on the Edge-IIoTset dataset, and achieved 99.04% accuracy for multi-class classification, outperforming other DL models. LLMs have recently been explored for security tasks that go beyond detection. The authors in [ 7 ] introduce ShieldGPT, which applies prompt engineering methods on GPT-4 to examine Distributed Denial of Service (DDoS) traffic and generate device-specific defense strategies. ShieldGPT’s four components, namely attack detection, traffic representation, domain knowledge injection, and role representation, allow LLMs to examine network traffic and propose mitigations. Additionally, the authors in [ 8 ] introduce ChatIDS, which is designed for IoT environments such as smart homes, and aims to make IDS outputs more accessible to non-expert users. ChatIDS takes alerts generated by Suricata, Snort, and Zeek, enriches them with metadata, and passes them to ChatGPT to generate explanatory text and suggest mitigations. The use of LLMs as judges is an emerging direction for automating evaluative tasks such as scoring, ranking, and verification. Authors in [ 13 ] review this line of work, outlining a structured evaluation pipeline with prompt-based querying, model selection, and post-processing. To enhance evaluation quality, they highlight techniques such as prompt engineering, majority voting, and fine-tuning on representative datasets. While ML and DL classifiers demonstrate promising performance in IoT attack detection, they lack the ability to analyze attack behaviors and provide actionable mitigation suggestions, limiting their practicality in real-world cybersecurity operations. LLM-based approaches address this by using prompt engineering for behavior analysis and

Read the complete text