AeroSec: a vertical domain large language model for air traffic management cybersecurity Authors: Ruochen Dong, Chengkai Piao, Buhong Wang, Yongjian Guan, Zhengyang Zhao, Xiangyu Xie Published: 2026 ABSTRACT With the development of smart civil aviation, the cybersecurity situation for air traffic management (ATM) continues to be critical. The increasingly exposed attack surface demands more advanced technologies and methods for protection. Large language models (LLMs) have been widely applied in cybersecurity, which has provided a novel paradigm for reconstructing ATM cybersecurity defense systems. However, existing vertical domain LLMs in ATM focus on providing passenger services and supporting daily operations, which do not concern cybersecurity. This neglect is a potential risk in the transition from security to safety. In this paper, a prompt-engineering-based method for generating question–answer pairs for ATM cybersecurity is proposed. Knowledge is extracted from multi-source heterogeneous primary sources and converted into uniformly formatted question–answer pairs, and an ATM cybersecurity fine-tuning dataset and benchmark are constructed. By fine-tuning the Deepseek-llm-7B-base model and DeepSeek-R1-Distill-Qwen-14B model using both instruction fine-tuning and reasoning fine-tuning methods, based on Low-Rank Adaptation (LoRA) and full-parameter fine-tuning technologies, a vertical domain LLM named “AeroSec” for ATM cybersecurity was constructed. By comparing the fine-tuned model with the base model, DeepSeek-V3 model, and DeepSeek-R1 model through the third-party model Qwen-Plus, the useful value of AeroSec in answering questions in specialized domains such as ATM network threat analysis and offensive-defensive techniques has been validated. Introduction As a major national critical infrastructure, the operational safety of air traffic management (ATM) systems directly impacts the safety of the nation and the lives and property of its people. With the increasing development of intelligent civil aviation in China and next-generation air transportation systems in Europe1 and the United States2, ATM systems have been transformed into cyber-physical systems3. Extensive external connectivity and cyber-physical interactions have also exposed additional attack surfaces. Cyberattacks such as Global Navigation Satellite System (GNSS) spoofing4 and Automatic Dependent Surveillance-Broadcast (ADS-B) signal injection5 increasingly impact operational safety6 through cybersecurity vulnerabilities. Safety is the lifeline of the civil aviation industry. The Civil Aviation Administration of China and other relevant departments and research institutions have issued multiple standards and specifications related to civil aviation cybersecurity. However, in the face of increasing cyber threats, more advanced technologies and methods are still expected to be adopted to ensure the cybersecurity of ATM systems. The development of large language model (LLM) technology has accelerated the reconstruction of ATM cybersecurity defense systems. By integrating the multimodal fusion and contextual reasoning capabilities of LLMs, these technologies can not only enhance threat modeling, converting natural language and cyber threat intelligence (CTI) into knowledge graphs7 and ATT&CK8, but also effectively handle complex challenges that are difficult for traditional security technologies to resolve. These include the integrated analysis of multi-source heterogeneous data, the early identification of potential threats, and the real-time defense against dynamic attacks. This provides a foundation for the transformation of ATM cybersecurity defense systems from passive response to intelligent defense. However, ATM cybersecurity is characterized by a high level of specialization. Existing general-purpose LLMs lack sufficient threat intelligence, offensive and defensive techniques, and complex domain knowledge, making it difficult to accurately identify and extract entity relationships and risk information. This will lead to serious limitations in practical applications. Furthermore, the models’ generalization capabilities are insufficient in dynamic attack scenarios, which will lead to a surge in false positive rates when facing novel attack patterns, making it difficult to achieve the effectiveness of traditional cybersecurity defenses. To deal with this issue, in this paper, a question–answer pair generation method for ATM cybersecurity is proposed. By collecting a large amount of multi-source heterogeneous professional corpora and threat intelligence, an ATM cybersecurity fine-tuning dataset is constructed. The Deepseek-llm-7B-base model and DeepSeek-R1-Distill-Qwen-14B model are used as base models for fine-tuning, to construct the lightweight ATM cybersecurity vertical domain LLM suitable for local deployment. Related work LLMs have found extensive application in cybersecurity9. SecGPT, the first open-source cybersecurity LLM, was released in 2023. It serves multiple critical security tasks including vulnerability analysis, anomaly detection, attack-defense reasoning, and command analysis. China Next Generation Cybersecurity Alliance constructed the open-source cybersecurity LLM NgSecGPT by fine-tuning and aligning DeepSeek-70B with massive cybersecurity-related datasets. Tencent Zhuque Lab and Tencent Security Keen Lab, as well as other teams, have proposed a large-scale, multidimensional benchmark dataset named SecBench. This dataset includes 44,823 multiple-choice questions and 3,087 question–answer pairs, designed to evaluate the performance of LLMs in the cybersecurity domain10. However, the aforementioned research works have been applied primarily to traditional cybersecurity domains, and there remains a lack of relevant researches on cybersecurity LLMs for ATM systems. In the civil aviation industry, LLMs are primarily applied in three key fields. First, through localized deployment of LLMs, building vertical domain models, and developing AI assistants based on these models, to support daily operations and improve passenger services. For example, “Huiyan” LLM Q&A and knowledge assistant by China Eastern Airlines, “Rongyi Easy” intelligent assistant by Fujian Air Traffic Control Branch, and the “Qianrang” LLM. In addition, LLMs are employed to process flight data for critical operational systems such as air traffic flow management11, flight trajectory reconstruction12, and flight trajectory prediction13. The third application involves fine-tuning vertical domain LLMs for aviation14–16, though the datasets used are predominantly accident investigation reports or communications, navigation, and surveillance (CNS) data, prioritizing operational safety over cybersecurity. Existing general-purpose and vertical domain LLMs can support cybersecurity operations, but there is still a lack of knowledge specific to ATM cybersecurity. To enhance the application of LLMs in the civil aviation industry and better support ATM cybersecurity, following contributions are proposed in this paper. (1) A method for generating ATM cybersecurity question–answer pairs based on context-aware prompt engineering is proposed. Existing ATM cybersecurity knowledge primarily originates from technical documentation, and knowledge extraction using LLMs often encounters restrictions due to sensitive terms. Through prompt engineering, a method for generating ATM cybersecurity question–answer pairs based on the DeepSeek model is proposed. This method constructs prompts using modular task descriptions, which enables the model to better understand task requirements. By defining the task context and precisely specifying sensitive term exclusion demands, the model can perform a more in-depth analysis of source documents. This method extracts relevant knowledge from technical documents such as academic papers and industry reports as comprehensively as possible, converting them into ATM cybersecurity question–answer pairs in a batch manner. (2) A fine-tuning dataset for ATM cybersecurity is constructed from multi-source heterogeneous corpora. In the aviation industry, the corpora used for fine-tuning LLMs are predominantly technical documents such as accident investigation reports or web-based information. In this paper, a large amount of multi-source heterogeneous professional corpora and threat intelligence for ATM cybersecurity are collected. These corpora not only include journal and conference papers targeting academic frontiers and industry reports in the civil aviation industry focusing on development strategies and standards, but also contain highly specialized, knowledge-integrated corpora such as Common Vulnerabilities and Exposures (CVE) for real cybersecurity vulnerabilities and an ATM cybersecurity knowledge graph for ATM entities and relationships. By converting the original multi-format corpora into unified question–answer pairs using the ATM cybersecurity question-pair generation method and the DeepSeek-R1 model, a fine-tuned dataset for ATM cybersecurity is constructed. This dataset includes instruction fine-tuning data and reasoning fine-tuning data. (3) The first vertical domain LLM for ATM cybersecurity has been fine-tuned. It uses the MITRE ATT&CK dataset from LEAD-Cyber17 as foundational cybersecurity knowledge, with an ATM cybersecurity fine-tuning dataset enhancing industry applicability. Based on the Deepseek-llm-7B-base model and DeepSeek-R1-Distill-Qwen-14B model, and combining instruction fine-tuning and reasoning fine-tuning methods, the first vertical domain LLM for ATM cybersecurity named “AeroSec” is proposed. In comparison to existing general-purpose LLMs, it has more abundant domain-specific knowledge in ATM cybersecurity and can be effectively applied to ATM network penetration testing and security operations. (4) An ATM cybersecurity benchmark dataset named “AeroSecBench” is proposed. Based on the architecture of ATM systems and the critical business systems, the fine-tuned dataset is divided into multiple subjects and proportionally sampled to construct the ATM cybersecurity benchmark dataset. This dataset is used to evaluate the cognitive capabilities of LLMs for ATM cybersecurity. Based on this benchmark dataset, Qwen-Plus comprehensively scores generated answers on indicators such as accuracy and conciseness. Subjective evaluations of fine-tuned LLMs are evaluated through three metrics: significance testing, model scoring, and WinRate, which assesses the specialized domain performance of fine-tuned LLMs. Methods In this section, the full-cycle construction methodology of AeroSec is detailed, as shown in Fig. 1. It includes the corpus collection and question–answer pair generation, data processing and dataset construction, model fine-tuning, and model evaluation.Fig. 1Process for the construction of AeroSec. Process for the construction of AeroSec. Method for generating question–answer pairs for ATM cybersecurity The existing knowledge about ATM cybersecurity primarily originates from technical documents such as academic papers and industry reports. The issue of how to convert relevant knowledge from these documents into a regularized question–answer pair dataset needs to be a critical research focus. Varun Badrinath Krishna utilized LLaMA 3 to assist in generating the cybersecurity question–answer dataset named AttackQA18. Based on the method proposed in this research, prompts were constructed as shown in Table 1 to process ATM cybersecurity related documents and generate question–answer pairs. However, the results were not satisfactory, with a limited number of question–answer pairs generated. Although the selected documents are highly relevant to ATM cybersecurity and contained substantial knowledge elements, the generated question–answer pairs are fixed at 10 in most cases.Table 1Prompt for generating question–answer pairs based on AttackQA. Table 1Prompt for generating question–answer pairs based on AttackQA. Prompt for generating question–answer pairs based on AttackQA. In this paper, to improve question–answer pair generation quality, extensive comparative experiments are conducted based on prompt engineering. Given the interpretability challenges of LLM content generation, the following issues are identified. Three of them primarily affect the quantity of generated question–answer pairs, while one mainly impacts their quality.Security interception mechanism. Since the document contains numerous sensitive terms with aggressive connotations such as “intrusion” and “attack,” the LLM automatically activates a security interception mechanism upon detecting these sensitive terms, forcibly limiting the output to 10 items.Specific term activation restrictions. Because the ATM system is a critical national infrastructure with high industry sensitivity and confidentiality, the output is also restricted when the topic includes aviation-related keywords like “air traffic management” or “air traffic control.”Model-specific limitations. Models like DeepSeek have token limits for both input and output of a single request. When documents are excessively lengthy or the number of generated question–answer pairs is too large, a truncation process is required. This leads to context loss, affecting both the quantity and relevance of generated question–answer pairs.Prompt style mismatch with the model. The prompts in AttackQA are customized for the LLaMA 3 model. Their prompt style and identifiers are not suitable for the DeepSeek model. Security interception mechanism. Since the document contains numerous sensitive terms with aggressive connotations such as “intrusion” and “attack,” the LLM automatically activates a security interception mechanism upon detecting these sensitive terms, forcibly limiting the output to 10 items. Security interception mechanism. Since the document contains numerous sensitive terms with aggressive connotations such as “intrusion” and “attack,” the LLM automatically activates a security interception mechanism upon detecting these sensitive terms, forcibly limiting the output to 10 items. Specific term activation restrictions. Because the ATM system is a critical national infrastructure with high industry sensitivity and confidentiality, the output is also restricted when the topic includes aviation-related keywords like “air traffic management” or “air traffic control.” Specific term activation restrictions. Because the ATM system is a critical national infrastructure with high industry sensitivity and confidentiality, the output is also restricted when the topic includes aviation-related keywords like “air traffic management” or “air traffic control.” Model-specific limitations. Models like DeepSeek have token limits for both input and output of a single request. When documents are excessively lengthy or the number of generated question–answer pairs is too large, a truncation process is required. This leads to context loss, affecting both the quantity and relevance of generated question–answer pairs. Model-specific limitations. Models like DeepSeek have token limits for both input and output of a single request. When documents are excessively lengthy or the number of generated question–answer pairs is too large, a truncation process is required. This leads to context loss, affecting both the quantity and relevance of generated question–answer pairs. Prompt style mismatch with the model. The prompts in AttackQA are customized for the LLaMA 3 model. Their prompt style and identifiers are not suitable for the DeepSeek model. Prompt style mismatch with the model. The prompts in AttackQA are customized for the LLaMA 3 model. Their prompt style and identifiers are not suitable for the DeepSeek model. In this paper, to address the aforementioned issues, the prompts are optimized as shown in Table 2. Through modular definitions, the prompts become more easily understood by DeepSeek model.The background is explicitly defined in the system prompts using role and tasks.The purpose of the question–answer pairs is specified as academic research by using declarations, and sensitive terms are bypassed.The processing workflow for generated question–answer pairs is clarified. By modularizing document processing, knowledge elements are extracted at the sentence level to generate question–answer pairs, preventing knowledge element omissions. Cross-paragraph associations and cross-document knowledge integration are employed to ensure comprehensive knowledge coverage.Eliminate potential hallucination issues during model generation using explicitly defined constraints. Output question–answer pairs in a standardized JSON format to facilitate subsequent model fine-tuning.Table 2Prompt for generating question–answer pairs for ATM cybersecurity. The background is explicitly defined in the system prompts using role and tasks. The background is explicitly defined in the system prompts using role and tasks. The purpose of the question–answer pairs is specified as academic research by using declarations, and sensitive terms are bypassed. The purpose of the question–answer pairs is specified as academic research by using declarations, and sensitive terms are bypassed. The processing workflow for generated question–answer pairs is clarified. By modularizing document processing, knowledge elements are extracted at the sentence level to generate question–answer pairs, preventing knowledge element omissions. Cross-paragraph associations and cross-document knowledge integration are employed to ensure comprehensive knowledge coverage. The processing workflow for generated question–answer pairs is clarified. By modularizing document processing, knowledge elements are extracted at the sentence level to generate question–answer pairs, preventing knowledge element omissions. Cross-paragraph associations and cross-document knowledge integration are employed to ensure comprehensive knowledge coverage. Eliminate potential hallucination issues during model generation using explicitly defined constraints. Output question–answer pairs in a standardized JSON format to facilitate subsequent model fine-tuning. Eliminate potential hallucination issues during model generation using explicitly defined constraints. Output question–answer pairs in a standardized JSON format to facilitate subsequent model fine-tuning. Table 2Prompt for generating question–answer pairs for ATM cybersecurity. Prompt for generating question–answer pairs for ATM cybersecurity. In order to validate the optimization results of the prompt, 200 identical technical documents are selected, with the target number of generated question–answer pairs set to 50. The question–answer pairs are generated with both the pre-optimized and optimized prompts. The results are shown in Table 3. The optimized prompt generates approximately four times as many question–answer pairs as the pre-optimized prompt.Table 3Comparison of effectiveness between different prompt for question–answer generation.Prompt based on AttackQAPrompt for ATM cybersecurityNumber of technical documents200200Number of question–answer pairs24179431Average number of question–answer pairs1247 Table 3Comparison of effectiveness between different prompt for question–answer generation.Prompt based on AttackQAPrompt for ATM cybersecurityNumber of technical documents200200Number of question–answer pairs24179431Average number of question–answer pairs1247 Comparison of effectiveness between different prompt for question–answer generation. It is crucial to emphasize that the method proposed in this paper for generating question–answer pairs has a fundamental distinction from “jailbreak” attacks targeting LLMs. While jailbreak attacks exploit prompts to bypass the model’s constraints and generate harmful content, the approach presented in this paper enables the model to better comprehend the provided original documents and extract relevant knowledge to transform it into standardized question–answer pairs. All content originates from the provided documents; neither is the model instructed to generate it independently. Dataset construction (1) Corpora collection In this paper, a large amount of multi-source heterogeneous professional corpora and threat intelligence in the field of ATM cybersecurity are collected to construct a fine-tuning dataset for ATM cybersecurity. Through keyword retrieval and manual screening, the dataset includes the following five categories of data. 1) Journal and conference papers. In this paper, for the domestic and international databases of academic papers, including CNKI, IEEE Xplore, ScienceDirect, ACM Digital Library, AIAA Aerospace Research Central, etc., the focus is on collecting papers related to attack and defense techniques, threat models, system architecture, and protection mechanisms for ATM networks. Keywords include air traffic management security, civil aviation security, ADS-B security, GNSS security, ATM network threats, ATM blockchain, ATM data link security, ATM network attacks, etc., with a focus on papers published between 2021 and 2025. After manual screening and processing, a total of 352 papers were collected. 2) Industry reports. For domestic and international civil aviation organizations and government agencies, including the Civil Aviation Administration of China (CAAC), the Federal Aviation Administration (FAA), the Department for Transport (UK), and the Ministry of Land, Infrastructure, Transport and Tourism (MLIT, Japan), collecting industry reports on the security development strategies, assurance plans, standards and specifications for ATM networks. Keywords primarily include development plans for next-generation ATM systems across various countries and organizations, such as China’s Smart Civil Aviation Development, the U.S. Next Generation Air Transportation System (NextGen), the European Single Sky ATM Research (SESAR), and Japan’s Collaborative Actions for Renovation of Air Traffic Systems (CARATS). For the collection of reports, there was no time range limitation, except for those explicitly marked as obsolete. After manual screening and processing, a total of 89 reports were collected. 3) CVE. A collection of 40 real-world ATM cybersecurity vulnerabilities from databases like CVE, involving satellites, aircraft, airports, and related systems. 4) ATM cybersecurity knowledge graph. A set of ATM cybersecurity knowledge graphs19 is collected, as shown in Fig. 2, to represent assets and their relationships within ATM systems. It includes 212 entities and 921 relationships between entities.Fig. 2ATM cybersecurity knowledge graph. ATM cybersecurity knowledge graph. 5) MITRE ATT&CK. For the enterprise, mobile, and industrial control system (ICS) matrices within MITRE ATT&CK, a total of 2,262 technical and tactical items are collected. These items supplement the aforementioned four categories of ATM cybersecurity data, enhancing the knowledge of the AeroSec in the general cybersecurity domain. (2) Data Processing and Dataset Construction The original data can be categorized into four categories: 1) CVE lists, 2) ATT&CK matrices, 3) knowledge graphs, 4) papers and industry reports primarily in PDF format. For the non-document formats, the original data requires preprocessing to convert it into files readable by LLM before proceeding with knowledge extraction and question–answer pair generation. The data processing and dataset construction workflow is shown in Fig. 3.Fig. 3The workflow of data processing and dataset construction. The workflow of data processing and dataset construction. 1) Question–answer pair generation. At first, the multi-source heterogeneous original corpora are converted into uniformly formatted question–answer pairs using the ATM cybersecurity question–answer pair generation method proposed in Section "Method for generating question–answer pairs for ATM cybersecurity". For the CVE list, download the CVE JSON 5.0 format data, extract vulnerability data related to ATM cybersecurity based on the queried CVE_ID, and save it in JSON format. For the ATT&CK matrix, download the Structured Threat Information Expression (STIX) format data and convert it into a Pandas table. Descriptions which contain tactics, techniques, software, organizations, activities, and mitigation methods are extracted into content documents. Relationship tables are then used to establish mapping relationships, representing the tactics, techniques, and procedures employed by different attack organizations during Advanced Persistent Threat (APT) activities. For the knowledge graph, since it is stored in a Neo4j database, the structured data must be obtained using Cypher, which is a declarative query language for Neo4j graph databases. The command “MATCH (n) RETURN n” retrieves all entities, while “MATCH (s)-[r]->(t) RETURN s, r, t” retrieves all relationships. On this basis, the knowledge graph context shown in Table 4 is constructed, enabling the further extraction and question–answer pair generation by the LLM. The pre-processed original corpora are processed by using the DeepSeek-R1 model to generate ATM cybersecurity question–answer pairs. The question–answer pairs are divided into two categories: instruction fine-tuning and reasoning fine-tuning. Instruction fine-tuning pairs contain only generated ATM cybersecurity related questions and their corresponding answers, while reasoning fine-tuning pairs include detailed reasoning processes and references for each answer. The reasoning capability of the model is enhanced through the chain of thought for individual questions and cross-references between multiple questions.Table 4Knowledge graph context. Table 4Knowledge graph context. Knowledge graph context. 2) Data Processing. The quality of question–answer pairs generated by LLMs varies significantly, which requires further data processing to enhance data quality. As shown in Fig. 3, the primary focus is on removing the following three categories of data: low-quality question–answer pairs, low-relevance question–answer pairs, and high-similarity question–answer pairs. Low-quality question–answer pairs are primarily characterized by highly similar questions and answers, with the answer often embedded within the question. This category can be filtered using regular expression matching to identify data where questions and answers are identical or highly similar. Low-relevance question–answer pairs primarily represent low association with ATM cybersecurity. The reason for their existence is that the thought process or references contain some content related to ATM cybersecurity, but the question and answer are unrelated to it. This category can be filtered by using LLMs for secondary judgment, removing data unrelated or only loosely connected to ATM cybersecurity. High similarity question–answer pairs primarily exist across multiple question–answer pairs. Because the generated question–answer pairs originate from different corpora, the generation process may produce multiple identical questions or similar answers for a single knowledge element. This category can be filtered using regular expression matching. For instruction fine-tuning question–answer pairs, when the questions are identical and the answers are highly similar, they are marked as high-similarity question–answer pairs. For reasoning fine-tuning question–answer pairs, further comparison of the thought process and references is required. 3) Dataset Construction. In order to construct the ATM cybersecurity fine-tuning dataset and benchmark, all question–answer pairs are classified into the following six subjects through judgment by LLMs. a) Communication. It contains questions related to air traffic communication systems, such as Aeronautical Telecommunication Network (ATN), L-band Digital Aeronautical Communications System (LDACS), Controller Pilot Data Link Communications (CPDLC), VHF, HF, etc. b) Navigation. It contains questions related to air traffic navigation systems, such as Global Navigation Satellite System (GNSS), Global Positioning System (GPS), Beidou, Galileo, Ground-Based Augmentation Systems (GBAS), Satellite-Based Augmentation System (SBAS), etc. c) Surveillance. It contains questions related to air traffic surveillance systems, such as Automatic Dependent Surveillance-Broadcast (ADS-B), radar, Mulilateration (MLAT), Primary Surveillance Radar (PSR), Secondary Surveillance Radar (SSR), Wide Area Multilateration (WAM), etc. d) Aircraft. It contains questions related to aircraft and airborne systems, such as Aircraft Condition Monitoring System (ACMS), Flight Management System (FMS), Electronic Flight Bag (EFB), airborne networks, avionics, Aircraft Control Domain (ACD), Airline Information Service Domain (AISD), etc. e) Airport. It contains questions related to airports and the System Wide Information Management (SWIM), such as Airport Operation DataBase (AODB), Collaborative Decision Making (CDM), ATS Message Handling System (AMHS), Notice to Airmen (NOTAM), airport networks, etc. f) ATM. It contains questions related to ATM cybersecurity and relevant national and organizational policies and project plans. Questions unrelated to the aforementioned six subjects are categorized into “other.” After classifying all questions by subject, 30% of questions are extracted from each of the six subjects to construct the AeroSecBench. This dataset comprehensively covers relevant content across all ATM domains and is suitable for evaluating the cognitive capabilities of LLMs regarding ATM cybersecurity. The remaining 70% of questions, as well as those categorized into “other,” are used to form the ATM cybersecurity fine-tuning dataset for model fine-tuning. (3) Dataset Statistics The unprocessed dataset initially includes 45,673 question–answer pairs, in which there are 23,543 ATM cybersecurity question–answer pairs and 22,130 ATT&CK question–answer pairs serving as foundational cybersecurity knowledge. After the different stages of data processing, the number of question–answer pairs is shown in Tables 5 and 6. ATT&CK question–answer pairs, as foundational cybersecurity knowledge, only processed high-similarity question–answer pairs. Because high-similarity question–answer pairs require comprehensive comparison of all question–answer pairs, only the total number of question–answer pairs is counted.Table 5Statistics on the number of question–answer pairs for instruction fine-tuning data.CategoryInitial number of Q&A pairsRemove low-quality Q&A pairsRemove low-relevance Q&A pairsRemove high-similarity Q&A pairsChinese paper37153714232839,202English paper13,42613,42310,233CAAC343934022834Civil aviation organizations and agencies185218481613CVE195195194Knowledge graph916915911ATT&CK ICS771077107710ATT&CK Mobile811181118111ATT&CK Enterprise630963096309Total45,67345,62740,24339,202Table 6Statistics on the number of question–answer pairs for reasoning fine-tuning data.CategoryInitial number of Q&A pairsRemove low-quality Q&A pairsRemove low-relevance Q&A pairsRemove high-similarity Q&A pairsChinese paper37153714232839,866English paper13,42613,42310,234CAAC343934022835Civil aviation organizations and agencies185218481613CVE195195195Knowledge graph916915911ATT&CK ICS771077107710ATT&CK Mobile811181118111ATT&CK Enterprise630963096309Total45,67345,62740,24639,866 Table 5Statistics on the number of question–answer pairs for instruction fine-tuning data.CategoryInitial number of Q&A pairsRemove low-quality Q&A pairsRemove low-relevance Q&A pairsRemove high-similarity Q&A pairsChinese paper37153714232839,202English paper13,42613,42310,233CAAC343934022834Civil aviation organizations and agencies185218481613CVE195195194Knowledge graph916915911ATT&CK ICS771077107710ATT&CK Mobile811181118111ATT&CK Enterprise630963096309Total45,67345,62740,24339,202 Statistics on the number of question–answer pairs for instruction fine-tuning data. Table 6Statistics on the number of question–answer pairs for reasoning fine-tuning data.CategoryInitial number of Q&A pairsRemove low-quality Q&A pairsRemove low-relevance Q&A pairsRemove high-similarity Q&A pairsChinese paper37153714232839,866English paper13,42613,42310,234CAAC343934022835Civil aviation organizations and agencies185218481613CVE195195195Knowledge graph916915911ATT&CK ICS771077107710ATT&CK Mobile811181118111ATT&CK Enterprise630963096309Total45,67345,62740,24639,866 Statistics on the number of question–answer pairs for reasoning fine-tuning data. Based on instruction fine-tuning question–answer pairs, questions are categorized by subject and extracted. The number of original question–answer pairs, extracted question–answer pairs, and their respective proportions for each subject are shown in Table 7. Token statistics for the instruction fine-tuning dataset, reasoning fine-tuning dataset, and the benchmark dataset are presented in Table 8.Table 7Number of question–answer pairs and extraction quantity for each subject.SubjectNumber of initial Q&A pairsNumber of extracted Q&A pairsProportionCommunication210263011.5%Navigation252875813.8%Surveillance7342204%Aircraft249074713.6%Airport10343105.6%ATM9432282951.5%Total18,3205494100%Table 8Statistics on Tokens.CategoryNumber of Q&A pairsTokensAverage tokenInstruction fine-tuning dataset33,7081,701,67350Reasoning fine-tuning dataset34,1505,350,331156AeroSecBench5494271,38649 Table 7Number of question–answer pairs and extraction quantity for each subject.SubjectNumber of initial Q&A pairsNumber of extracted Q&A pairsProportionCommunication210263011.5%Navigation252875813.8%Surveillance7342204%Aircraft249074713.6%Airport10343105.6%ATM9432282951.5%Total18,3205494100% Number of question–answer pairs and extraction quantity for each subject. Table 8Statistics on Tokens.CategoryNumber of Q&A pairsTokensAverage tokenInstruction fine-tuning dataset33,7081,701,67350Reasoning fine-tuning dataset34,1505,350,331156AeroSecBench5494271,38649 Statistics on Tokens. Model fine-tuning In this paper, Low-Rank Adaptation (LoRA) and Full Fine-tuning (FFT) techniques are used for fine-tuning the Deepseek-llm-7B-base model and the DeepSeek-R1-Distill-Qwen-14B model. FFT updates and optimizes all layers and parameters of a pre-trained model to achieve superior performance. However, it demands substantial memory and computational resources to store and process training data, resulting in high computational costs20. LoRA is a parameter-efficient fine-tuning method, as shown in Fig. 4. This approach freezes the pre-trained model weights \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${W}_{0}\in {\mathbb{R}}^{d\times k}$$\end{document} and injects a trainable rank-decomposition matrix into each layer of the Transformer architecture. By constraining updates with low-rank decomposition \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${W}_{0}+\Delta W={W}_{0}+BA$$\end{document}, it reduces the number of trainable parameters for downstream tasks21, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B\in {\mathbb{R}}^{d\times r}$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A\in {\mathbb{R}}^{r\times k}$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r\ll min\left(d,k\right)$$\end{document}.Fig. 4LoRA fine-tuning. LoRA fine-tuning. Model evaluation In this paper, the generated answers are scored using the Qwen-Plus model, which comprehensively evaluates multiple dimensions including accuracy and conciseness. The fine-tuning models are evaluated by comparing scores across models and the WinRate metric. Because there is currently no open-source LLM for ATM cybersecurity, to fully evaluate the performance of AeroSec, in addition to the Deepseek-llm-7B-base model and DeepSeek-R1-Distill-Qwen-14B model before fine-tuning, DeepSeek-V3 and DeepSeek-R1 are also included in the comparison scope. (1) Instruction fine-tuning. The outputs of DeepSeek-V3 and the instruction fine-tuning model are compared, with evaluation results using the Qwen-Plus model shown in Figs. 5, 6, 7 and 8. The distributions of scores across models are shown in Figs. 5 and 6. The output scores of the fine-tuning models are significantly higher than those of the base model and DeepSeek-V3. Furthermore, the LoRA fine-tuning approach is slightly more effective than FFT. The reliability of the fine-tuning effectiveness is validated through significance testing in Fig. 7. The “***” annotations above each figure indicate p-values less than 0.001. The distributions of WinRates across models are shown in Fig. 8. Compared to the DeepSeek-V3 model, the Deepseek-llm-7B-base model and DeepSeek-R1-Distill-Qwen-14B model, both fine-tuned using the LoRA method, achieved the WinRates of 83% and 80.7% respectively. The Deepseek-llm-7B-base model and DeepSeek-R1-Distill-Qwen-14B model fine-tuned using the FFT method achieved the WinRates of 78.7% and 76.2% respectively.Fig. 5The distribution of model scores—Box plot.Fig. 6The distribution of model scores—Violin plot.Fig. 7Significance test.Fig. 8The distribution of the WinRates across different models. The distribution of model scores—Box plot. The distribution of model scores—Violin plot. Significance test. The distribution of the WinRates across different models. (2) Reasoning fine-tuning. The outputs of DeepSeek-R1 and the reasoning fine-tuning model are compared, with evaluation results using the Qwen-Plus model shown in Figs. 9, 10, 11 and 12. With the same results as the instruction fine-tuning comparison, the model outputs from the reasoning fine-tuning achieved significantly higher scores than both the base model and DeepSeek-R1. The LoRA fine-tuning method demonstrated slightly better performance than FFT, and statistical significance tests validated the reliability of the fine-tuning effects. The distributions of WinRates across models are shown in Fig. 12. Compared to the DeepSeek-R1 model, the Deepseek-llm-7B-base model and DeepSeek-R1-Distill-Qwen-14B model, both fine-tuned using the LoRA method, achieved the WinRates of 82.8% and 80.5% respectively. The Deepseek-llm-7B-base model and DeepSeek-R1-Distill-Qwen-14B model fine-tuned using the FFT method achieved the WinRates of 76% and 76.6% respectively.Fig. 9The distribution of model scores—Box plot.Fig. 10The distribution of model scores—Violin plot.Fig. 11Significance test.Fig. 12The distribution of the WinRates across different models. The distribution of model scores—Box plot. The distribution of model scores—Violin plot. Significance test. The distribution of the WinRates across different models. Discussion The proposed vertical domain LLM for ATM cybersecurity achieves better applicability to specialized domains by fine-tuning models with relatively small parameter counts, enabling lightweight model deployment for relevant institutions. The following three directions for future research can further enhance model performance and facilitate subsequent application development. (1) Collection of original corpora and enhancement of the quantity and quality of question–answer pairs. A significant portion of the original corpora in this paper consists of open-source academic papers and industry reports. Although the corpora are filtered by keywords during collection to ensure high relevance to ATM cybersecurity, their quality remains inconsistent. The quality of generated question–answer pairs can be further improved by collecting knowledge-intensive, high-quality original corpora or internal industry documents. Furthermore, the proposed method for generating question–answer pairs for ATM cybersecurity is more suitable for the batch processing of technical documents. When dealing with certain high-quality or lengthy corpora, it may face challenges in insufficient knowledge extraction. Therefore, multiple-turn conversation or context engineering22 can be employed for these corpora to fully extract subject-related knowledge. (2) Retrieval-Augmented Generation (RAG). The fine-tuned vertical domain LLMs outperforms current general-purpose LLMs in dealing with specialized domain problems. If new corpus data becomes available, RAG technology can be leveraged to enhance model performance. Additionally, the ATM cybersecurity knowledge graph can be further enriched for threat modeling (such as GraphRAG23). (3) Intelligent question-answering and penetration testing. Two approaches, instruction fine-tuning and reasoning fine-tuning, are employed to fine-tune the vertical domain LLM for ATM cybersecurity. Instruction fine-tuning is suitable for developing systems like intelligent question-answering. The dataset used for reasoning fine-tuning incorporates both the thought process behind questions and original references. By interconnecting multiple questions and causal reasoning, it enhances the reasoning capabilities of the model. Consequently, an intelligent multi-agent penetration testing system can be developed with the reasoning model to further enhance ATM cybersecurity capabilities through proactive defense. However, some limitations remain in this paper. Due to the high level of specialization within the civil aviation cybersecurity field, the amount of open-source data and documents available for collection is relatively limited. Therefore, the data used in this paper validates the effectiveness of the methodology. Relevant departments possessing large amounts of original data can further enhance the model based on this approach. Conclusions In this paper, a method for generating question–answer pairs for ATM cybersecurity is proposed. By collecting and processing multi-source heterogeneous ATM cybersecurity corpora, including academic papers, industry reports, CVE lists, ATT&CK matrices, and knowledge graphs, an ATM cybersecurity fine-tuning dataset and benchmark dataset are constructed. These datasets can be used for model fine-tuning and evaluation of vertical domain LLMs. Through instruction fine-tuning and reasoning fine-tuning modes, based on open-source LLMs and two fine-tuning techniques, LoRA and FFT, the vertical domain LLM “AeroSec" for ATM cybersecurity is proposed. It improves the application capability of LLM in the field of ATM cybersecurity and provides an innovative solution for the local deployment of LLMs used for security operations and penetration testing in the ATM domain. In order to avoid evaluator style bias, the DeepSeek series models were selected for fine-tuning and comparison, while Qwen-Plus was used for evaluation of model fine-tuning effectiveness. Evaluations were conducted using the AeroSecBench benchmark dataset. Comparisons with pre-fine-tuned models demonstrate that the fine-tuning data and process effectively enhance model performance. Comparing the fine-tuned models (7b and 14b parameters) with Deepseek v3 and Deepseek r1 (671b parameters) demonstrates that fine-tuning can indeed enable lightweight models to acquire domain-specific knowledge, suitable for local deployment on lightweight or edge devices. The evaluation results validate the practical value of AeroSec in answering specialized questions related to air traffic management network threat analysis and offense-defense technologies.