Using LLMs for Security Advisory Investigations: How Far Are We?
1 st Bayu Fedra Abdullah, 4 th Raula Gaikovina Kula, 2 nd Yusuf Sulistyo Nugroho, 5 th Kazumasa Shimari, 3 rd Brittany Reid, 6 th Kenichi Matsumoto · 2025
Read Using LLMs for Security Advisory Investigations: How Far Are We? by 1 st Bayu Fedra Abdullah, 4 th Raula Gaikovina Kula, 2 nd Yusuf Sulistyo Nugroho, 5 th Kazumasa Shimari, 3 rd Brittany Reid, 6 th Kenichi Matsumoto (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-13161.txt.
Introduction
Bayu Abdullah and co-authors test how far ChatGPT can be trusted with software security advisories, in a 2025 paper. The team fed the model one hundred real and one hundred fake vulnerability IDs and judged what came back by hand. The model wrote plausible advisories for ninety six percent of real IDs, and for ninety seven percent of the fake ones.
Shown its own advisories later, the model sometimes invented a fresh fake ID instead of recovering the true one. Fluent output, the authors warn, is not proof of fact, and security teams should keep language models on a short leash.
Catalog record
| CREATOR | 1 st Bayu Fedra Abdullah, 4 th Raula Gaikovina Kula, 2 nd Yusuf Sulistyo Nugroho, 5 th Kazumasa Shimari, 3 rd Brittany Reid, 6 th Kenichi Matsumoto |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-13161
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Using LLMs for Security Advisory Investigations: How Far Are We? 1 st Bayu Fedra Abdullah, 4 th Raula Gaikovina Kula, 2 nd Yusuf Sulistyo Nugroho, 5 th Kazumasa Shimari, 3 rd Brittany Reid, 6 th Kenichi Matsumoto Abstract Large Language Models (LLMs) are increasingly used in software security, but their trustworthiness in generating accurate vulnerability advisories remains uncertain. This study investigates the ability of ChatGPT to (1) generate plausible security advisories from CVE-IDs, (2) differentiate real from fake CVE-IDs, and (3) extract CVE-IDs from advisory descriptions. Using a curated dataset of 100 real and 100 fake CVE-IDs, we manually analyzed the credibility and consistency of the model’s outputs. The results show that ChatGPT generated plausible security advisories for 96% of given input real CVE-IDs and 97% of given input fake CVE-IDs, demonstrating a limitation in differentiating between real and fake IDs. Furthermore, when these generated advisories were reintroduced to ChatGPT to identify their original CVE-ID, the model produced a fake CVE-ID in 6% of cases from real advisories. These findings highlight both the strengths and limitations of ChatGPT in cybersecurity applications. While the model demonstrates potential for automating advisory generation, its inability to reliably authenticate CVE-IDs or maintain consistency upon re-evaluation underscores the risks associated with its deployment in critical security tasks. Our study emphasizes the importance of using LLMs with caution in cybersecurity workflows and suggests the need for further improvements in their design to improve reliability and applicability in security advisory generation. I Introduction In recent years, software vulnerabilities have become the main focus for software developers, especially due to the increase in cybercrimes [ 1 ] . Developers may attempt to mitigate vulnerabilities [ 2 ] , but hackers continue exploiting weaknesses to attack users and steal sensitive data [ 3 ] . To aid the secure development of software, the Common Vulnerabilities and Exposures (CVE) system maintains a database 1 1 1 https://www.cve.org/ of publicly disclosed vulnerabilities in software packages, labeling each with a unique CVE-ID and providing advisory information. CVEs are a common way of disseminating security information within the software community, and thus recognizing and identifying them is important to the security of software. With the rise of generative AI, developers increasingly turn to Large Language Models (LLMs) for assistance in various domains, such as health [ 4 ] , education [ 5 ] , programming [ 6 ] , including software security [ 7 ] . Despite their growing popularity, concerns remain regarding the accuracy and reliability of outputs [ 8 ] . In cybersecurity, some practitioners use tools like ChatGPT to retrieve CVE-IDs or generate advisories, though LLMs’ trustworthiness in this context is uncertain. While LLMs show promise in vulnerability detection [ 9 ] and automated security bug repair [ 10 ] , their tendency to hallucinate information [ 11 ] raises risks. Relying on AI-generated advisories without verification may create false security assumptions or even delay critical mitigation efforts. Although LLMs are not designed as database query systems, our study reflects real-world misuse scenarios where users treat them as trusted security sources. This study investigates the capabilities of LLMs in generating accurate and reliable security advisories, identifying inconsistencies in vulnerability data, and detecting fake CVE-IDs. Prior research has examined LLMs in vulnerability detection and code security analysis. However, limited work has explored their reliability in security advisory generation. To address this gap, we formulate the following research questions: • RQ 1 : How trustworthy are the LLM-generated security advisories, given a known CVE-ID as input? • RQ 2 : Do LLMs have the capability to detect fake CVE-IDs? • RQ 3 : Can LLMs accurately and consistently produce CVE-ID from a provided advisory description? To answer these questions, we constructed a dataset comprising 100 real CVE-IDs and their corresponding security advisories from the GitHub Security Advisory (GHSA) database, 2 2 2 https://github.com/advisories capturing key metadata such as CVE-ID, title, severity, and affected products. To evaluate the LLM’s ability to detect fabricated vulnerabilities, we generated an additional 100 fake CVE-IDs using a Python script and verified their absence in public vulnerability databases. Both real and fake CVE-IDs were provided to ChatGPT, and the generated advisories were analyzed for accuracy, consistency, and reliability. The findings highlight both the potential and the risks of utilizing LLMs for security advisory generation. While LLMs show impressive generative capabilities, their tendency to produce misinformation raises concerns about their deployment in critical cybersecurity tasks. Our study highlights the importance of validating AI-generated security advisories before adoption and serves as a foundation for future research aimed at improving LLM reliability in cybersecurity applications. II Related Work II-A LLMs for Software Engineering Since the first release of the LLM, numerous studies have been conducted on its implementation in software engineering. Studies reveal that LLMs or generative AI tools like Google Bard (now called Gemini), ChatGPT, or CoPilot have proven that they can improve productivity in software engineering [ 12 , 13 ] . The use of LLMs has also benefited many software engineering tasks. For example, LLMs can accelerate the development cycles and reduce the time spent on repetitive coding tasks [ 14 ] , increasing productivity [ 15 ] , improving software quality and development efficiency [ 16 ] , and even the LLMs can help in identifying the software vulnerability-related subtle patterns [ 17 ] . Despite the advantages of the use of LLMs in software engineering, it has been unknown how far the LLMs can distinguish between real or fake vulnerabilities and their descriptions. II-B Identifying Software Vulnerabilities As one of the weaknesses in computer security, vulnerabilities are often used by an attacker to exploit the system to perform unauthorized actions [ 18 , 19 ] . To mitigate this, developers sometimes notify other programmers by citing a specific vulnerability identifier in code comments to indicate that the code may contain vulnerability [ 20 ] . Other strategies were also proposed in several researches, such as the development of automatic identification of vulnerable versions for CVE [ 21 ] , the implementation of TF-IDF and Doc2Vec for automatically tracing the related CAPEC-IDs from CVE-ID [ 22 ] , and a machine-learning based technique to assign pertinent CWE identifiers to new CVE entries [ 23 ] . These trigger software developers to make efforts and help them in addressing security issues consistently. In addition, some prior works were conducted by focusing on the vulnerability descriptions. For example, a study proposed a novel method to compose CVE descriptions by extracting some vulnerability aspects from ExploitDB [ 24 ] . Their results indicate that the method can achieve high accuracy in composing CVE descriptions. Another study augmented the vulnerability description by scrapping third-party references (hyperlinks) [ 25 ] . The findings have shown potential regarding summary fluency, completeness, correctness, and understanding. Due to the increasing of AI implementations in the software engineering field, the CVE description can also be potentially written with the help of AI, such as ChatGPT. Thus, in this paper, we study to what extent an LLM can generate the security advisory description. III Study Setting III-A Research Questions To guide our study, we formulated the following three research questions and their motivations. RQ 1 : How