VulRTex : A Reasoning-Guided Approach to Identify Vulnerabilities from Rich-Text Issue Report

Ziyou Jiang, Mingyang Li, Guowei Yang, Lin Shi, Qing Wang ∗ · 2025

Read VulRTex : A Reasoning-Guided Approach to Identify Vulnerabilities from Rich-Text Issue Report by Ziyou Jiang, Mingyang Li, Guowei Yang, Lin Shi, Qing Wang ∗ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-03875.txt.

Introduction

Ziyou Jiang and four co-authors' 2025 research paper takes up VulRTex, a tool for finding security vulnerabilities hiding inside open-source issue reports. Developers often disclose flaws through issue trackers, where genuine vulnerability reports sit buried among routine bugs. Human triage is slow, and attackers can exploit the gap before a dangerous report gets noticed.

Earlier automated detectors read only plain text, ignoring rich content such as screenshots and formatted code. VulRTex reasons instead. First, a large language model studies historical reports to build a database of vulnerability reasoning. For a new report, similar past cases become reasoning guidance for judging its rich text and predicting a weakness category. The evaluation covers nearly a million issue reports. Under class imbalance, VulRTex tops the best baseline on detection quality and weakness prediction, at half the running time of rival reasoning approaches. Faster recognition shrinks the window between disclosure and defence.

Catalog record

CREATORZiyou Jiang, Mingyang Li, Guowei Yang, Lin Shi, Qing Wang ∗
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-03875

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VulRTex : A Reasoning-Guided Approach to Identify Vulnerabilities from Rich-Text Issue Report Ziyou Jiang, Mingyang Li, Guowei Yang, Lin Shi, Qing Wang ∗ Abstract . Software vulnerabilities exist in open-source software (OSS), and the developers who discover these vulnerabilities may submit issue reports (IRs) to describe their details. Security practitioners need to spend a lot of time manually identifying vulnerability-related IRs from the community, and the time gap may be exploited by attackers to harm the system. Previously, researchers have proposed automatic approaches to facilitate identifying these vulnerability-related IRs, but these works focus on textual descriptions but lack the comprehensive analysis of IR’s rich-text information. In this paper, we propose VulRTex , a reasoning-guided approach to identify vulnerability-related IRs with their rich-text information. In particular, VulRTex first utilizes the reasoning ability of the Large Language Model (LLM) to prepare the Vulnerability Reasoning Database with historical IRs. Then, it retrieves the relevant cases from the prepared reasoning database to generate reasoning guidance, which guides LLM to identify vulnerabilities by reasoning analysis on target IRs’ rich-text information. To evaluate the performance of VulRTex , we conduct experiments on 973,572 IRs, and the results show that VulRTex achieves the highest performance in identifying the vulnerability-related IRs and predicting CWE-IDs when the dataset is imbalanced, outperforming the best baseline with +11.0% F1, +20.2% AUPRC, and +10.5% Macro-F1, and 2x lower time cost than baseline reasoning approaches. Furthermore, VulRTex has been applied to identify 30 emerging vulnerabilities across 10 representative OSS projects in 2024’s GitHub IRs, and 11 of them are successfully assigned CVE-IDs, which illustrates VulRTex ’s practicability. 1. Introduction Software vulnerabilities widely exist in open-source software (OSS) projects. When developers discover the vulnerabilities, they may submit the IRs to describe the details of these vulnerabilities. To identify these vulnerability-related IRs, security practitioners usually spend a lot of time manually analyzing their contents ( Householder et al. 2017 ) . They often track these IRs with issue-tracking systems ( bug 2023 ; Atlassian 2023 ) , then classify these vulnerability-related IRs with Common Weakness Enumeration (CWE) ( CWE 2023 ) , which categorizes the types of vulnerabilities with their causes, behaviors, and consequences. Some of these identified vulnerabilities will be disclosed in the Common Vulnerabilities and Exposure (CVE) ( CVE 2023 ) , a security database that provides a standardized method to catalog publicly disclosed vulnerabilities, and to alert downstream users in the supply chain to the security risks in the project ( iso 2018 ) . However, the manual identification of vulnerability is tedious and time-consuming. The time interval between the creation of vulnerability IRs and the vulnerability disclosure can be exploited by attackers (e.g., zero-day attacks ( Bilge and Dumitras 2012 ) ) to harm the system, resulting in millions of dollars of losses in today’s businesses ( Pan and Tomlinson 2016 ) . To alleviate this risk at an early stage, researchers have proposed automatic approaches to facilitate the identification of vulnerability-related IRs with their textual description ( Gegick et al. 2010 ; Oyetoyan and Morrison 2021 ; Pan et al. 2022 ) . However, we find that 39.1% of vulnerability-related IRs only have a few-text information in our manual analysis, and they utilize rich-text information to describe the vulnerabilities, e.g., page screenshots, video streams, music files, code snippets, etc., which implicitly indicate how the vulnerabilities are triggered, thus helping security practitioners identify vulnerability-related IRs and analyze their CWE-IDs. These previous works focus on textual descriptions but lack a comprehensive analysis of IR’s rich-text information, so their practical usage is limited. Among them, we find that over 95% of rich-text IRs utilize page screenshots and code snippets to illustrate how the vulnerabilities are triggered, and their details and contributions are shown as follows: • Page Screenshots: Some developers and users will use page screenshots to display special states during the program’s runtime, thereby reflecting possible vulnerabilities in the system. By capturing the specific element in the page screenshots or the transitions between different pages, we may identify which type of vulnerability the project will encounter. • Code Snippets: Some users will provide the simple Proof-of-Concept (PoC) with code snippets to validate the existence of vulnerabilities, and developers will show the vulnerability-related warnings or bug reports with specific code lines in the projects. To intuitively illustrate how the relationships between these rich-text elements reflect the vulnerabilities, we will introduce them in the following Section 2 . From the previous investigations, we focus on exploring how the vulnerabilities are triggered by analyzing the rich-text elements of the page screenshots and code snippets. However, since some information is implicitly described by IR’s texts and rich-text elements, it is challenging to identify vulnerability and predict the CWE-IDs: Challenge-1: Difficulty in analyzing the triggering process of vulnerability. The first challenge comes from how to accurately understand the textual semantics of the rich-text elements and construct the relationships between rich-text elements that describe the triggering process of vulnerabilities. To address it, we utilize the text understanding and reasoning ability of LLMs to construct the relationships between these rich-text elements. Challenge-2: Factual errors in LLM’s reasoning process. The second challenge comes from the factual errors in the reasoning graphs that are inconsistent with the real-world vulnerability triggering process, mainly because the pre-trained data may have some flaws and be outdated. To address it, we incorporate a module to correct the factual errors in the LLM’s reasoning steps. Challenge-3: Heavy time cost in LLM’s reasoning process. The third challenge comes from heavy time costs. The security practitioners who manage vulnerability identification need to deal with thousands of IRs. If the efficiency of the automated approach is lower than the manual analysis, its value of practical usage is limited. For each target IR that needs to identify vulnerabilities, LLM needs to fully explore all the relationships between page screenshots and code snippets to reason how the pages are redirected and determine the vulnerability’s type. To improve the effectiveness of reasoning, we utilize the LLM’s Retrieval Augmented Generation (RAG) in the reasoning process ( Lewis et al. 2020 ) , where we can retrieve some relevant cases in the history, then use these cases to guide the reasoning of the target IRs ( Guo et al. 2024 ) . In practical analysis, we find that vulnerability-related IRs with the same CWE-ID may have commonalities in describing the vulnerabilities. For example, both Fuel-CMS/issues/536 ( ir_ 2019 ) and PhpldapAdmin/issues/130 ( ir_ 2020 ) belong to the CWE-79, which utilizes the redirection of page screenshots to show how the attackers set XSS payloads to attack the project. We can treat the first IR as a retrieved case for the second one, which incorporates how the vulnerability is triggered and can guide the vulnerability identification of the second IR. In this paper, we propose VulRTex , which is a reasoning-guided approach to identify vulnerabilities from IRs with rich-text information. Specifically, it prepares the Vulnerability Reasoning Database from historical IRs with LLM, which treats the LLM as an agent ( Yao et al. 2023 ) that interacts with external tools to understand the semantic information of screenshots and code snippets, and utilize the reasoning ability of LLM ( Wei et al. 2022 ) to construct

Read the complete text